iPXE Setup For Many OS's Under BIOS and UEFI

Zer0Cool

Note: This is a Work in Progress, please bear with me while I complete the guide 6/15/2018

Base Setup

FOG version: 1.5.3
OS: CentOS 7.5.1804
FOG IP: 10.0.0.2

FOG acts as DHCP, all machines are on the same subnet as FOG, additionally installed Samba.

Goals of this Guide

My aim in this guide is to provide the steps needed to successfully PXE boot and install many OS’s using FOG and the underlying iPXE boot loader.

Most of this guide will be providing the parameters to enter in the FOG web GUI for iPXE entries. That would be either FOG Configuration | iPXE New Menu Entry or if editing an entry that exists iPXE Menu Item Settings.

This guide will make several assumptions. First, you have already installed FOG and hosts get an IP via DHCP and boot to the menu. The guide applies to the versions, base OS and other mentions above in Base Setup. Its also assumed that firewall is setup and SELinux is in permissive mode.

This guide is heavy influenced by this post here: https://forums.fogproject.org/topic/10944/using-fog-to-pxe-boot-into-your-favorite-installer-images and I will be cross linking to it at relevant points throughout this guide. Thanks to @george1421 for his post, help and insights. I also discussed this post with him prior to making my own.

Initial Setup

I will first cover the basic setup I am using to facilitate this and then provide a post for each OS I have setup and tested. My testing was done on a physical Dell workstation, a Dell rackmount server and inside an ESXi 6.7 VM all using both BIOS and UEFI booting (when possible).

First I installed Samba using:

sudo yum install samba

I created the following directories on the FOG server:

mkdir /pxeshare
mkdir /tftpboot/os
mkdir /mnt/iso

I also made a sub directory under /tftpboot/os/ for each OS I intend to make available for PXE booting. IE: /tftpboot/os/centos, /tftpboot/os/win/7, /tftpboot/os/win/10, /tftpboot/os/fedora, etc.

I then created a symlink to the /tftpboot/os directory in two places. The commands are below:

ln -s /tftpboot/os /pxeshare/os
ln -s /tftpboot/os /var/www/html/os

The symlinks will make the OS install files in /tftpboot/os available via http and smb as well as via tftp. My experience has been that http is much faster than tftp and smb will be needed for PXE installing Windows OS’s. Feel free in the OS directions iPXE parameters to change http:// to tftp:// or whatever protocol you prefer.

A basic setup for samba in /etc/samaba/smb.conf would look like:

[global]
unix extensions = no
[pxeshare]
comment = FOG server SMB Share
path = /pxeshare
valid users = @smbgrp
guest ok = No
browsable = Yes
writable = Yes
follow symlinks = Yes
wide links = Yes

I have created a user on my FOG server strictly for SMB without the ability to login to the server (based on info found here). I have also added my user(s) on the FOG server to the smbgrp I will have setup as follows (replace ‘user’ with your desired username).

Basic setup for existing users (like the “user” you use to manage the server):

groupadd smbgrp
usermod user -aG smbgrp
smbpasswd -a user

chmod -R 0770 /pxeshare
chown -R user:smbgrp /pxeshare
chcon -R -t samba_share_t /pxeshare

Setting up a Samba user on FOG server without a home folder and with no login ability:

adduser --no-create-home --shell /sbin/nologin --user-group fogpxeu
passwd fogpxeu

The above creates the user and the password in Linux but not the smb password, which is what will be used and will be visible to anyone PXE booting Windows. To set the SMB credentials repeat the usermod and smbpasswd steps above for the fogpxeu user (or whatever name you want to give this user).

The end result is we can mount this SMB share on any machine connected to the network using any user credentials that have been added to the smbgrp and given a password for SMB. Further, we have a user that we will use for Windows PXE booting to connect to the SMB share and pull the files that cannot login to the FOG server directly.

Conclusion before Individual OS setup

At this stage we should have laid all the groundwork to make PXE booting using HTTP and SMB possible using iPXE. Below is a list of the tested OS’s that link to the directions for each (let me know if you would like me to add other OS’s):

Install

Run

Gparted 0.31.0
Dban 2.3.0
Clonezilla 2.5.5-38

Setup

WinPE

Note: As of this posting, the iPXE menu entries created in FOG are listed in the order of creation (top to bottom in the menu) and cannot be easily reorganized. Plan ahead and create them in the order you want them to appear on hosts when PXE booting!

Other Resources:

Creating a single iPXE menu entry that works with UEFI and BIOS found in my guide here. I use this method for entries like ESXi in which UEFI and BIOS entries need different parameters. This method allows one iPXE menu entry to behave one way if BIOS booting or another if UEFI booting, passing it different commands for either.
Batch file for generating WinPE images with networking/smb connection here. I have altered the batch file a bit as detailed in Windows/WinPE directions.

Zer0Cool

1
Reserved for updates

Zer0Cool

2
Reserved for updates

Zer0Cool

3
Reserved for updates

Zer0Cool

4
Reserved for updates

Zer0Cool

5
Reserved for updates

Zer0Cool

6
Reserved for updates

Zer0Cool

7
Reserved for updates

Zer0Cool

8
Reserved for updates

Zer0Cool

9
Reserved for updates

Zer0Cool

Fedora Workstation 28

Works with: UEFI or BIOS

To start we need the installation files in our /tftpboot/os/fedora/ws28 directory. I was unable to find an ISO with the proper file structure to allow instllation, so instead I did an rsync to get the files needed. I will provide the directions for using the ISO in case it works for others. A list of official Fedora mirrors can be found here. The following presumes Fedora Workstation 28 but could apply to other versions and spins.

You do not have to do the steps in BOTH “Using Files from ISO” and “From rsync”, its one or the other.

Using Files from ISO:

Create the directory for fedora if it doesnt already exist:

mkdir -p /tftpboot/os/fedora/ws28

Mount the ISO and copy the required files to the above directory:

mount /{path to the iso}/filename.iso /mnt/iso/
cp -R /mnt/iso/* /tftpboot/os/fedora/ws28
umount /mnt/iso

Using rsync:

Create the directory for fedora if it doesnt already exist:

mkdir -p /tftpboot/os/fedora/ws28

run the rsync command using a rsync url from the list of mirrors you want to use. This can take hours to run:

rsync -avzP --log-file=/var/log/rsync_cron/rsync_fedora_ws28_mirror.log --delete rsync://mirrorurl/pub/fedora/linux/releases/28/Workstation/x86_64/os/ /tftpboot/os/fedora/ws28

Change the rsync path to your selected rsync mirrors path and adjust the lof path/filename as desired. I placed the rsync command above in my crontab -e without the -P switch for rsync and have it sync once a day. Doing so is a bit beyond the context of this guide though.

Also note that if the mirror is down the command can fail, you may need to switch to another mirror or if setting up cron having it sync multiple mirrors just in case.

Adding iPXE entry:

From the FOG web GUI, create a new iPXE entry (or edit an existing one) as follows:

Menu Item: name as desired (ex: os.fedoraws)
Description: as desired (ex: Fedora Workstation 28 - Install). Note: This is what is seen in the iPXE menu on the host machine!
Parameters:

kernel http://${fog-ip}/os/fedora/ws28/images/pxeboot/vmlinuz
initrd http://${fog-ip}/os/fedora/ws28/images/pxeboot/initrd.img
imgargs vmlinuz initrd=initrd.img repo=http://${fog-ip}/os/fedora/ws28
boot || goto MENU

Menu Show With: as desired.
Save changes

Alternate directions here.

Zer0Cool

This post is deleted!

Zer0Cool

WinPE

Preparing Windows for PXE

Windows requires some additional setup to PXE boot. First off it will load WinPE which in turn runs the setup of the desired Windows version. This part of the guide will detail how I do this, what adjustments you may consider and I will provide other links to supporting information.

When WinPE boots, we have it run startnet.cmd which is a batch file in the WinPE image containing commands to connect to our Samba share and get the instillation files to run the install. At least thats the basic description, it can be scripted to do much more.

The first thing for us to do is create a custom WinPE image. I use a batch file to do this. Following is the overall setup required to do this.

Requirements

Windows machine (I use a Server 2016 box, you may use Windows 10 or possibly other versions).
Windows ADK installed (Official link here). Usually the latest available, 1803 as of this post.
WinPE10 Driver Pack (Official link here). More info found here. Despite being from Dell, they are generically useful it seems. Feel free to check with your OEM to see if they have their own packs. My guide presumes this file however.

I am not going to cover the installation of ADK, as many sources including some links I have provided detail it. After installation dism should be usable from the command line on the Windows machine.

Whatever media you want to use to PXE install Windows, you will want to copy the iso contents including install.wim and boot.wim to the Windows machine we are prepping from to get information about it. Run the following command on the install.wim file:

Dism /Get-WIMInfo /WimFile:C:\pathtowinfiles\sources\install.wim
Dism /Get-WIMInfo /WimFile:C:\pathtowinfiles\sources\boot.wim

replacing C: and pathtowinfiles with your drive and path to the root of the ISO contents. This command will list all of the images in the wim file along with index number(s) which we will need in our script to point to the correct install. You could for example have an ISO for Windows 10 Pro x64 with the ability to install many different versions of Windows 10. Its possible the one you want has the index number 1 or 5 or 7, etc. This command lists them all and lets us find out which index number to use later.

Generating WinPE Image with Batch File

My batch file to generate the WinPE image is based on the post found here (Thanks to @Quazz). I have adjusted it to meet my needs, which may or may not benefit you. I took out the x86/x64 portion and assume x64. I also added a portion to check if the destination folder exists and to delete it if it does prior to running. I also added a loop mechanism to the mounting of the share in case it fails (as for me it 99% of the time eventually connects). I also added a line to inject the WinPE10 drivers from the driver pack.

@echo off

rem The fileserver IP
set FILESERVER=10.0.0.2

rem Share on the fileserver.
set SHARE=pxeshare\os\win\10

rem Username for the share
set SHAREUSER=fogpxeu

rem Password for the share
set SHAREPASS=yourpassword

rem amd64 or x86
set ARCH64=amd64

rem Path to hold working files. Needs about 500MB of free space.
set PEPATH64="D:\winpe\%ARCH64%"

rem ##########################################################
rem     Don't edit anything below here
rem ##########################################################

rem *********** x64 ***********************

echo Remove old amd64 directory
IF EXIST %PEPATH64% rmdir /S /Q %PEPATH64%

echo Creating the PE image
call copype.cmd %ARCH64% %PEPATH64% > NUL

echo Mounting the image
dism /Mount-Wim /WimFile:%PEPATH64%\media\sources\boot.wim /index:1 /MountDir:%PEPATH64%\mount /quiet

echo Adding commands to the startup script in PE
echo. >> %PEPATH64%\mount\windows\system32\startnet.cmd

echo :loop >> %PEPATH64%\mount\windows\system32\startnet.cmd
echo ping %FILESERVER% >> %PEPATH64%\mount\windows\system32\startnet.cmd
echo net use i: \\%FILESERVER%\%SHARE% /user:%SHAREUSER% %SHAREPASS% ^|^| goto :loop >> %PEPATH64%\mount\windows\system32\startnet.cmd

echo i: >> %PEPATH64%\mount\windows\system32\startnet.cmd

echo i:\setup.exe >> %PEPATH64%\mount\windows\system32\startnet.cmd

echo Creating the pxeboot directory
mkdir %PEPATH64%\pxeboot > NUL
mkdir %PEPATH64%\pxeboot\Fonts > NUL
copy /y %PEPATH64%\mount\windows\boot\Fonts\*.* %PEPATH64%\pxeboot\Fonts\ > NUL
copy /y "%WinPERoot%\%ARCH64%\Media\Boot\boot.sdi" %PEPATH64%\pxeboot\ > NUL
copy /y "%WinPERoot%\%ARCH64%\Media\Boot\BCD" %PEPATH64%\pxeboot\ > NUL

echo Adding WinPE Driver Pack
dism /add-driver /image:D:\winpe\amd64\mount /driver:C:\Users\user\Downloads\winpe_3620\x64 /recurse /forceunsigned

echo Unmounting the image
dism /unmount-Wim /MountDir:%PEPATH64%\mount /Commit /quiet

echo Optimizing the image
imagex /EXPORT %PEPATH64%\media\sources\boot.wim 1 %PEPATH64%\pxeboot\boot.wim > NUL

echo.
echo All the files you need for your PXE server are in: %PEPATH64%\pxeboot\

The script should have its variables adjusted as needed, including IP, share path (remember this is a Samba share on the FOG server), and smb user and password. Change the index # in the script to match the index number discovered with the dism commands earlier in this post. Adjust any paths to match the drive and path you have on your machine to the needed contents.

Warning!!

Be aware that the username AND password for that user for Samba will be visible when PXE booting on the screen. This is why I do not use a user that can login to the server. Its a limited account I use only for connecting to the share from WInPE and I dont care if others PXE booting know the password as the only thing they can do with it is connect to the share. If anyone knows of a way to prevent the credentials from appearing when startnet.cmd runs let me know.

I run this batch file using the following command:

cmd /k winpe.bat

This will run it and leave the command window open until complete, making it easier to follow the process and troubleshoot issues.

Presuming it completes as intended, the result will be a folder called amd64. I simply copy this whole folder to the /tftpboot/os/win/10 directory on my FOG server. Many of the files we reference in the iPXE menu entry will be within this folder (and its subfolders). I also copy the boot.wim over as it has had the drivers from the driver pack injected.

I repeat this batch file for each version of Windows I want to install via PXE, editing the batch with correct paths, files, etc for that version.

Zer0Cool

Windows 7, Windows 10, Windows Server 2012 R2, Windows Server 2016

Works with: UEFI or BIOS (Windows 7 only with BIOS in my testing, hypothetically could be modified to UEFI boot))

To start we need the installation files in our /tftpboot/os/win directory. This can be done by manually copying the required content from a Windows iso. I will be providing directions for Windows 10 Pro x64, but the same general steps apply to all versions of Windows mentioned in this posts “title” and likely other versions as well. I had only tested x64 versions for myself.

Using Files from ISO:

Create the directory for Windows/version if it doesnt already exist:

mkdir -p /tftpboot/os/win/10

Mount the ISO and copy the required files to the above directory:

mount /{path to the iso}/filename.iso /mnt/iso/
cp -R /mnt/iso/* /tftpboot/os/win/10
umount /mnt/iso

Preparing Windows for PXE:

See the post on WinPE. It is assumed going forward that you have a working WinPE image that connects to the Samba share setup in the first post, correctly pointed to the version of Windows to install and with the proper drivers injected to run.

Adding iPXE entry:

From the FOG web GUI, create a new iPXE entry (or edit an existing one) as follows:

Menu Item: name as desired (ex: os.winlatest)
Description: as desired (ex: Windows 10 Pro - Install). Note: This is what is seen in the iPXE menu on the host machine!
Parameters:

iseq ${platform} efi && goto win10_efi || goto win10_bios
:win10_efi
kernel http://${fog-ip}/os/win/wimboot gui
initrd --name segmono_boot.ttf http://${fog-ip}/os/win/10/amd64/media/Boot/Fonts/segmono_boot.ttf segmono_boot.ttf
initrd --name segoe_slboot.ttf http://${fog-ip}/os/win/10/amd64/media/Boot/Fonts/segoe_slboot.ttf segoe_slboot.ttf
initrd --name segoen_slboot.ttf http://${fog-ip}/os/win/10/amd64/media/Boot/Fonts/segoen_slboot.ttf segoen_slboot.ttf
initrd --name wgl4_boot.ttf http://${fog-ip}/os/win/10/amd64/media/Boot/Fonts/wgl4_boot.ttf wgl4_boot.ttf
initrd --name BCD http://${fog-ip}/os/win/10/amd64/media/EFI/Microsoft/Boot/BCD BCD
initrd --name boot.sdi http://${fog-ip}/os/win/10/amd64/media/Boot/boot.sdi boot.sdi
initrd --name boot.wim http://${fog-ip}/os/win/10/amd64/media/sources/boot.wim boot.wim
boot || goto MENU
:win10_bios
kernel http://${fog-ip}/os/win/wimboot gui
initrd --name segmono_boot.ttf http://${fog-ip}/os/win/10/amd64/media/Boot/Fonts/segmono_boot.ttf segmono_boot.ttf
initrd --name segoe_slboot.ttf http://${fog-ip}/os/win/10/amd64/media/Boot/Fonts/segoe_slboot.ttf segoe_slboot.ttf
initrd --name segoen_slboot.ttf http://${fog-ip}/os/win/10/amd64/media/Boot/Fonts/segoen_slboot.ttf segoen_slboot.ttf
initrd --name wgl4_boot.ttf http://${fog-ip}/os/win/10/amd64/media/Boot/Fonts/wgl4_boot.ttf wgl4_boot.ttf
initrd --name BCD http://${fog-ip}/os/win/10/amd64/media/Boot/BCD BCD
initrd --name boot.sdi http://${fog-ip}/os/win/10/amd64/media/Boot/boot.sdi boot.sdi
initrd --name boot.wim http://${fog-ip}/os/win/10/amd64/media/sources/boot.wim boot.wim
boot || goto MENU

Note: for something like Windows 7 that may not be able to UEFI boot, you can do something like (for the efi label, similar to ESXi examples):

:win7_efi
prompt --timeout 30000 Host must be BIOS booted for this option. Press any key to return to the menu... && goto MENU || goto MENU

Menu Show With: as desired.
Save changes

Alternate directions here.

Zer0Cool

ESXi 6.5 U1 and 6.7

Works with: UEFI (BIOS booting is only possible if you compile your own undionly.kpxe iPXE kernel to include the IMG_COMBOOT parameter. FOG does not ship with an iPXE kernel supporting this. I have not tested this and compiling is outside the scope of this guide)

To start we need the installation files in our /tftpboot/os/esxi directory. This can be done by manually copying the required content from an ESXi iso. The following will presume ESXi 6.7 but altering the version to your desired version in the steps should work.

Using Files from ISO:

Create the directory for CentOS/version if it doesnt already exist:

mkdir -p /tftpboot/os/esxi/6.7

Mount the ISO and copy the required files to the above directory:

mount /{path to the iso}/filename.iso /mnt/iso/
cp -R /mnt/iso/* /tftpboot/os/esxi/6.7
umount /mnt/iso

Preparing ESXi Media for PXE

ESXi is a bit unique in that we need to modify some of its files to work with PXE booting. This seems to be one of the bigger areas of confusion for those trying to setup iPXE to boot ESXi.

The file to modify is boot.cfg and ESXi has 2 of these files, one is used for BIOS booting and the other for UEFI booting. We are only concerned with the UEFI version of the file, but it may be worthwhile to do both to avoid issues or if you do want to compile your own kernel to support BIOS installs.

The UEFI version of the file is located at /tftpboot/os/esxi/6.7/efi/boot/boot.cfg

I will be editing it with vi/vim but you can use whatever text editor you like:

First we need to remove all of the / characters from the boot.cfg file using:

sed -i.org 's&/&&g' /tftpboot/os/esxi/6.7/efi/boot/boot.cfg

Open the file to edit:

vi /tftpboot/os/esxi/6.7/efi/boot/boot.cfg

add the prefix line between timeout and kernel (change FOG-IP to your FOG servers IP address, ie: 10.0.0.2…hostname may work too):

prefix=http://FOG-IP/os/esxi/6.7/

change the kernelopt line to:

kernelopt=runweasel

Save and close the boot.cfg

Adding iPXE entry:

From the FOG web GUI, create a new iPXE entry (or edit an existing one) as follows:

Menu Item: name as desired (ex: os.esxilatest)
Description: as desired (ex: ESXi 6.7 A01 - Install (UEFI Only). Note: This is what is seen in the iPXE menu on the host machine!
Parameters:

Supporting only UEFI:

iseq ${platform} efi && goto esxi67_efi || goto esxi67_bios
:esxi67_efi
kernel http://${fog-ip}/os/esxi/6.7/efi/boot/bootx64.efi -c http://${fog-ip}/os/esxi/6.7/efi/boot/boot.cfg
boot || goto MENU
:esxi67_bios
prompt --timeout 30000 Host must be UEFI booted for this option. Press any key to return to the menu... && goto MENU || goto MENU

Supporting BIOS and UEFI (assuming you compiled your own kernel for it to work and edited both the /tftpboot/os/esxi/6.7/efi/boot/boot.cfg and /tftpboot/os/esxi/6.7/boot.cfg files as per the above steps):

iseq ${platform} efi && goto esxi67_efi || goto esxi67_bios
:esxi67_efi
kernel http://${fog-ip}/os/esxi/6.7/efi/boot/bootx64.efi -c http://${fog-ip}/os/esxi/6.7/efi/boot/boot.cfg
boot || goto MENU
:esxi67_bios
kernel http://${fog-ip}/os/esxi/6.7/mboot.c32 -c http://${fog-ip}/os/esxi/6.7/boot.cfg
boot || goto MENU

Menu Show With: as desired.
Save changes

Alternate directions here.

Zer0Cool

CentOS 7.4 and 7.5

Works with: UEFI or BIOS

To start we need the installation files in our /tftpboot/os/centos directory. This can be done by manually copying the required content from a CentOS iso or by using rsync to sync from an online mirror. A list of official mirrors with rsync links can be found here. The following will presume the latest version of CentOS (at this time 7.5.1804) but altering the version to your desired version in the steps should work.

You do not have to do the steps in BOTH “Using Files from ISO” and “From rsync”, its one or the other.

Using Files from ISO:

Create the directory for CentOS/version if it doesnt already exist:

mkdir -p /tftpboot/os/centos/7.5.1804

Mount the ISO and copy the required files to the above directory:

mount /{path to the iso}/filename.iso /mnt/iso/
cp -R /mnt/iso/* /tftpboot/os/centos/7.5.1804
umount /mnt/iso

Using rsync:

Do not create the subdirectory for the CentOS version, just have /tftpboot/os/centos ready, rsync will create the subdirectory.
run the rsync command using a rsync url from the list of mirrors you want to use. This can take hours to run:

rsync -avzP --log-file=/var/log/rsync_cron/rsync_yum_mirror.log --delete --include="/7.5.1804" --include="/7.5.1804/**" --exclude="*" rsync://mirrorurl/centos/7.5.1804/os/x86_64/ /tftpboot/os/centos

feel free to change the path/name of the log file. I use this rsync to also pull updates allowing me to use my FOG server as a local YUM repo as well. If interested in doing so change the rsync path to be rsync://mirrorurl/centos/ instead. This will make the amount of data downloaded much greater, but will later let you use the FOG server as a YUM repo. I placed the rsync command above in my crontab -e without the -P switch for rsync and have it sync once a day. Doing so is a bit beyond the context of this guide though.

Also note that if the mirror is down the command can fail, you may need to switch to another mirror or if setting up cron having it sync multiple mirrors just in case.

Adding iPXE entry:

From the FOG web GUI, create a new iPXE entry (or edit an existing one) as follows:

Menu Item: name as desired (ex: os.centoslatest)
Description: as desired (ex: CentOS 7.5.1804 - Install). Note: This is what is seen in the iPXE menu on the host machine!
Parameters:

Manual or rsync without wanting YUM server too:

kernel http://${fog-ip}/os/centos/7.5.1804/images/pxeboot/vmlinuz
initrd http://${fog-ip}/os/centos/7.5.1804/images/pxeboot/initrd.img
imgargs vmlinuz initrd=initrd.img root=live:http://${fog-ip}/os/centos/7.5.1804/LiveOS/squashfs.img repo=http://${fog-ip}/os/centos/7.5.1804
boot || goto MENU

With rsync pulling all data for YUM:

kernel http://${fog-ip}/os/centos/7.5.1804/os/x86_64/images/pxeboot/vmlinuz
initrd http://${fog-ip}/os/centos/7.5.1804/os/x86_64/images/pxeboot/initrd.img
imgargs vmlinuz initrd=initrd.img root=live:http://${fog-ip}/os/centos/7.5.1804/os/x86_64/LiveOS/squashfs.img repo=http://${fog-ip}/os/centos/7.5.1804/os/x86_64
boot || goto MENU

Menu Show With: as desired.
Save changes

Alternate directions here.

iPXE Setup For Many OS's Under BIOS and UEFI

225

11.6k

17.1k

154.5k