@austinjt01 Is it haging before the FOG iPXE menu?
If so, lets update ipxe to the latest release. https://forums.fogproject.org/topic/15826/updating-compiling-the-latest-version-of-ipxe
@austinjt01 Is it haging before the FOG iPXE menu?
If so, lets update ipxe to the latest release. https://forums.fogproject.org/topic/15826/updating-compiling-the-latest-version-of-ipxe
@iyoung That is most likely the case, the php-ldap module, before the thread, was a user responsibility to add it in and not provided by the fog installer (if I remember correctly). Once Tom updated the ldap plugin it was logical that if The FOG Project supplied the plugin that the installer should install all required modules.
I intend to document the stuff I learned so far in regards to pxe booting and FOG’s interaction.
Before I get into some of the details of what I’ve learned chatting with Tom and through chats with others like Sebastian, I want to cover some of the basics about pxe booting a target computer into the FOG environment.
PXE booting (known then as diskless booting) has been around in the *nix world since the mid to late 1980s. During that time it was known as network bootstrapping. In 1999 Intel defined a standard called “Wired for Management” which defined the structure that is now known as Pre eXecution Environment (PXE). If I remember correctly on-board PXE code was added as part of PC 2001 design specification by Intel (PC 201 specs also meant the death of the ISA bus). Prior to the PC 2001 specification the PXE boot code was typically added to the network adapter in a PXE boot ROM.
Booting the FOG client OS (I still don’t have a good name for the operating system that runs on the target computer, which does the actual work of imaging the target. For this document it will be known as FOS) requires several services to work in concert. If one actor drops the note the whole concert is ruined. To get the FOS system running the following actors are required. DHCP, TFTP, IPXE, and HTTP.
PXE ROM
When the target computer is powered on and pxe booting is selected, the first thing the PXE code on the target does is attempt to acquire an IP address from the local DHCP server. Once the target has an IP address, it again queries the DHCP server for its environment. This is where the target computer learns about 2 specific and important dhcp options. These important dhcp settings are option 66 (Boot server IP) [in our case the FOG server] and option 67 (Boot file name) [in our case the iPXE network boot firmware]. These two options are passed into the PXE client’s network boot code. The typical network boot code reaches out to device referenced by dhcp option 66 (Boot server IP) using the TFTP (trivial file transfer protocol) to download the file referenced by dhcp option 67 (Boot file name). Once this file is downloaded from the tftp server the pxe boot strap code then chain executes to the file that was downloaded. In the case of the FOS booting, the file that is downloaded is a flavor of iPXE http://ipxe.org/ which is intended to extend the simple capabilities of the built in PXE environment found on the target computer’s NIC rom.
iPXE
The iPXE network boot firmware then reconnects to the device pointed to by dhcp option 66 (i.e. the FOG server) and downloads its configuration file to know what to do next. The configuration file iPXE retrieves (via tftp) is the file default.ipxe. This config file instructs the iPXE network boot firmware to reconnect to the FOG server this time using the http protocol to access the boot.php web page. The iPXE environment also passes the mac address of the target computer to the boot.php page as a parameter. This web page then interacts with the FOG database to constructs the FOG boot menu. In its simplest form the iPXE network boot firmware is an executing operating system with a limited command set. You’ll notice that the PXE boot rom built into the computer will query for a IP address from your dhcp server, and then when the iPXE network boot firmware starts, it will again requrey for an IP address form your dhcp server. And when FOS starts it will again require for an IP address from your dhcp server. It sounds redundant, but each environment is an independent operating system.
FOG Boot Menu
The fog boot menu is created based on the current state of the target computer. If there is a pending job, the target computer is instructed to execute the job. If no job is pending the default FOG menu is displayed. The fog boot menu is constructed of iPXE commands to execute once the IT technician selects an option. For example if you were to look at the iPXE commands for Quick Registration. You would see the iPXE instruction to download the bzImage(32) kernel (the FOS operating system) and execute it with certain parameters as well as the instruction to download the inits(_32).xz (the FOS virtual hard drive). Both of these files will be downloaded to the target computer’s RAM over the http protocol and then executed.
FOS
Once both files required to boot FOS has been transfered to the target computer, the iPXE code chain boots the bzImage (FOS kernel) which begins execution and starts the FOS booting. During the booting process the FOS kernel connects to the downloaded virtual hard drive and starts the init process to boot the entire FOS environment. Again FOS will query your dhcp server for its IP address then execute the commands issued by the FOG Server.
More to come soon
(place holder)
@livet-cir The root of the problem is that the EFI partition comes AFTER the C drive partition. FOG can shrink this disk layout, no problem. The problem is that the EFI partition is not resizable, so FOG can’t expand the C drive because the no resizable EFI is sitting in the way. FOG 1.5.5 is not smart enough to know to just move the EFI partition to the end of the disk then resize C drive.
For FOG 1.5.9 you install in parallel. Install FOG 1.5.9 using git
method. Once FOG 1.5.9 is installed then do this.
Change into the git make fogproject
folder on your FOG. server. Then send these commands as root user.
git pull
git checkout dev-branch
git pull
cd bin
./installfog.sh
That will upgrade your FOG install to 1.5.9.110 or later. 1.5.9.110 should be able to move that EFI partition on the disk. You will need to recapture the golden image with 1.5.9.110 to make resizing working OK.
One last comment. Normally the EFI partition is the first partition on the disk with the C drive next. So normally this isn’t a problem. With Win10 20H1 microsoft puts the recovery partition at the end of the disk behind the C drive partition which case FOG 1.5.9 problems just like you see now.
@Wirefall It sounds like the target computer is picking up an IP address and pxe booting since the FOG iPXE menu is being displayed and you can pick registration. The FOS engine (the customized linux OS that captures and deploys images on the target) is starting up, but its FOS that can’t seem to get an IP address.
This could be a spanning tree issue, if it is I’m a bit surprised that it didn’t show up earlier in the booting process. You can test this by putting a dumb (unmanaged) switch between the booting computer and the building network switch. If the computer boots correctly into registration and imaging then its probably a spanning tree issue.
It could also be a driver issue in the FOS engine. FOS uses linux 4.8.1 (if I remember correcly) so it should support that nic adapter no problem. If you manually register that target computer then schedule a debug capture (create a dummy image definition, assign that dummy image to the host, then schedule a caputure of that host but remember to check the debug option when you schedule it, then finally pxe boot the target computer). The FOS engine will boot and after a few key presses drop you at the FOS command prompt. (I suspect you’ve done this before since you ran the lspci command on the target computer). I would be interested in seeing what the result of ip addr show
is telling us.
This last weekend I created a new production FOG server and I thought I would document how I moved the images off the root partition and onto a disk of their own. Personally I don’t like storing any dynamic or user data on the root partitions. Too many times in the past I’ve see the root partition fill up and the *nix operating system break hard and the only way to repair them is to rebuild the OS. Putting user files on the root partition is basically the same as for a Windows file server putting the user’s group and home drives on the drive.
For this new production server I created a new VM with 3GB of RAM and a 16GB virtual hard drive. For this project a 16GB virtual hard drive is more than sufficient for the OS and the FOG files. But it isn’t big enough for the typical image files. But we’ll fix that a bit later. While I’m going to cover the process for a virtual machine, the process if very similar for a physical server.
So at this point I have a single VM with 1 vCPU 3GB of RAM and 16GB virtual hard drive. The remainder is the process I used to setup this new production server.
yum upgrade -y
from the command promptlsblk
command to show you the drive structure.lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 298.1G 0 disk
├─sda1 8:1 0 294.3G 0 part /
├─sda2 8:2 0 1K 0 part
└─sda5 8:5 0 3.8G 0 part [SWAP]
sdb 8:16 0 100.0G 0 disk
Now that we know our 100GB disk is sdb lets connect get down to business prepping the disk for our FOG server.
6) We’ll use LVM for this volume, but lets first create a partition on this disk that fills the entire disk.
fdisk /dev/sdb
n
p
<enter 3 time>
w
pvcreate /dev/sdb1
vgcreate fog_disk /dev/sdb1
lvcreate -l 100%FREE -n fog_files fog_disk
mkfs.ext4 /dev/fog_disk/fog_files
mount -a
df -h
command to show you the mounted devices. You should see a line in the df output that looks similar to this:mv /images /opt/fog
mkdir /images
ls -la /images
you should not have any files in that directory (hint: we just created it).ls -la /opt/fog/images
you should see at least 2 directories (/opt/fog/images/dev and /opt/fog/images/postdownloadscripts). If these commands give us what we need then we have everything in place to do the magic of the last step.mount -a
ls -la /images
You should see the same content as if you listed the /opt/fog/images directory.#ls -la /images
total 40
drwxrwxrwx 10 root root 4096 Feb 7 09:55 .
dr-xr-xr-x. 20 root root 4096 Feb 7 09:45 ..
drwxrwxrwx 2 root root 4096 Oct 14 14:45 dev
-rwxrwxrwx 1 root root 0 Feb 5 18:57 .mntcheck
drwxrwxrwx 3 root root 4096 Jan 21 08:54 postdownloadscripts
ls -la /images
command to confirm you still have files in that location.showmount -e 127.0.0.1
Export list for 127.0.0.1:
/images/dev *
/images *
One thing I did not comment on above, by setting up a new vmdk file for the /opt directory, we can expand the vmdk file as needed as our captured images increase. I’ll follow up a bit later on how to add space to your /opt partition with just a few commands.
@travi OK what we want you to do is this.
uname -a
lspci -nn | grep -i net
grep -i -e "firmware" /var/log/syslog
Take a clear picture of the outputs and post it here.
@Tom-Elliott said in FOG will not boot - "Failed to get an IP via DHCP!:
The “tftp: client does not accept options.” message is a misnomer. It doesn’t mean anything in regards to the problem(s) you’re seeing.
I agree, that tftp message is just a warning message.
We’ve seen a similar issue before with realtek nics and green ethernet/power saving/802.3az being enabled on the switch. But this is the first time seeing an intel nic do this.
Part 1a
This is more of a proof of concept than as actual desire of mine. Keeping everything in the *nix world is my motto.
With that said. I’ve been able to setup nfs on a windows 2012 R2 server with the following powershell commands.
Import-Module ServerManager
Add-WindowsFeature FS-NFS-Service
Import-Module NFS
mkdir c:\share
mkdir c:\share\images
mkdir c:\share\tftpboot
mkdir c:\share\snapins
mkdir c:\share\snapins\ssl
New-NfsShare –Name "images" –Path c:\share\images –Authentication sys -AllowRootAccess $True -EnableUnmappedAccess $True –Permission Readwrite
Enable-NetFirewallRule -DisplayGroup “Server for NFS” -Verbose
And then on the FOG Master server the following commands will setup the rest of the fog required bits.
mount -t nfs <win_storage-node_ip>:/images /mnt
mkdir /mnt/dev
touch /mnt/.mntcheck
touch /mnt/dev/.mntcheck
umount /mnt
Create a local FTP user for FOG to use.
net localgroup fog_users /add
net user fog_user "mi5ty_cl0ud" /add /EXPIRES:NEVER /PASSWORDCHG:NO /active:YES /Y
net localgroup fog_users fog_user /add
icacls c:\share /grant "fog_users:M"
Setup the FTP server
Install-WindowsFeature Web-FTP-Server,Web-FTP-Service,Web-FTP-Ext -IncludeManagementTools
New-WebFtpSite -Name "FOGFtpSite" -Port 21 -PhysicalPath "c:\share" -IPAddress "<win_storage-node_ip>"
Set-ItemProperty "IIS:\Sites\FOGFtpSite" -Name ftpServer.security.ssl.controlChannelPolicy -Value 0
Set-ItemProperty "IIS:\Sites\FOGFtpSite" -Name ftpServer.security.ssl.dataChannelPolicy -Value 0
Set-ItemProperty "IIS:\Sites\FOGFtpSite" -Name ftpServer.security.authentication.basicAuthentication.enabled -Value $true
Set-ItemProperty "IIS:\Sites\FOGFtpSite" -Name ftpserver.userisolation.mode -Value 4
Add-WebConfiguration "/system.ftpServer/security/authorization" -value @{accessType="Allow";roles="fog_users";permissions="Read,Write";users=""} -PSPath IIS:\ -location "FOGFtpSite"
Restart-WebItem "IIS:\Sites\FOGFtpSite"
The next part is the web server setup to hand out the FOS kernel and inits. Since I already installed the FTP service which relies on IIS, IIS is already installed. So all we need to do is prep for the FOS files.
Lets first create the directory structure that mimics the path on the FOG master server.
New-Item "IIS:\Sites\Default Web Site\fog" -type Directory
New-Item "IIS:\Sites\Default Web Site\fog\service" -type Directory
New-Item "IIS:\Sites\Default Web Site\fog\service\ipxe" -type Directory
Now that we have the directory, we need to tell IIS to hand out any file that is requested. By default IIS will only pass out files with known extensions like htm, html, asp, and so on. But in our case we want IIS to hand out the inits that end in .xz and the bzImage which doesn’t have an extension. To do this we need to tell IIS to just hand out any file type request (the bzImage is what caused me some pain here, since we are saying any files requested from IIS it will hand out, which could be a security risk if the web site gets hacked)
(Update: 15-May-2017)
You must go manually into IIS management console and add a new mime type of “.*” (dot star without the quotes) and with a type of “application/octet-stream”
There is a better way to configure IIS to send out files with any extension (even no extension) on a per directory basis.
Create the following file: C:\inetpub\wwwroot\fog\service\ipxe\web.config
Insert the following code into that web.config file
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<staticContent>
<mimeMap fileExtension="." mimeType="application/octet-stream" />
<mimeMap fileExtension=".*" mimeType="application/octet-stream" />
</staticContent>
</system.webServer>
</configuration>
(End Update: 15-May-2017)
Now that IIS is all setup and ready you will need to copy all of the files from /var/www/html/fog/service/ipxe to the IIS server in the windows path IIS:\Sites\Default Web Site\fog\services\ipxe
You can use the following process to copy the files form your master fog server to the windows storage node.
In an elevated windows command prompt enter
nfsshare fogipxe=C:\inetpub\wwwroot\fog\service\ipxe -o rw sec=sys root unmapped=yes
On your fog server key in:
mount -t nfs <win_storage-node_ip>:/fogipxe /mnt
cp /var/www/html/fog/service/ipxe/* /mnt
umount /mnt
And then finally back on the Windows storage node
nfsshare fogipxe /delete
(Update: 15-May-2017)
The files copied from the fog server seemed to come in with the wrong permissions so we need to reset the permissions on all files from the fog directory and below to the defaults by enabling inheritance on all of the files.
the following command is wrong, its a place holder until the right syntax can be derived
icacls C:\inetpub\wwwroot\fog /grant "fog_users:M"
(End Update: 15-May-2017)
After you’ve copied the files to the correct directory on IIS you should test your setup.
First lets pull the FOG background. Open your browser and key in the url to the background. On my IIS server I’ll use this path, you will need to change the IP address to match your IIS server.
http://<fog_server_ip>/fog/service/ipxe/bg.png
If all goes well you should see the picture of the fog background.
Now lets get a little daring. Lets pull memdisk (a binary file).
http://<fog_server_ip>/fog/service/ipxe/memdisk
If all goes well you should be prompted with a save file dialog.
And then one last test, lets pull a file with an unknown extension.
http://<fog_server_ip>/fog/service/ipxe/refind.efi
Again you should be prompted with a save file dialog.
Onto the next part. For this section we need to install a tftp server to allow pxe booting from your windows storage node. Windows does have a natively built in tftp client, but no tftp server. So for this part we will use an freeware tftp server that I’ve used for years (Tftpd32).
Go to the following URL: http://tftpd32.jounin.net/tftpd32_download.html
Download the tftpd64 service edition (installer)
Launch the installer you just downloaded.
Read and agree to EULA if you accept it continue.
Select (all) Options: Add start menu shortcuts, Add desktop icon, Start service Tftp32_svc, start service monitoring
Use default install location: C:\Program Files\Tftpd64_SE
Tftpd64 Service console should launch
Select the Settings button
Select the GLOBAL tab
Uncheck all options except TFTP Sever. The only selection option we need is “TFTP Server”.
Select the TFTP tab
For the base directory, select the browse button and then navigate to the c:\share\tftpboot folder
Select OK
In the tftp options section enable PXE Compatibility option. Leave all other settings at their default
Press OK
From a command windows with elevated rights
netsh advfirewall firewall add rule name="TFTP Server" dir=in action=allow program="%ProgramFiles%\Tftpd64_SE\tftpd64_svc.exe"
sc stop Tftpd32_svc
sc start Tftpd32_svc
The remaining steps are to copy the contents of the FOG server’s /tftpboot directory to your Windows storage node’s c:\share\tftpboot folder. For this we’ll use NFS to export the directory on the FOG server and then mount that nfs share with your windows server.
In an elevated windows server console key in
nfsshare fogpxe=C:\share\tftpboot -o rw sec=sys root unmapped=yes
On the FOG server
mount -t nfs 192.168.1.205:/fogpxe /mnt
cp -R /tftpboot/* /mnt
umount /mnt
On the Windows storage node from an elevated command prompt
nfsshare fogpxe /delete
If you can make it this far in the setup your storage node should be setup.
Todo list:
@travi Well that is good (and bad) you posted the wrong picture because it should have been working on that old network adapter.
@Sebastian-Roth Good catch on the system swap.
Network adapter [8086:15fb] first appeared in linux kernel 5.5. So the current kernels offered by FOG should work.
From your latest picture it appears that FOS Linux is seeing the network adapter because it has a device name [enp0s31f6]. But ip a s
is not giving us an ip address.
I still don’t understand why in your initial picture you are getting “No network interface found” and yet we see it listed in ip a s
. It not having an ip address at the moment, we can test for.
At the fos linux prompt where its telling you it had the network driver but no IP address. If you are booting this fresh get to the fos linux command prompt and then wait 30 seconds then run this command: /sbin/udhcpc -i enp0s31f6 --now
and see if it picks up an IP address. Lets see if time fixes what is wrong.
@Wirefall I think our next step is to get the target computer to boot into a debug capture console to see what is going on. If a dumb switch with no fancy support doesn’t work I think a hub would be a waste of time (unless you had one under your desk already).
@francois Have you checked to see if you have the latest firmware on these troubled Fujitsu computers?
I agree that 944 KB/min is terrible. Who has time for that? (joke).
I would first start by making sure firmware is up to date. Then we can debug more to see if its network or slow disk making imaging slow for you.
Is this a uefi or bios system?
Did you disable secure boot?
The error message is roughly about starting the bzImage on this computer
What precisely do you have for dhcp option 67? I’ve seen a similar error message when and older pxelinux.0 is used for some reason to pxe boot.
Are you running a dnsmasq server or just a dhcp server?
@jonathands This in theory should be working. I do see something that make me question.
First let me say in the ltsp.conf file these are the only rules that apply since its a bios computer.
# The boot filename, Server name, Server Ip Address
dhcp-boot=undionly.kpxe,,192.168.1.250
Looking at your picture I can see you are pxe booting a bios based computer. The computer is getting an ip address of 192.168.0.112 with a subnet mask of 255.255.255.0. Your dhcp server and gateway are one device (maybe a soho/home router). The proxy dhcp address is correct according to the ltsp.conf file, so that tells us DNSMASQ is talking to the target computer. So that bit is working.
Now to the point I’m seeing. The FOG server (I assume is also where dnsmasq is running) is on 192.168.1.250. That is a different subnet than where the computer is at 192.168.0.112. Do you have a router configured to route between the two subnets?
First let me explain that I don’t use fog snapins so this may be a bit off point.
But the FOG Client runs as SYSTEM (unless you changed the default). The SYSTEM account has no rights outside of the box that the FOG Client is running on. Even if another computer has a SYSTEM account, that account is different and not a domain account.
To make your script work, it must run as, or connect to the remote share as someone who has rights on the remote share. If it was me I’d use a command like.
net use t: \\server1\share1 /user:domain\bob
and then supply the password for domain\bob
to connect to the remote share to retrieve the file.
Then when your done issue the command net use t: /delete
to remove the share.
@Tom-Elliott Needing the 4.9.x linux kernel with the patch that Sebastian found? Its the same network adapter if I remember correctly. The system is getting to stage 3 of the booting process (FOS) and failing to get an ip address??