@jape said in FOG unable to PXE boot beyond the VLAN/subnet that the server is on:

The Fog server is a Ubuntu 18 system it is also the DHCPD server .I have not changed the option 66 or 67.

OK now we have a direction. So can you tell me why you are using the FOG server as a dhcp server? Do you have a campus dhcp server or is FOG on a dedicated imaging network.

The question is not as cheeky as it sounds. There are valid use cases for having FOG be the dhcp server, I just want to make sure you have one of those cases.

@rogalskij Using a windows computer, install the tftp client feature. Turn off the windows firewall then open a command window to see if you can tftp get the file ipxe.efi. I’m only interested if it downloads the file or not.

@lcis said in Trying to access /images on deploy-debug:

For curiosity, why couldn’t I mount $storage ?

While debugging your post install script, use debugPause to put a break point in your code. Put an echo statement in just before the debugPause to say “Stop me here”. So when you are debug executing your deployment, when you get to “Stop me here” press ctrl-C to exit the fog script. Now run the set command and confirm where $storage is pointing to.

Also side note, I always set $variable using ${variable} so the bash interpeter really knows what I want as a variable.

When you hit the ctrl-c that should be the exact state of the machine. The /images directory on the FOS target computer should already be mapped to the FOG server /images directory at this point (during the execution of the postinstallscript)

@ariederer26 If you know what the root user account is mysql (this is different than the root user for linux) use that user id and password. If not there are instructions on the internet to reset the root user’s password for mysql. That should have full rights.

Also in /opt/fog/.fogsettings there should be the user account and password the fog code uses to access the mysql database that account has full read / write access to the fog database too.

@mrilly ok two things that jump out at me

1. You are still using the 4.19.x kernel for some reason.
2. The intel nic 8086:0d4c was first added to the 5.5 version of the linux kernel.

You need to go back and review why the 5.15 versions of the linux kernel are not installed in your fog server.

From the fog server’s linux console you can use file /var/www/html/fog/service/ipxe/bzImage to read the version of the kernel to confirm it got updated correctly.

@user419 How did you turn on https protocol in FOG? It almost appears that ipxe did not get recompiled with the same certificate that apache is using for the web server.

@lpetelik For a bit of clarity I have a few more questions.

1. Is the sql server process crashing (no longer in memory) or is it just not responding to requests.
2. What version of FOG are you running?
3. It sounds like you have a fog master node and multiple storage nodes running on VMs? Is that accurate? How many storage nodes do you have in this configuration?
4. How many target computers are in your environment that have the fog client installed?
5. When inspecting syslog/messages/sql server logs do you see any error messages relating to sql server, like too many connections?
6. What is the fog server host operating system?

Just for clarity in definition updating “the kernel” relates to the FOS Linux kernel [bzImage] that gets transferred to the target computer during imaging. This reference has nothing to do with the FOG server host OS kernel.

It looks like the code that runs on FOS Linux doesn’t use these timeout values.

ref: https://github.com/FOGProject/fos/blob/af9e624a59752e126308afda18bfbd4a398e7b2a/Buildroot/board/FOG/FOS/rootfs_overlay/usr/share/fog/lib/funcs.sh#L769

It looks like its possible to add them with little effort. The next step would be to see if we can trap the timeout so the code could issue a power off command.

Edit: With some crude testing it appears that udp-receive will exit with exit code 0 on a successful reception and 255 on a receive timeout (–start-timeout) so we can trap when it unsuccessfully starts the stream.

@alexamore90 PC specifications in what form?

When the computer is in the iPXE menu before image registration you are restricted to what information is available in iPXE. Registration uses FOS Linux so better quality of information about the PC is accessible there.

@rtarr FOG is designed to have one imaging network.

(I’m going to read between the lines here, if this isn’t how you have it setup then please claify)

All devices on that network must be able to reach the master node during pxe booting to find out where its assigned storage node (master or slave) is located. Replication between the master and storage nodes also happen in the identified imaging network, using the IP addresses defined storage configuration panels. Dual IP addresses for FOG imaging infrastructure is not supported.

With that said there are some things you can do.

Your fog server can function as you mentioned with a management nic and an imaging nic. The only thing the management nic can do is manage your FOG server, it can’t be used for any imaging functions.

I’m suspecting that you have a master node at one location and a storage node at a different location where the defined imaging networks are not connected for some reason. That is why you want to replicate the raw data on a different interface than our imaging network. Can this be done, not with FOG, but it can be done. Simply disable the FOG Replication service then use rsync and cron to schedule to replicate the /images directory between the master node and storage node on a timed basis. rsync also has bandwidth restrictions if you need to slow down the transfer. With rsync, just use the IP address on whatever network you want to send the raw files across.

Just be aware that the storage node must be in contact with the FOG server 100% of the time or imaging won’t happen in the remote location, because the storage node is using the database from the master node to get its instructions.

@sniffski OK that bit looks good.

So since you are comfortable with wireshark that will make the next bit easier. Since the FOG server and target computers are on the same subnet, lets use tcpdump from the FOG server perspective. (or wireshark on the fog server if that is where you are currently doing the capture from). Using wireshark on a witness computer will pick up the broadcast dhcp messages but it won’t the unicast messages. After the dhcp process the target computer reaches out to the fog server (dhcp option 66) and requests the file (dhcp option 67) and then transfers the boot loader (ipxe.efi) to the target computer over tftp. This is done over uicast messaging. So the witness computer won’t see it.

I have a tutorial here how to do this with tcpdump: https://forums.fogproject.org/topic/9673/when-dhcp-pxe-booting-process-goes-bad-and-you-have-no-clue

If you want to use wireshark on the fog server then use this capture filter port 67 or port 68 or port 69 or port 4011 Right after the DHCP DORA process it should ask the FOG server twice for the file defined in dhcp option 67, the first will be to check the size of the file the second to request the file.

Also be aware for uefi that safeboot needs to be disabled on the target computer or it won’t boot the nbf (network boot file) if it does transfer it OK.

If you can’t get a solution then post the pcap to a file transfer site and either post the link here or use FOG DM to chat the URL to me and I’ll take a look at it.

@ayy_nelson said in PXE boot issue with HP Probook 450 G8 (Realtek Nic):

I too purchased a handful of new G8 HP Laptops.

Some things we found with this new generation of systems with this realtek network adapter. Update your FOS Linux kernel to 5.15.x series (we just discovered this) and also update your iPXE version using this method. https://forums.fogproject.org/topic/15826/updating-compiling-the-latest-version-of-ipxe

Right now we only have anecdotal evidence that it fixes what changed in this generation of hardware.

@jtappen TBH I can’t remember if I responded to you or another poster in a different thread, but we’ve seen an uptick of issues with these new HP generation using the realtek nics.

This issue is specific to iPXE and the HP hardware. So for uefi there are a few .efi boot loaders I want you to try.

The default is ipxe.efi this contains all of the known (at the time of compiling) nic drivers. This boot loader is much like linux where the divers are compiled into the operating system.

The second is snponly.efi. This boot loader uses the driver built into the network cards. This should have the maximum compatibility. I would suggest that you try this to see if it works for these hp computers.

The third boot loader I want you to try is realtek.efi. This boot loader has only the realtek drivers compiled into the boot loader. See if this one works.

For reference you have iPXE version [g4bd0]

Do you have a cheap unmanged switch that we can have you insert between this computer and the building network switch? I’d like to see if you can get an IP address with that cheap switch in between the computer and the building switch. We need to do this test to rule out something common.

@flareimp Ah ok. You will want to do that with a post install script. A post install script will execute just after the image is pushed to the computer and just before the cloning completes.

With the post install script the ideal is to connect to the deployed image’s hard drive, make the adjustments to the .fogsettings file. The continue on with the reset of imaging. The only thing you will need to do before you capture the image is create a script that will run within the context of the newly imaged computer to run the installer command with the -y to execute the installer. Before SystemD (i.e. SysV) I would just create a 99local file in the startup folder that would execute the first time boot up commands. That 99local script would then remove itself from the startup folder so it would only execute once. There maybe something more elegant that can be done with SystemD.

The basic concepts are laid out here. I know this post is windows centric, but the way we go about it is very similar under linux. I just haven’t had a chance to write a linux equivalent of this tutorial: https://forums.fogproject.org/topic/11126/using-fog-postinstall-scripts-for-windows-driver-injection-2017-ed

@jonathan-cool said in Golden W10 21h2 : fog.drivers : wrong partition on HP 640 G2 laptop:

First I recommended lsblk not blkid but it did give me enough info.

That non-movable partition 4 will keep the c drive partition from expanding.

My best chance is to recreate the W10x64 21h2 UEFI image at the beginning, no ?

The easy answer for me to say is yes, you should rebuild it.

The why is a bit harder to explain but the sins of the past are now slowly dragging you into a fiery pit of hell.

1. You manually converted a mbr disk into a gpt disk, and relocated the efi partition to some other location than disk 1 partition 1. UEFI firmware will find the partition because it scans all of the partitions to locate its uefi boot partition.
2. This efi boot partition is behind the OS partition keeping it from expanding correctly because the EFI partition is not expandable or portable.
3. You have a non microsoft standard disk layout
4. You upgraded from 1909 to 21h2, that’s 5 OS generations different. In place upgrades always have strange issues (in my experience). 21h2 is a completely different OS than 1909, yes they are windows 10 but under the hood everything is different.

I think you would be best served by starting clean once again, with all of the current patches and applications for your organization. I know it will take time to recreate your golden image but I think a redo is the best answer.

NOW TO THE CAVEAT, if 21h2 is doing something different with the disk partitions and adding more windows directories its possible that even after you rebuild a clean 21h2 disk, the script may still fail.

@Backpain This looks like a normal FOG config file. It is setup exactly how you need it.

So let me ask a bit more about your network.
1, It looks like all devices (fog server, pxe booting computer) are on the same subnet. Is that correct?
2. If yes is that vlan 4 an isolated network used just for imaging or is it a general purpose vlan with many types of devices on it?
3. Is there any chance another dhcp server is responding to the pxe boot requests? Either by a dhcp-relay / helper service on your vlan router or by directly having a second dhcp server on your network?

Beyond the questions, I think our next step is to see if we can get a packet capture of the pxe booting process to see where the client is being directed and by whom. I have a tutorial here: https://forums.fogproject.org/topic/9673/when-dhcp-pxe-booting-process-goes-bad-and-you-have-no-clue?_=1673391869214

Upload the pcap to a file share site and post the link here and I’ll take a look at it. If you want to review it yourself, use wireshark to open the pcap file. Look at the dhcp OFFER packet. This will be from your dhcp server. Look at the dhcp header for the next-server and boot-file fields. They should contain data… but there are two many what-ifs here, its probably best to let me look at it

@Omar_Medhat How big is the iso image.

There are some limitations you need to be aware of.

1. Memdisk will only work in bios mode.
2. Memdisk is a 32 bit application so that limits the size of the iso image to 2GB.
3. To boot out of an in memory iso image, your system needs enough ram to contain the entire contents of the iso image AND enough free ram to run the OS from the ISO.

To the second part of the question, can you boot a .efi file via a custom menu. Probably.

Your custom menu configuration will probably look like something similar to this:
Menu Item: os.special_fw_update
Description: Firmware Update for this system
Parameters:
kernel tftp://${fog-ip}/bios_file_updater.efi
boot || goto MENU
Menu Show with: All Hosts

Where bios_file_updater.efi is the name of you unnamed update file. That file will need to be in the /tftpboot directory on the fog server.

ref: https://forums.fogproject.org/topic/10944/using-fog-to-pxe-boot-into-your-favorite-installer-images/9

@brian-mainake So do you have the fog location plugin installed? If yes you allocated each storage node (master or slave) to a specific location?

The last bit during fog target host registration, did you allocate the target computer to a certain location? From your explanation it sounds like either the storage node (master or slave) or the target computer is not in the same location so it goes to the master node.

@ikt_le Before you go and setup dhcp on FOG server, answer me this: Does the network where the target computers will PXE boot have an existing dhcp server? If yes is it a linux or windows 2012 or later dhcp server?

If yes on both questions then we can use that dhcp server so there is no need for FOG dhcp.

@DPR-SIO OK then you have a good chance to make this work.

so it looks like for your ftp user account your images should be in /images/dev If what you are showing me is the pwd when you are logged in using ftp.

So if you look at your storage node configuration you have /share/FOG/images Which is probably your absolute path on your NAS. From the FTP account it looks like root directory for the user account is at the /share/FOG path. SO… long story short in the storage node configuration change the FTP path from /share/FOG/images to /images and the snapins path to /snapins

Now the image Path will be the NFS path. For now don’t change that, but inspect that directory on the NAS. Do you see a subdirectory in that path with a name that looks like a mac address? If yes you have the image path set correctly. If I remember right where the image capture was failing was on the FTP login not image capture.

At the end of this workaround, you can delete any directory and contents in the /images/dev/ where the directory name appears as a mac address. That directory should only exist during an actual image capture task.