@alperi The bit if detail you are missing is what the kernel parameters were that was sent to the fog client. From what you posted it appears that the FOG server has all of the bits in the right spots.

In the kernel parameters that are passed to bzImage during boot up it lists where the FOS engine can find the deployment server. I would verify the IP addresses are correct. If everything appears correct with the parameters, we can debug this a bit more by debug deploy and then manually interact with the fos engine from the target PC’s console.

@djgalloway Just to add a bit of detail here. All of the work you did was on the iPXE side, which is great work by the way. The kernel driver I updated was after you select an FOG iPXE menu item that is when bzImage is loaded and run. It relies on kernel parameters that is provided by iPXE to find the root file system. This is technically what you fixed by ensuring that default.ipxe/boot.php from the fog server was being called. At the end of the day, I’m glad you got that working because your setup is definitely an edge case that works well in your environment.

@djgalloway Well you have a pretty complex environment when you are chain loading other boot loaders. What I would do on a temporary debugging bases, update your dhcp server settings to use the fog server’s IP for dhcp option 66 and ipxe.efi for dhcp option 67. This will use a fog only solution. Something else that might cause the issue is that the linux kernel that is booting, is not the kernel shipped with FOG. Or the opposite, the init.xz image is not a FOG image. The fog developers compress the init.xz image with an uncommon (in regards to linux file systems) compressor, if the booting linux kernel doesn’t have that image decompressor built in you will get that error about unknown block (because its still compressed).

It does look from your screen shot that both a bzImage and init.xz is being copied over to the target computer. Another but less likely issue could be that you are not using the custom ipxe.efi boot loader that was shipped with FOG.

As I was connecting the firmware to the kernel I thought to myself, it would be great to see the log files to see what firmware the nic needed. When I looked at the nic’s firmware directory there was only one file to I compiled it into kernel so I just added that file and moved on. While it was compiling I looked at the error again and came to the conclusion that the DPP package error is a bit of a red herring. While its an error that you will eventually have to deal with, it didn’t cause this specific error.

The issue is the kernel panic unable to mount the root fs. Initializing the nic comes after the root fs (init.bz) is mounted. Please verify that both bzImage (which is the kernel that is booting) and init.xz is being transferred via ipxe to the target computer. The kernel bzImage is having an issue mounting init.xz (root fs).

Now with that said, here is a current kernel with the intel nic firmware included: https://drive.google.com/file/d/1rISBOUuqAlnV_cq9HZgsFdjfygpB4JLT/view?usp=drive_link

@jmeyer Myself and another form moderator (Wayne Workman) came up with the concept of a mobile fog deployment server. The concept is that you would load FOG on a portable device (laptop or mini computer) with all of the necessary items for image deployment. IMO if this was going to be a truly portable computer a laptop with the built in screen and keyboard would be a better option. If you functioning as a MSP wanted to sell an imaging service than the mini or micro computer would be a better choice.

So the concept of the mobile fog deployment server is that the portable computer would have a full fog system installed on it. To minimize setup the mobile deployment server will have its IP address assigned by the remote site’s dhcp server. In this case FOG would not be your remote’s site dhcp server but it would function as a dhcp client (more on this in a bit). The next part you need to address is how to get the pxe boot information in the remote sites dhcp environment. You will do that with dnsmasq configured in a proxy dhcp mode (I have a tutorial on how to set this up in fog in the tutorial section). In this mode the FOG server (dnsmasq) will only provide pxe boot details leaving the remote site’s dhcp server untouched. With this configuration once the mobile fog server is removed from the site no pxe booting information is left behind to cause the remote site’s issues.

The issue you will have is that because FOG’s configuration is intended to be static, having the fog server’s IP address being assigned by dhcp will cause the FOG server to fail to pxe boot, to fix that issue Wayne created a script to automatically update the statically defined fields in FOG to make the IP addresses a bit more dynamic (note this is a fog server issue and has nothing to do with the target network or computers) That script is here: https://github.com/FOGProject/fog-community-scripts/tree/master/MakeFogMobile Looking at it that script is 8 years old, I can’t speak to the suitability of that script with the current version of FOG. It may need to be tweaked, but that’s the beauty of opensource software, if it doesn’t do what you need, you can fix it yourself.

It is possible to create a mobile fog deployment server, and back in the day the one I used worked great. So it is possible to do with little effort.

@djgalloway said in The DDP package file was not found or could not be read:

The DDP package file was not found or could not be read. Entering Safe Mode

This specific issue is the default fog kernel doesn’t contain the required firmware that is required to communicate with your E810 network adapter.

If I built a custom fog kernel for you in the past you already know this, but for the folks that find this post in the future… The fog developers in an effort to make a super fast imaging engine designed it using the 90/10 rule in that they will build a kernel for 90% of the deployments (which mean desktop computers) and leaving the rest for one-off builds. The supermicro servers or servers in general are in a different class than desktop/workstation computers. The desktop/workstation computers are pretty much the same even from different vendors. So the 90% rule has almost all of the hardware drivers built into the kernel. Servers class computers on the other hand have specialty components to aid in redundancy, performance, or monitoring (that other 10%) that are not typically found in the workstation class computers. Natively supporting that remaining 10% means almost doubling the size of the FOS engine kernel as well as having an impact on imaging speed. That is why the native FOS kernel doesn’t have every hardware driver built in. In your case the Intel E810 is a server class network adapter with QSFP28 ports (not something typically found in a workstation class computer).

With that said, I’m sure either the FOG kernel developers or I can create a one-off kernel for you. I will need to resetup my development environment because I just built a new linux server and haven’t move the files over from my old server, so it may take me a day or so to be in the position to create a current kernel with the required firmware built in for the nic.

@romprager The simple answer is depends on the iso image and what OS will boot from the ISO image.

I do have several how tos in the forum that shows how to net boot several different installers.

https://forums.fogproject.org/topic/10944/using-fog-to-pxe-boot-into-your-favorite-installer-images

@lsdo FWIW blanking out information regarding a private network address space isn’t helpful to get a full picture of the situation. If that information is for public ip addresses then you have every right to blank it out, but knowing your fogserver is at 192.168.1.100 is of little use to a hacker and it makes trying to understand what is going on a bit harder.

Based on the error message it appears this is a download (being able to see the results of the type variable would help to know for sure) but it appears that FOS (the OS that captures and deploys images) can’t find a hard drive in the target computer to deploy to. I’ve seen this in Dell computers if the sata adapter is in Raid-On mode, a quick test is to switch it to ahci mode in the uefi firmware to see if it can then locate the hard drive.

If you can’t get it by switching to ahci mode, lets dig a bit deeper into this target computer’s hardware.

@christop said in Plugin Hooks Not Running at Sub-Site:

lab computers in the desired room and thus used to multicast the room

just a data point here. Only the master fog server “the one with the database” can multicast images. Storage nodes can not (unless something changed in the last few years). Storage nodes are basically NAS devices with a little programming.

Now we used the location plugin to create storage locations. We assigned the storage nodes to a location and then target computers to locations, so as the target computer boots it finds the storage node it should image from. But that won’t help with the multicast part because storage nodes can only unicast images.

@christop When I was using fog I worked at a company that had many sites. There was a master fog server at HQ and storage nodes at each site. I don’t know what the subnet group plugin is so I can’t give direction there. But for us we had one image for global deployment. Then we used a fog post install script that updated the target computer’s unattend.xml file at imaging time. This allowed us to update the system’s default keyboard, locale, language, and destination OU the system was assigned to. We did not use FOG snapins for deployment but another tool once imaging is done. This other solution was used to deploy applications that could not be baked into the golden image because they had deploy time IDs that we did not want replicated to all deployed systems (like Antivirus system IDs).

If you are interested in this method I do have some tutorials that will give you a head start. But if you are using this subnet group assignments to add the machines to a snapin deployment group then my method will not work well.

@chevengur there is not enough information to help you in that picture. If we could see more of the error message that partclone threw (i.e. just above the top of your picture) we would know what happened. At the point in the script all we know is that partclone wasn’t happy.

@JasonNaughton This is a strange one. The linux kernel just doesn’t just invent mac addresses. It would be interesting to look up the first 6 characters of the mac address to see if you could identify the manufacturer.

So are you saying there are 4 physical nics in this computer. LOM, PCie 1, PCie 2? Does that mac address belong to the LOM?

I can say that we are dealing with 2 kernels here. The iPXE boot loader, and FOS Linux. Its technically possible to get to the fog ipxe menu and then when you start up FOS it doesn’t get an IP address because either the nic order has changed or there is missing firmware that is needed to init certain nics.

@chevengur I can tell you how I would go about figuring this this.

1. Take a computer that represents the finished design of how your disk are laid out.
2. Schedule a deployment to that computer, but before you hit the schedule task button, tick the debug checkbox then schedule the deployment. No worries as long as you pick debug mode since it will never get to the deployment phase.
3. Now pxe boot the target computer, it should boot into the FOS linux console. After a few screens of text you need to clear with the enter key you will be dropped to the FOS linux command prompt.
4. From there issue, the efibootmgr command with no parameters. It should print something similar to below (note this is from my laptop)

thunder@lightning:~$ efibootmgr
BootCurrent: 0005
Timeout: 2 seconds
BootOrder: 0005,0004,0000,0001,0002,0003
Boot0000* UEFI BC511 NVMe SK hynix 256GB SN9BN62231050BJ2H 1	HD(1,GPT,d00df89f-1edb-44f8-b325-245b607b2321,0x800,0x100000)/File(\EFI\Boot\BootX64.efi){auto_created_boot_option}
Boot0001* ONBOARD NIC (IPV4)	PciRoot(0x0)/Pci(0x1f,0x6)/MAC(b44,0)/IPv4(0.0.0.00.0.0.0,0,0){auto_created_boot_option}
Boot0002* ONBOARD NIC (IPV6)	PciRoot(0x0)/Pci(0x1f,0x6)/MAC(b440)/IPv6([::]:<->[::]:,0,0){auto_created_boot_option}
Boot0003* UEFI HTTPs Boot (MAC:B445065BDC4B)	PciRoot(0x0)/Pci(0x1f,0x6)/MAC(b445065bdc4b,0)/IPv4(0.0.0.00.0.0.0,0,0)/Uri(){auto_created_boot_option}
Boot0004* debian	HD(1,GPT,d00df89f-1edb-07b2321,0x800,0x100000)/File(\EFI\debian\shimx64.efi)
Boot0005* Ubuntu	HD(1,GPT,d00df89f-1edb-607b2321,0x800,0x100000)/File(\EFI\ubuntu\shimx64.efi)

You can see from this the default BootOrder is 5, 4, 0, 1, 2, 3 this lists the different boot managers found by the firmware.

So it will boot ubuntu first, then debian, the hard drive, onboard nic v4, onboard nic v6, http boot.

Now lets say I wanted debian to boot first I might issue the command.
efibootmgr -o 4,5, 0,1, 2, 3

Now reboot the computer with the reboot command see if it changes the boot order specific to your options.

After you get this worked out, you will need to clean up this deploy task on your fog server so it doesn’t do this moving forward. But for debugging as long as the fos engine doesn’t complete, every time you reboot the computer will enter the FOS debug console. This helps with debugging and tweaking your post install script.

@chevengur I have not had to do this before, but I can tell you in concept how to go about it.

You will need to create a post install script, that script gets executed just after the image is pushed to the computer and before its rebooted. This script is a bash shell script (remember the FOS engine is linux based).

Since it is linux based you will need to use linux command line tools to reset the boot image. The tool named is efibootmgr. This command is built into FOS linux engine.

So on its simplest form, you will create a FOG post install script and that script will call the efibootmgr to set the boot image. Understand that MS Windows will change this order without notice and at random times during its life.

Its not hard to do, but it will take a little effort on you to work out what is needed.

I can’t give you a step by step on how to do this but I can give you a general direction to look in if you want to go down this path.

@JasonNaughton Looking at the code the error would indicate that the target computer can’t reach the fog server.

https://github.com/FOGProject/fos/blob/8893d32bfb702dcf7b8f5427ccd6748fac15df17/Buildroot/board/FOG/FOS/rootfs_overlay/etc/init.d/S40network#L64

What I want you to do is to pxe boot the computer into debug mode. Schedule a deployment to this computer but before you hit the schedule task button tick the debug checkbox. Now pxe boot the target computer, you will still get the errors but you will be dropped to the fos linux command prompt.

run this command ip a s that should show if your network interface has an IP address. If not then issue this command.
/sbin/udhcpc -i enp128s31f6 --now where enp128s31f6 is the name I gleaned from your screen shot that appears to have picked up an IP address. See if it gets an IP address now. See if you can ping the fog server’s IP address.

The ‘checker’ script makes this call to verify your fog server is reachable.
curl -Ikfso /dev/null "${web}"/index.php --connect-timeout 5 replace the entire ${web} with the IP address of your fog server. See if that returns a value.

Finally search the system messages to see if there is something related to firmware.
grep -i -e firm /var/log/syslog I think syslog is the right file, if it returns nothing try /var/log/messages One error could be the network adapter needs a specific firmware for the network adapter to communicate. That firmware may need to be added to the linux kernel.

@AlexisPHC said in Use serial number as hostname in Fog:

did you ever get round to writing a guide for this?

Yes. I think the previous comment to your post referenced the files.

https://forums.fogproject.org/topic/14278/creating-custom-hostname-default-for-fog-man-reg?_=1762381023512

the file fog.customhostname uses a linux command dmidecode to extract the serial number from the smbios. And the rest of the ‘hack’ will pump that name into the full registration files. When I wrote that script I worked for a company that had a composite host name with the site code, a hardware type and then the dell asset tag appended onto the end. That is what this tutorial shows.

Now Tom mentioned that {SYSSERIAL} in the quick registration field works too. I wasn’t aware of that feature, it must be new. I know FOG version 0.30 had that feature but it was removed when FOG 1.x was released. If its back, that’s great!! that makes my script(s) unnecessary.

@AlexisPHC said in Issues with Windows DHCP Server:

because we run them in HA mode

but there doesn’t seem to be another DHCP server present

My interpretation of these two statements sounds in opposition.

If you are running windows server in HA or failover mode, make sure that both dhcp servers have the dhcp boot options configured. If I remember right these settings are not copied over between the dhcp HA nodes. Understand this might have changed with later releases of dhcp server but with 2016 the pxe boot stuff needed to be set on each node individually.

@AlexisPHC If it magically stops working again, then I would check to see if you have two dhcp servers on your network. If it continues to work, then move on to the next issue. But in general I don’t like it when stuff just starts working, because the tides can shift the other direction with out notice too.

@sega said in FOG Multicast on different VLANs:

Is there something that I can look up, to see where the problem is?

Well your router supporting PIM is a good sign. Some routers have a service called igmp proxy or igmp relay, or even igmp snooping (you will find this more on switches, but check). This service typically has a number of interfaces to listen on and then a master interface where your multicast source lives. Its job is to relay the multicast requests to the proper interface.

PIM has two modes sparse and dense, that may be just for switches, I don’t remember. Sparse mode only sends the multicast traffic to the subscriber’s port, where dense just blasts out the muticast to all ports. You want sparse mode if possible, that way only the ports with a multcast receiver will see the traffic and not flood your network.

@Kram-Man This won’t work. When fog captures an image it also creates a metadata file that describes the target hard drive. Also fog when it captures the image it creates partition base image files, not a monolithic file describing all partitions. Could you do this outside of the fog image capture process, probably. FOG uses partclone and zstd to capture and compress the partitions. You would just need to create the metadata data file that represents the target system layout. Its possible to do, but just creating your golden image and then capturing with fog would be just as quick IMO.