@Mightmar I wonder if the devs for iPXE has changed something in the ipxe source code to cause this error message about autoexec.ipxe not found. This should be supplied by the fog project add on files. I’ll take a look at the compiler to see if something has changed. You should not see this error.

Reinstalling 1.5.10 will fix the error of the latest build of iPXE. Also you mentioned about a later version of FOG. Yes you can install that over 1.5.10 without issue. It should also have updated (but not the newest version of iPXE).

@Mightmar Just a few points of info to give you.

1. iPXE is managed by a different project than the FOG Project. They have a quicker release cycle than the FOG Project so they will/may support newer hardware quicker.
2. When a specific version of FOG is released it contains the current version of iPXE at the time a specific version of FOG is released.
3. If you manually update iPXE using the following instructions: https://forums.fogproject.org/topic/15826/updating-compiling-the-latest-version-of-ipxe and then reinstall FOG 1.5.10 it will replace the updated version of iPXE created by the previous script with the version shipped with FOG 1.5.10. This is by design in case you make a change that breaks FOG you can always fix it by rerunning the FOG installer putting back the FOG files to a known good state.

So to say it another way, if you find issues with iPXE not supporting certain hardware, I would always upgrade the version of iPXE first and just remember that if you reinstall FOG it will replace the updated versions of iPXE and the FOS Linux kernel with the ones that were shipped with the current release of FOG. While I don’t know what the current release of FOG is I believe there are sub release later that 1.5.10 that fix a few discovered issues.

@alexamore90 Lets try to unpack this.

You have 3 ESXi servers connected to an unmanaged switch.

This part is a bit unclear. After deploying on a few pcs the speed dropps to 900mb/min (watch your unit of measure). Normally its up to 22GB/min (again assuming you are getting this number from partclone).

After deploying to a few PCs the speed drops. Is this simultaneous deployments or consecutive deployments the speed drops?

Unmanaged switch: So this isn’t an enterprise class switch. You may have issues with throughput on the switch itself. The aciscs might not be fast enough for the amount of data your are trying to push through. Look at the throughput listed in the technical documentation for the switch.

You didn’t mention the uplink speed from the ESXi servers to this switch. On a well managed 1GbE network you should get around 6GB/min throughput. On a well managed 10GbE network I’ve seen 13-15GB/min. So your 22GB/min seems a bit higher than expected. I can say with a physical FOG server connected via 1GbE link, I can saturate that 1GbE link with 3 concurrent image deployments. When that happens the error rate increases and the throughput drops off quote a bit.

The other thing if you are trying to do concurrent deployments with these 3 FOG servers, make sure the drives on ESXi are SSD or NVME drives and not spinning disks for performance reasons.

In the end I don’t think this is a FOG server issue specifically, rather something in the environment that is causing the speed issue.

@ALV_SUPECOLES This seems to be similar to this thread: https://forums.fogproject.org/topic/17759/ipxe-initialising-devices

In this case uefi firmware was updated then iPXE stops at initializing devices. If compiling and installing the latest version of iPXE doesn’t solve the problem then we will have to wait for either iPXE developers or Lenovo to fix the problem. https://forums.fogproject.org/topic/15826/updating-compiling-the-latest-version-of-ipxe

@nicolas-moraes We have to be careful some Soho routersthe don’t behave well with pxe booting. Some will put its IP address as the boot server even if you have dhcp option 66 configured. I don’t know the edgerouter, but if there is a bootp option make sure that is turned on too.

The edgerouter is based on vyos so it should behave correctly. Make sure when you define dhcp option 66 its the IP address of the fog server and for dhcp option 67 the boot file name needs to be all lower case. Make sure you don’t have a leading or trailing white space in the name.

If you have double checked everything then get a witness computer and load wireshark on it. Make sure you setup a capture filter of port 67 or port 68 start the scanning then pxe boot the computer until you get the error message then stop scanning.

What you are looking for is the DORA sequence
Discover (sent by target computer, akin to hello world, please configure me)
Offer (sent by one or more dhcp server, I would there to be only one of these and from your edgerouter).
Request (sent by the target computer)
Ack/Nack (sent by the dhcp server)

The interesting packet will be the OFFER.

Look at the packet, there should be an ethernet header section with the following fields.
Next-Server: this should be the IP address of your fog server (this is the bootp part)
Boot-FIle: this should be the name of the ipxe file to load.

If either of those are blank some pxe clients will not boot if they look at the bootp fields.

In the dhcp options section you should see option 66 and 67 that should mirror the values in the bootp part.

@JYost well there is two ways.

1. download the git version so you have all of the bits in place and recompile it.
2. redownload the released version of the ipxe files from here: https://github.com/FOGProject/fogproject/tree/stable/packages/tftp

I think you will be better served by option 1 in the case of upgrades, but the quickest route is #2 which will download the older binaries for iPXE.

@RocksAndRolls I’m not sure how much additional help I can provide here but where it failing is at the end of the imaging process, the target system logs into the ftp server on the FOG server using the fogproject user account. That fogproject user MUST be able to rename the captured file in /images/dev from the mac address and the move the directory from /images/dev/<mac_address> to /images/<image_name>.

If you have verified you can log into the fog server via ftp as the fogproject user using the password saved in the fog ui then it has to be a permission issue on the /images directory.

The image capture process is

FOS Engine -> connects to /images/dev NFS share and creates a directory that matches it’s mac address
FOS Engine -> Uploads the disk connect as the root user (on the FOS Engine)
FOS Engine -> Connects to the FOG server as fogproject user and issues a mv command to move the directory from /images/dev/<mac_address> to /images/<image_name>
FOS Engine -> Logs out of ftp and completes the cleanup before reboot.

@danieln said in "Windows Other" vs "Windows 10" when creating new Windows 11 images?:

Windows 10 - (9)

This selector was put in place many years ago to deal with the quirks in the OS between the early versions of windows. While there are some minor differences between win10 and win11 disk formats the FOS imaging engine is now robust enough to manage it. So from an imaging standpoint the developers could change the label to “Windows 10&11 - (9)” and be syntactically correct. Who knows what Win 12 will bring.

@rurap said in Slow restoration of Windows 11 with FOG on Proxmox:

02:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8161] (rev 15)
03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15) - This is the built-in network card

and the other two are Realtek cards, which appear to be identical.

Technically they are not identical, they have the same family name but one is a 8161 and the other is a 8168 of the rev 15 generation.

Just confirming that the /var/log/messages file exists and the /var/log/syslog one doesn’t ? The 8168 model has been around for years as you can see by the rev numbers. I believe they need nic specific firmware to operate. That should be called out in the boot up messages.

In researching this it seems one instant the 8169 driver was being installed instead of the 8168 linux kernel driver. This will take some looking by lspci -knn | more will list out all installed pci hardware with the “kernel drivers in use” for the hardware. As a hint for looking through the big list the section you are interested in starts with “03:00.0 Ethernet controller” since that is the built in nic you found in the previous post. Lets see if its using the 8169 driver instead.

Edit: I keep seeing reference to these kernel parameters in posts about debugging this nic r8168.aspm=0 r8168.eee_enable=0 pcie_aspm=off Not sure the importance, but noting it here for future reference.

Edit2: This command may help give the answer on driver than looking through the entire list of lspci commands: inxi -Naz

@ITCC Just to be clear you complied the latest version of iPXE using the instructions I provided? If that’s the case and it failed, then there is not much we can do because something in Lenovo’s firmware causing the software to hang. If rolling back the firmware isn’t an option (I don’t know what they were correcting with the update) I think we are stuck unless the iPXE developers have any idea why it might be getting stuck.