New revelations have been made. As I said I now replicated the whole setup on a local isolated network (with exception of the apparmor modifications, not necessary on a bare-metal installation).
I have my witness-computer connected to both networks with 2 seperate NICs. Only one is active at a time. In both setups I first started wireshark on the witness computer and booted a notebook into pxe.
Main network: No packets or barely some DHCP-ACKs (that were not from the booting laptop).
Isolated local network: Discover -> Offer -> Request -> ACK. Laptop got an IP from the router and loaded the appropriate bootrom from the fogserver and booted from it.
Conclusion: Since the witnessing of the DHCP-packets have nothing to do with fog itself it is safe to say that there is some sort of broadcoastfiltering of the DHCP-relevant ports. The (almost) exact same installation worked in an isolated network but not on the main network.
Thank you so much, I finally know the exact cause of the problem and am able to proceed. I now have to write up a request if the networking team would be so kind to allow the broadcoasting of those port-packets to a single static IP that I own (I really hope they’ll allow that. Now that I think of it this makes total sense. If people are able to plug in their own devices into the network that behave like a DHCP-Server and then handle the IPs before the main DHCP does you are in a golden MITM-position and can intercept with the network-packets to your desire). But as far as fog goes, this is nothing from its side.
So I think I can say this is solved for the cause of “troubleshooting”. Even if my journey is not exactly at its end yet. Thanks again!
@quinniedid OK in my test lab I’m running the build listed in my post. That is with 6.5u1 and the Oct 2017 patch. Point being if I can get a few minutes of time I want to see if I can duplicate what you are seeing to see if VMWare addressed/changed/fixed the issue with a vSphere update. I don’t know one way or the other, but this sounds like a uefi firmware issue to me.
Then in vcenter for esxi add a datastore with the network file system option
give esxi the unraid servers nfs address
tell it where to mount it on esxi
and give the datastore a name, like images
Then on your fog vm, shutdown, edit settings, and then add a new virtual disk from that data store.
Then mount that as /images in your /etc/fstab. You might have to format it before you can mount it with mkfs ext4 or something of that sort.
If that overview isn’t helpful and you want to try this method, I’ll test it out a little and document the process more thoroughly when I get a chance.