2019 Macbook Air PXE woes
-
Hello everyone, I am the network admin for an entirely apple organization. Ive been using fog for years, and I am able to clone all macs/macbooks up to the 2019 model, but now im having some issues with getting it running right on our new 2019 macbooks.
My current setup for imaging consists of a small 400mb partition on the local disk itself (blessed of course) that boots grub and lets me pick from the imaging options. As a temporary workaround to the pxe booting issue, I have a copy of clonezilla on the disk itself that copies itself to RAM and mounts a cifs share to save/restore images from automatically, though this doesnt give me as many features as capturing with fog would (compression size, multicast etc…), it does give me the basic functionality of being able to plug an ethernet cable into a machine and image it over the network.
I compiled my own ipxe binaries from the latest source, and I added debugging output to my ipxe.efi binary. I have tried 3 of the pxe binaries, each giving different results, I have also tried using the pre-compiled fog binaries as well as premade ones from netboot.xyz:
All testing is done with the official apple belkin adapter:
https://www.apple.com/shop/product/HJKF2ZM/A/belkin-usb-c-to-gigabit-ethernet-adapter
or this USB C dock:
https://www.amazon.com/Anker-Adapter-Ethernet-MacBook-ChromeBook/dp/B071G83L1JiPXE.efi - I get ipxe initializing devices…OK!, net0 errors out (internal wifi card) with:
wl_ioctl: dhd_wl_ioctl_cmd failed. cmd 263, ret -14, ifidx = 0
wl_iovar: wlIoctl failed name cur_etheraddr, ret -14
BCMDHD_UNDIStationAddr: error setting cur_etheraddr !I assume everything that starts with wl is referring to a wireless card, so hopefully shouldnt be having any effect on the ethernet NIC.
Then net1 (ethernet adapter) complains about “no configuration methods succeed”, rinse repeat until I hit a pxe shell.
snp.efi - Same as ipxe
snponly.efi - It goes to load, I see a white block of color, top left says “Welcome to grub” but nothing else appears, normally underneath I would see somthing like:
“iPXE 1.20.1 (version number) Open Source Network Boot Firmware” but the screen just hangs after the intial loading of the binary.I would be happy to provide any additional information you need, all of my testing is done with either a windows DHCP server or an ISC-DHCP server (I have special apple classes defined in ISC, windows is just option 66 and 67), I also have a pretty decent homelab/multiple test environments at work (plus a wide variety of diverse hardware to test on), so I dont mind getting my hands a bit dirty. I however, am very much NOT a programmer, so im afraid I wont be of much help there
Thank you,
Smoblikat -
@Smoblikat Congrats! You seem to be one of very few users who managed to still image Macs even though Apple has put up a fair amount of hurdles over the years. Well done!
Unfortunately I don’t think we’ll be able to help you much with the issue you face right now. It’s very specific to iPXE and while we have done some debugging on this it’s rare that we can really dig up such issues deep down in the iPXE code.
But let me still try to throw some ideas at you and see if that is of any help.
Do you build custom iPXE binaries or do you copy those to your drive which we provide with FOG? If you know how to compile your own binaries you can add debug output and try to trace down why it would not pick up an IP.
Have you used wireshark/tcpdump yet to see of it actually sends a DHCP request or not?
-
@sebastian-roth Thanks
I do compile my own binaries (tried pre-compiled also), I just compiled ipxe.efi with DEBUG=dhcp, the output I get is:
DHCP 0x60030488 could not transmit UDP packet Input/Output error (ipxe.org/1d6a5498)
https://ipxe.org/err/1d6a54 (looks like dokuwiki, I actually might be able to contribute to this if I ever get it figured out!)
I have not done a wireshark analysis yet, but I need to run one for some weird VOIP issues im having anyway, so I should be able to get to that pretty soon. I will definitely keep you guys posted, thanks for the help!
-
So I ran some wireshark tests, and from what I see, the issue is completely related to ipxe not being able to use the USB NIC. If I boot the computer with the option key held down, I see DHCP requests flow normally from the machine.
Even sitting in the grub menu on my imaging partition shows a successful DHCP transaction, but as soon as I load into pxe there is 0 activity going on from the network card.
Running ifstat show that the link is up, but there are TX/RX Input/Output errors. Interestingly, the apple NIC behaves slightly differently than my USB C dock, if I unplug and re-plug the apple adapter while in a pxe shell, I can manually run DHCP and it does seem to pick up an address, but I cant ping/communicate with anything (still showing I/O errors). I tried to manually chain the fog server from the pxe shell after getting an address, but it times out. Maybe im using the wrong server path? What would be the correct path to chain from, and would I use http or tftp?
chain http://<my-ip>/tftboot/??
Thank you,
Smoblikat -
@smoblikat You are the first I hear of to get the macbooks to pxe boot. Normally we have people use a USB boot stick On that USB stick we boot right into FOS linux via a grub boot loader. The issue we see on newer macs is that apple has put that T2 (encryption) chip in the middle of everything. So to image the newer apples you need a special FOS linux kernel that has the T2 patch (hack) installed. I have that one-off kernel if you need it.
-
@Smoblikat Well that sounds not too bad. Got an address from the DHCP?! Wow! Did you see that communication in Wireshark?
With FOG you’d run:
chain tftp://x.x.x.x/default.ipxe(put in the FOG server IP instead ofx.x.x.x…)
