Windows on ARM

MarkG

@george1421 Thanks for the info. I’m going to be purchasing one of these Lenovos so I will try and assist if I can.

george1421

@MarkG OK let us know because there is some info that you can collect on these devices to help the developers validate the code.

MarkG

@george1421 Great. Let me know what you need and how to get it and I’ll do that as soon as I’m able to.

stokehall

@george1421 We have purchased 2 ARM Dell Laptops and want to try and get fog to capture and deploy Windows 11 images on these. We are finding it unable to PXE boot and wanted to ask if there is specific files we would need to modify or replace in FOG and if you had any advice on how to go about attempting this? I am not very experienced with Linux but want to see if I can get this working.

george1421

@stokehall ok, let me start out with neither the developers or myself have an ARM system (other than an RPi, but that is a different beast) so we will rely on the forum members to help us debug this.

With that said, let me tell you how it should work. When you pxe boot the computer it sends out to the dhcp server “hey I’m this kind of computer, please configure me” message. We need to see this message but more on that later. When the fog server or dhcp server sees this message it will send two optional dhcp fields. One is the boot server’s ip address and the second is the name of the file to bootstrap the computer. The boot loader/strap file that FOG uses is iPXE. The default ipxe files need to be hardware specific. For bios x86 systems the default should be undionly.kpxe, For uefi x86 systems ipxe.efi (or snp.efi) The first issue here is that the fog server or dhcp server always assumes your computer is x86 based. What you need to do to get past this part is to tell your dhcp server to send the ARM boot loader (full disclosure I don’t know what its called, but its on the fog server). Look on the fog server in the /tftpboot directory. Search for all files that end in efi ls -la /tftpboot/*.efi one of those efi files should have arm in the title. Get that file name and key it into the dhcp server/s option 67 value (understand this is a global setting and will send the arm file name to all pxe booting computers, but right now its just a test). With that arm boot loader in place you should get to the FOG iPXE menu. If you do then wonderful. Try to get into the hardware compatibility checker. Hopefully the ARM version of bzImage will be sent to the target computer. If you get into the hardware compatibility checker. That tests the FOS engine, if it runs it should image.

george1421

@george1421 For debugging pxe booting issues, I need to see the output.pcap file that is generated from this tutorial. https://forums.fogproject.org/topic/9673/when-dhcp-pxe-booting-process-goes-bad-and-you-have-no-clue

This task will show us the “hey I’m this kind of computer, please configure me” message. In that message I need to see what exactly the computer is saying what type it is. We will need this to figure out a long term solution to updating dhcp option 67

MarkG

@george1421
I have mode some progress: Using FOG 1.5.10.1593 on Ubuntu Server 24.04
I was able to find the DHCP boot parameters using wireshark, and I’ve added this to my dhcpd.conf:

class "UEFI-ARM64" {
    match if substring(option vendor-class-identifier, 0, 20) = "PXEClient:Arch:00011";
    filename "arm64-efi/snponly.efi";
}

This makes the iPXE part work and I can boot to the FOG menu. I then tried to run a Full Host Registration. The first problem was that FOG is sending it the x86 bzImage and init, so I (temporarily) made this change to the FOG config, having downloaded the ARM kernel and the ARM init using the FOG GUI.

This gets me a little further but now the process stops here for a few seconds, then the laptop reboots. Same hapens if I try the Compatability Checker.

So currently the summary is that FOG is sending the x86 bzImage and init, and when I hack it to send the ARM ones, those don’t work.

I don’t remember which version of the ARM inits and kernels I tried, so I’ll try a few more and if I get any further I’ll post back.

Tom Elliott

@MarkG So that leaves me with a few notes already:

Need a storage for global arm Kernel and Init
Need to associate architecture to appropriate kernel/init.
Later we need to get kernel/init actually loading/booting?

MarkG

@Tom-Elliott

Need a storage for global arm Kernel and Init
Need to associate architecture to appropriate kernel/init.

Yeah. I don’t know how you detect that you need the ARM kernels at that point but atm it thinks it needs to send the x86 kernels. (I’m not actually using any x86 PCs so I haven’t actually broken anything by changing that config )

Later we need to get kernel/init actually loading/booting?

Not completely sure I’ve understood that, but with the config hack I’ve done it is definitely downloading the arm_Image and the ARM init, I can see that the filenames are correct just before it gets to the point where it stops.

I tried the most recent 3 builds (including the experimental one) of the ARM kernels and inits but it made no difference.

Tom Elliott

@MarkG What i mean is loading the actual system.

RIght now it seems (to me from the image provided) that it doesn’t get you into FOS. It loads, just gets stuck?

MarkG

@Tom-Elliott
Yes I think that’s correct if I’m correct that getting into FOS would mean the kernel started to boot. What you see there is what happens immediately after I make a selection from the FOG menu. It downloads the kernel and the init. The next thing that appears onscreen is what you see there, and then after a couple of seconds it reboots.

stokehall

@george1421 Thanks for such a fast and detailed response. Yes I fully understand that we are in uncharted waters and that few people have this hardware yet so I am happy to be usefull to the community with these devices for as long as I can keep them before they are provisioned.

I will attempt to do this after the weekend and report back my results.

How do I go about this step? “Get that file name and key it into the dhcp server/s option 67 value”

george1421

@Tom-Elliott said in Windows on ARM:

So that leaves me with a few notes already:

Tom, I did some testing back in 2021 trying to get a rpi to pxe boot. I never did but here is what I found with FOG and a few changes that needed to happen: https://forums.fogproject.org/topic/14959/raspberry-pi-4-unable-to-pxe-boot

I’m only adding this here for reference.

george1421

@stokehall said in Windows on ARM:

How do I go about this step? “Get that file name and key it into the dhcp server/s option 67 value”

The question back to you is what is your dhcp server for the imaging network? That is where you would set the value for the boot loader. BUT the OP of this thread has already done that and has proven how to get into the FOG iPXE menu, so the inits work. He has already figured out the arch type (11) for the dhcp server (what i needed the tcpdump/wireshark pcap for). Right now we have an issue with the fos linux (engine that clones disks) starting up. If I remember right the person that developed the FOG arm kernel was building it for a cortex CPU. It might not have all of the bits in it needed to boot the dell/lenovo. This is nothing unsolvable, we just need to find out about the processors in your system(s) and look at the kernel configs. This hardware is so new there aren’t a lot of info out there about it.

It would be interesting to see if you could find a live linux OS for the arm cpus and get that to boot. We could then reverse engineer the kernel settings for that live boot OS. (just thinking out loud)

george1421

@MarkG said in Windows on ARM:

class “UEFI-ARM64” {
match if substring(option vendor-class-identifier, 0, 20) = “PXEClient:Arch:00011”;
filename “arm64-efi/snponly.efi”;
}

Well done working out the hard bits. It sounds like there needs to be some fog ui updates and we need to get that FOS kernel to boot cleanly. In the FOG Global configuration settings. There should be a field for log level, set that to 7 to see if there are any additional log entries that gets displayed. The default value of 0 or 4 masks most of the kernel messages. The level 7 is only used for debugging.

george1421

FWIW It looks like snapdragon support will first be available in linux kernel 6.11

https://www.phoronix.com/news/Qualcomm-Adreno-X1-85-GPU-Linux

additional linux support docs
https://docs.qualcomm.com/bundle/publicresource/topics/80-70014-3/features.html

rodluz

I will be getting a snapdragon laptop on Monday for FOG/FOS testing.
It looks like kernel 6.11 will most likely be released Sunday but there already is a release candidate that I could build to test with.

@Tom-Elliott Implemented some changes to support ARM by default on the 1.6-alpha branch of FOG.
I am currently porting it over to the 1.5 dev-branch

george1421

@rodluz On that laptop, what is the processor model? From what I’m gathering (I’m a bit ignorant on arm systems) but snapdragon is akin to the word pentium. There are a few models that fit behind that banner. Is your system the x elite or one of the older processors like the 850?

rodluz

@george1421 I’m getting an X Plus, X1P-64-100 (10 cores up to 3.4GHz, NPU integrated). I’ll be getting an X Elite probably a few weeks later but from my understanding, they are the same generation.

MarkG

@george1421 This is all very encouraging! I tried kernel debug level 7 but there was no additional information.

Windows on ARM

248

11.6k

17.1k

154.6k