What is the status of aarch64 support for FOG?

[Sbergeron]

We have a bunch of of new arm blades that we are integrating into our FOG managed installation. It appears that while ipxe can be compiled to target efi on arm, the FOG specific modifications of ipxe don’t appear to play nicely (I’m getting exec format errors.) What’s the status of FOG support for arm, and would it be helpful for us to provide systems for testing builds?

[Tom Elliott]

I would need to see your ipxe embed script as well as the src/config/general.h,settings.h,console.h files.

It seems to me, the problem isn’t the ARCH building, but rather missing options in the configuration files.

[Sbergeron]

The files used were sourced from here: https://github.com/FOGProject/fogproject/tree/master/src/ipxe/src-efi

Thanks for the help

[pberberian]

@tom-elliott (I work with Sbergeron) We have cross compiled ipxe, and the arm host can successfully chainload to ipxe. ipxe runs the default.ipxe script and from there tries to grab the bzImage32 kernel. The problem is that the bzImage kernels provided are built for x86 / x86_64. We would be happy to work with you in cross compiling these kernels so that there is a kernel and initrd image that works on arm.

There is also the issue of being able to dynamically detect for arm hosts. From preliminary testing, this seems to be a good solution (at the beginning of default.ipxe):

Before:

cpuid --ext 29 && set arch x86_64 || set arch i386

After:

cpuid --ext 29 && set arch x86_64 || set arch ${buildarch}

So, if the cpu does not support x86_64 instructions, we use the buildarch which will either be i386 or arm64. Although, I am ultimately not sure if this is needed. Our DHCP server tells the arm host to boot from the arm-compiled ipxe image, and we can configure fog to make the host to boot from an arm specific kernel once we build one.

Many thanks!

[george1421]

@pberberian I’ve talked with one of the other mods on this exact issue a few weeks ago. Someone else asked for arm boot images. If you are capable of cross compiling iPXE then you should be able to do the remaining bits. You need to cross compile bzImage as well as the inits (as you mentioned).

The Developers build bzImage from the linux kernel source files found here: https://www.kernel.org/ You can get the kernel config template file from here: https://github.com/FOGProject/fogproject/tree/master/kernel You will need to update the settings required for your ARM processor.

The inits are generated by buildroot https://buildroot.org/ You can get the buildroot configuration and overlays from here: https://github.com/FOGProject/fogproject/tree/master/src/buildroot Of course you will need to adjust the settings to be appropriate for your ARM processor. While buildroot can build both bzImage and the init.xz files, the developers have chosen to build them separately. Personally I feel both could be built by buildroot, but to each his own.

Now once you get the ARM bzImageARM and init_ARM.xz you can copy to the fog server in /var/www/html/fog/service/ipxe directory. From there you can change the bzimage and init.xz on a host by host, or by host group, or globally depending on your needs.

Now finally if you get all of this to work, please contribute your results back to the FOG project.

[pberberian]

@george1421 Thank you so much! We will get working on this. We would love to be able to add full arm64 support to FOG

[Joe Schmitt]

@george1421 we have actually moved FOS out of the fogproject repo.

@pberberian see our fos repository, which contains build scripts for the kernels, filesystem, and x86/x64 architectures:
https://github.com/FOGProject/fos . It also contains example usage of our fos-builder docker image, which has all of the needed dependencies built in. Ultimately if we can get arm support going, we’d love to have a pull request for it, and we can add it to our automated building system.

[george1421]

@joe-schmitt should we then kill the other files so some block head moderator doesn’t link to them in the future?

[Joe Schmitt]

@george1421 they are removed in the working/dev branch. Once v1.5.0 is stable, the changes will be merged to master.

[pberberian]

@joe-schmitt Looking at that repo, it appears I will need to generate a new buildroot config and update the build.sh script to have a aarch64 option. Is that the jist of it? I will probably have to create a customized docker image that is either arm64 based or can cross compile.

Thank you all again for the help!

[Joe Schmitt]

@pberberian I believe so, @Tom-Elliott would be able to tell you more.

For the docker image, you should just need to add cross-compiling support (see https://www.linuxquestions.org/questions/slackware-14/cross-compiling-for-arm-with-buildroot-859107/ for a general idea). Optionally, you can also use a Debian system to build it as well, if you prefer to skip docker. Just make sure it has all the dependencies installed that the Dockerfile has: https://github.com/FOGProject/fos/blob/master/docker/Dockerfile#L4

[george1421]

@pberberian said in What is the status of aarch64 support for FOG?:

I will need to generate a new buildroot config

Generate a new base build root config and then probably manually merge the FOG settings in. (or generate a arm base config file and then merge the differences between arm and ia64 with a text editor. I’m not sure which would be easier). Its been a while since I messed with buildroot, and surely Tom would know what exactly needs to be done.

[Sebastian Roth]

@pberberian @Sbergeron I am sorry to bring up this kind of oldish topic again. Lately I have been working on a couple of things and stumbled upon something that makes me think that ARM support never made it into FOG completely (exit type calling ARM binaries that don’t exist in FOG). So I am wondering if you are still interested or maybe got it worked out yourself? I am more than happy to add what is needed to fully support ARM clients but on the other hand I don’t fancy maintaining the ARM support if it’s not used at all.