Modify FOS with new package
-
Hello I need some guidance in modifying FOS to include a new package/software.
I have some new HP laptops that don’t have an ethernet port, this means that I have to use a USB-C to Ethernet adapter. When I register those laptops to FOG, the MAC address it adds is from the adapter instead of the passthrough system MAC address. It seems to me like the issue is with HP’s UEFI/BIOS programming, because it also doesn’t passthrough when in Windows/Linux.
HP has a software called “HP Flash and Replicated Setup Utilities for Linux” which I can use to retrieve the system’s MAC address from Linux.
My idea is to modify FOS and add that software to it, then modify the FOG registration script to check if the UEFI/BIOS has a system MAC address that differs to the one FOS can see. (Side note: I know that this would only work with HP computers and it is fine because we only use HP)
I know that I can modify the registration script using this https://wiki.fogproject.org/wiki/index.php/Modifying_the_Init_Image, but I am limited in my Linux knowledge in modifying FOS with the software.
Here are the install commands from “user guide”:
To install the kernel module: • tar xzf hpuefi-mod-x.x.x.tgz • cd hpuefi-mod-x.x.x/ • make • sudo make install To install the utility applications: • tar xzf hp-flash-<version><architecture>.tgz • cd hp-flash-<version><architecture> • sudo ./install.shSince FOS is stripped down a lot, it doesn’t have the make command or the Linux headers to be able to run the make command, also it would be horrible to have to install the software every time FOS loads.
I would greatly appreciate any help possible.
Thank you in advanced! -
@rodluz First FOS Linux is its own “version” of linux. It is built from source using buildroot.
That’s not 100% accurate in that buildroot is used to build init.xz which is the virtual hard drive. The kernel is build separately for FOG (even though buildroot could build both).
If you look into your linked hpuefi-mod-x.x.x.tar.gz file, its a kernel module so it needs to be built into bzImage not init.xz. The problem here is that FOS Linux doesn’t use dynamic loadable drivers, they are all built into the FOS Linux kernel. So what you need to look towards is adding that module as part of the FOS linux kernel.
This should not be an impossible task, it might just take a bit of research.
edit1: ref: https://stackoverflow.com/questions/11710022/adding-new-driver-code-to-linux-source-code
-
@rodluz OK I ran through and compiled that module into the fos linux kernel, but I don’t see how it applies to getting the mac address to pass through like you need it.
If I remember correctly, I started working on this problem because what was happening is that the current version of iPXE was seeing the pass through mac address but linux was not. This caused a conflict because you could register the computer, but then when you rebooted back into iPXE it would say the computer was unregistered. I was working on a kernel flag to say to update the mac address to what ever ipxe said it was, but I don’t ever remember finishing the project.
-
@rodluz said:
HP has a software called “HP Flash and Replicated Setup Utilities for Linux” which I can use to retrieve the system’s MAC address from Linux.
Have you tried it out yet and were you able to get the system MAC using these utils?
Before we think about adding this to the FOS kernel and init I suggest you install some Linux distro on one of your machines, compile&install the kernel module and tools to give it a try.
Beyond adding this to FOS I can see another issue. Before FOS is loaded iPXE is in charge. iPXE is booted an pulls the boot menu using the MAC address it finds. So if we can’t fix iPXE as well you still need to boot FOS from a USB thumb drive or from disk to make this work.
-
@george1421 Thank you for the explanation. I am trying to do this not as a fix for the passthrough to work, instead as a patch/workaround. @Sebastian-Roth I don’t think this should be added to FOS for everyone since it doesn’t really fix anything, again it’s just a workaround for me. I have read many threads of people bringing up this issue and maybe this can be a sort of solution for now but it would only work on HP machines.
In my case, iPXE is getting the correct MAC (passthrough from BIOS) but FOS is not. My thought was to run this utility when registering the computer to FOG so that the MAC address attached to it is the passthrough one instead of the adapter.I tried this utility on a laptop running Ubuntu and I know that I can retrieve the system MAC address for sure, and that it is correct. I am limited in my Linux knowledge to know how to add the module to FOS and then compile it. I honestly got lost with the reference that @george1421 provided.
-
@rodluz OK let me take a slightly different approach to explaining this.
That patch has 2 components. The first component is a kernel model driver. This kernel model driver lets the linux kernel talk to the hp specific hardware. Linux comes out of the box with 100s of kernel mode drivers supplied by the linux developers. HP has provided an add on one in this patch. I have successfully added that HP custom driver to the one-off (read not official) FOS Linux kernel. This is 1/2 of the challenge completed.
The second component of this patch is a compiled user land (level) program that talks to the hp kernel level driver I compiled into FOS Linux kernel. HP has released this precompiled user land driver for specific linux distros (FOS Linux is not one). BUT I suspect the precompiled version for ubuntu 18.04 might work with FOS Linux. The next step is to identify which of the precompiled programs will work with FOS Linux. I don’t have HP hardware so once I can run the user level program I can’t test to see if it actually works. But if it runs that is this part of the challenge.
Now all we need to do if we make it here is write a post init script to inject this HP user land program into FOS Linux as it boots up. Then call it with any required commands to read the pass through mac address.
Challenges 1 already passed, Challenge 2 needs some work, Challenge 3 is trivial.
-
@rodluz said in Modify FOS with new package:
I don’t think this should be added to FOS for everyone since it doesn’t really fix anything, again it’s just a workaround for me.
While you are right this is not a proper solution to an issue caused by the HP firmware I can still see us adding this if this actually helps a few people. On the other hand we might just provide thourough information for you to compile your very own kernel and init and scripts.
I have read many threads of people bringing up this issue and maybe this can be a sort of solution for now but it would only work on HP machines.
Be aware that many people just seem to have an issue until they find out how passthrough MAC settings really work. I am not exactly sure but I think I have heard few people actually having the issue you describe.
I tried this utility on a laptop running Ubuntu and I know that I can retrieve the system MAC address for sure, and that it is correct.
Well that’s a good start. So we know retrieving the correct MAC via the tools does actually work.
-
Hello, it has been a while since I have tried anything for this.
I now have more time and more equipment to actually work on this.I have taken some time to learn a bit more about the Linux kernel and adding a module to it (still so much more to learn).
@george1421 I used the reference link, excluding the arm specific parts of the Kconfig, and compiled the kernel. I can see that the driver was added during the compiling process because I see this outputCC drivers/hpuefi/hpuefi.o, but I can’t figure out if I did it correctly.
When I pxe boot to Debug Mode I can’t runlsmodormodprobeto find if it was installed.These are the steps I did to compile the kernel. Please let me know if this is correct.
1 . Clone FOS repo from GitHub
2. Run./build.sh --kernel-only --arch x64
3. When the output “We are ready to build. Would you like to edit the config file [y|n]?” comes up, I opened another terminal and then did this:a. Made directory
hpuefiin/path/to/fos/kernelsourcex64/drivers/
b. Copied the hpuefi module fileshpuefi.c&hpuefi.hto that directory.
c. MadeMakefilein/path/to/fos/kernelsourcex64/drivers/hpuefiwith content:obj-$(CONFIG_HPUEFI) += hpuefi.od. Made
Kconfigin/path/to/fos/kernelsourcex64/drivers/hpuefiwith content:config HPUEFI tristate "HP UEFI Kernel Module"e. Modified
/path/to/fos/kernelsourcex64/drivers/Makefileadding:obj-$(CONFIG_HPUEFI) += hpuefi/f. Modified
/path/to/fos/kernelsourcex64/drivers/Kconfigadding beforeendmenu:source "drivers/hpuefi/Kconfig"g. Modified
/path/to/fos/kernelsourcex64/arch/x86/configs/x86_64_defconfigadding:CONFIG_HPUEFI=y4 . Returned to the original terminal with the prompt still open and entered
n.
5. Enteredyfor the prompt:HP UEFI Kernel Module (HPUEFI) [N/y/?] (NEW)
6. PressedEnterfor everything else.After it finished, I copied the bzImage to my testing FOG server.
I can boot into Debug Mode but don’t know if the module was added successfully.Sorry for the long read but hope this is enough information to figure out if/what I did wrong.
-
@rodluz Sorry if this seems a bit disjointed but my brain is only functioning on a few brain cells since its been a long day.
If you booted with your custom kernel get into debug mode. Then as you discovered that lsmod will not find anything since that is for dynamically loaded modules. You can view the installed modules by looking in /proc pseudo file system. Somewhere in that directory structure there will be a modules folder and it will list all of the statically linked in modules. That will show you what modules are compiled in. I can’t give you the exact path, but if you went to compile your own kernel then you should have the skills to find that directory.
Something else to look for is on fos linux (what you pxe booted into) look at the /var/log/messages file. Search for the name of the driver you installed. If it loaded you should see the bootup message from that driver. Also look to see if there is missing firmware mentioned in that messages file. Some network drivers need an alternate firmware loaded to allow the driver to work.
When I make one off kernels, I don’t use the FOG method but the direct kernel build method since I’m accustom to using that method for other embedded OS’ I’ve created over the years.
This process you git clone the current linux kernel from kernel.org. You mentioned you cloned the FOS repo from github. In there is a configs directory (https://github.com/FOGProject/fos/tree/master/configs) Since you are building ARM based kernel then copy kernelarm64.config to the kernel root directory and save it as .config Now run nconfig command. This will startup the ncurses kernel configurator. You can add / remove static modules using this method to the kernel. When you have the kernel edits complete then save the configuration and exit the configurator. The last step is to compile the kernel with
make -j4 bzImageThe -j# tells the compiler how many cores to use for compiling. I would add the -j# up to the number of cores your development computer has for compiling. If you don’t know -j4 is a good place to start. At the end of compiling it will tell you the path where to get bzImage from. You don’t have to use this method to compile the FOS Linux kernel, its just this method you have a bit more control into what gets added into the kernel. -
@george1421 Okay so I found it in /proc/devices and I was able to get the HP program working!!
I had to add the program files to the init.xz filesystem and modify parts of the program to not use themodprobe&rmmodcommands.
I also had to create the directories needed for dynamically loaded modules and add the compiled module into it so that the program can look for it there.I now have to do the third part which is custom postinit scripts to use the system MAC address instead of the dongle address. This should be pretty easy.
Once I have that working, I am going to do a full write-up of it for reference for myself in the future and anyone else that comes across this.
-
@rodluz said in Modify FOS with new package:
I had to add the program files to the init.xz filesystem and modify parts of the program to not use the modprobe & rmmod commands.
Hmmm, yeah. The FOS Linux kernel is build without loadable module support as it makes things just a little more complex and is not useful in the use case of FOG. While modifying the program/scripts is one way to go you could also look into enabling module support in the kernel. Probably not something we would merge into the official FOS kernel but building your custom kernel might still make things easier for you.
These are the steps I did to compile the kernel. Please let me know if this is correct.
…Seems fine in general. I guess I could give you some hints on how to simplify this process using
patchandsed. Just let me know if you are interested. -
@sebastian-roth I figured out how to simplify it using
patchyesterday. I will look into the link that you posted.As you said, probably not something to be merged into the official FOS kernel. I’m guessing that I would not be the only one using FOG that would find this helpful so (unless there is a better way) I’m going to fork the FOS repo and make my changes there. Like I said before, I will write up what I did and try to streamline it as much as possible for anyone to reproduce.
