Not able to TFTP boot. Invalid Argument Error

[hancocza]

Hello,

I’m trying to tftp boot into the server to register some new hosts. When trying to boot, i get an “invalid argument” error. This only happened after upgrading to 1.5.5.

Things to note:
-I use the HTTPS setup switch for the installer.
-In order to get around the 066 067 DHCP setup (dont have access to the companies DHCP server), i use a ipxe USB key to point to the fog server.
-Server is running Ubuntu 16.04 LTS

[Sebastian Roth]

@hancocza See if you can upload the picture again.

I use the HTTPS setup switch for the installer.

Are you aware of the fact that you need to compile iPXE binaries to be used with a HTTPS enabled FOG server:

cd /path/to/pogproject/utils/FOGiPXE
./buildipxe.sh

Now you might use those for USB booting.

In order to get around the 066 067 DHCP setup (dont have access to the companies DHCP server), i use a ipxe USB key to point to the fog server.

Ever thought about using dnsmasq?

[hancocza]

@Sebastian-Roth Here’s the picture: http://tinypic.com/r/2lsvh9l/9

I don’t recall doing that for previous upgrades. I’ll try that! Previously i had to go and edit the default.ipxe file to show chain http://xxx.xxx.xxx.xxx/fog/service/ipxe/boot.php instead of chain https://xxx.xxx.xxx.xxx/fog/service/ipxe/boot.php. I really only use the HTTPS setup for the webserver and fogservice client connection.

As for dnsmasq, when i first put this server into production, I was told that it would not work within our network. They recommended going the USB route. It was fairly simple to setup the USBs so I never really pursued the alternative.

[george1421]

@hancocza How are you usb booting? Are you loading iPXE onto a usb? If so, what does your boot time iPXE script (default.ipxe) look like. The FOG iPXE kernels have specific parameters captured and sent to the fog server.

[george1421]

@hancocza the default.ipxe that is compiled into the FOG iPXE boot loader is here: https://github.com/FOGProject/fogproject/blob/master/src/ipxe/src/ipxescript

Also you will need one boot USB for bios based systems and one for UEFI based systems.

[hancocza]

@george1421 I made a usb using the easy method in this guide: https://wiki.fogproject.org/wiki/index.php?title=USB_Bootable_Media. I made both a uefi usb as well as a bios mbr usb. The mbr one i made using ISO to USB using an iPXE ISO.

When using the UEFI USB, the screen in the previous post comes up, where i need to enter in the IP address and then the tftp server does the rest. For the BIOS USB, i need to enter the command line and type in the next-server and the filename. It’s messy, but i rarely use the BIOS USBs now. I’ve tried both and they both gave me the invalid argument.

[george1421]

@hancocza If you look at the github site I referenced this is the ipxe script.

#!ipxe
isset ${net0/mac} && ifopen net0 && dhcp net0 || goto dhcpnet1
echo Received DHCP answer on interface net0 && goto proxycheck

:dhcpnet1
isset ${net1/mac} && ifopen net1 && dhcp net1 || goto dhcpnet2
echo Received DHCP answer on interface net1 && goto proxycheck

:dhcpnet2
isset ${net2/mac} && ifopen net2 && dhcp net2 || goto dhcpall
echo Received DHCP anser on infterface net2 && goto proxycheck

:dhcpall
dhcp && goto proxycheck || goto dhcperror

:dhcperror
prompt --key s --timeout 10000 DHCP failed, hit 's' for the iPXE shell; reboot in 10 seconds && shell || reboot

:proxycheck
isset ${proxydhcp/next-server} && set next-server ${proxydhcp/next-server} || goto nextservercheck

:nextservercheck
isset ${next-server} && goto netboot || goto setserv

:setserv
echo -n Please enter tftp server: && read next-server && goto netboot || goto setserv

:netboot
chain tftp://${next-server}/default.ipxe ||
prompt --key s --timeout 10000 Chainloading failed, hit 's' for the iPXE shell; reboot in 10 seconds && shell || reboot

It would be a trivial task to remove the dhcp bits and hard code in the IP address of the FOG server. You would have to recompile the ipxe boot loaders, but it would stop you from having to manually do stuff.

In your case look at the chain line of the ipxe script it called default.ipxe on the fog server and not boot.php directly. the default.ipxe on the fog server picks up several needed parameters that gets passed to boot.php. Without then boot.php gets lost.

[hancocza]

@george1421 Thanks for that, I’ll look at doing that once I get this up and running again. Once it fails, how can I see what params were passed that might be incorrect? Is there a log file stored anywhere?

[george1421]

@hancocza If you are calling boot.php directly from your usb drive then you are not passing any parameters. That is the problem. If you look at the default.ipxe file in /tftpboot of the FOG server you will see what parameters its passing (like mac address, arch, and so on).

Just to save you time:

#!ipxe
cpuid --ext 29 && set arch x86_64 || set arch ${buildarch}
params
param mac0 ${net0/mac}
param arch ${arch}
param platform ${platform}
param product ${product}
param manufacturer ${product}
param ipxever ${version}
param filename ${filename}
param sysuuid ${uuid}
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
:bootme
chain http://192.168.5.22/fog/service/ipxe/boot.php##params

[edit] just for clarity thats from FOG 1.5.4. I haven’t installed 1.5.5 into production yet.

[hancocza]

@george1421 Oh, got ya. I don’t call it directly. It first goes to default.ipxe and then from there calls the boot.php

[george1421]

@hancocza OK so we now know how you got here.

Looking at your picture its making a http call to boot.php. You made reference to using https? You are still at the ipxe prompt so something happened with the chain to boot.php

[hancocza]

@george1421 Correct. In the past, since I started using HTTPS, I’ve had to change the default.ipxe file to point to http vs https. https never worked for me (I’m guessing because i never compiled the binaries after updating). So I’ve always changed the chain address to be http instead of https. Now that doesn’t work either.

[george1421]

@hancocza Ok just to be clear you haven’t changed anything in FOG’s default.ipxe other than http and https?

What do you get if you key the following into a browser?

http://<fog_server_ip>/fog/service/ipxe/boot.php?mac=00:00:00:00:00:01

That should give you a screen of text, which is the FOG ipxe menu.

[hancocza]

@george1421 said in Not able to TFTP boot. Invalid Argument Error:

http://<fog_server_ip>/fog/service/ipxe/boot.php?mac=00:00:00:00:00:01

Here is what I get. I do change the IP section from a FQDN to the IP. When installing I put the FQDN as the IP address so that I don’t have to change too much.

#!ipxe
set fog-ip xxx.xxx.xxx.xxx
set fog-webroot fog
set boot-url https://${fog-ip}/${fog-webroot}
cpuid --ext 29 && set arch x86_64 || set arch i386
goto get_console
:console_set
colour --rgb 0x00567a 1 ||
colour --rgb 0x00567a 2 ||
colour --rgb 0x00567a 4 ||
cpair --foreground 7 --background 2 2 ||
goto MENU
:alt_console
cpair --background 0 1 ||
cpair --background 1 2 ||
goto MENU
:get_console
console --picture https://xxx.xxx.xxx.xxx/fog/service/ipxe/bg.png --left 100 --right 80 && goto console_set || goto alt_console
:MENU
menu
colour --rgb 0xff0000 0 ||
cpair --foreground 1 1 ||
cpair --foreground 0 3 ||
cpair --foreground 4 4 ||
item --gap Host is NOT registered!
item --gap -- -------------------------------------
item fog.local Boot from hard disk
item fog.memtest Run Memtest86+
item fog.reginput Perform Full Host Registration and Inventory
item fog.reg Quick Registration and Inventory
item fog.deployimage Deploy Image
item fog.multijoin Join Multicast Session
item fog.sysinfo Client System Information (Compatibility)
choose --default fog.local --timeout 3000 target && goto ${target}
:fog.local
sanboot --no-describe --drive 0x80 || goto MENU
:fog.memtest
kernel memdisk initrd=memtest.bin iso raw
initrd memtest.bin
boot || goto MENU
:fog.reginput
kernel bzImage32 loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=127000 web=https://xxx.xxx.xxx.xxx/fog/ consoleblank=0 rootfstype=ext4 storage=xxx.xxx.xxx.xxx:/images/ storageip=xxx.xxx.xxx.xxx loglevel=4 mode=manreg
imgfetch init_32.xz
boot || goto MENU
:fog.reg
kernel bzImage32 loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=127000 web=https://xxx.xxx.xxx.xxx/fog/ consoleblank=0 rootfstype=ext4 storage=xxx.xxx.xxx.xxx:/images/ storageip=xxx.xxx.xxx.xxx loglevel=4 mode=autoreg
imgfetch init_32.xz
boot || goto MENU
:fog.deployimage
login
params
param mac0 ${net0/mac}
param arch ${arch}
param username ${username}
param password ${password}
param qihost 1
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
param sysuuid ${uuid}
:fog.multijoin
login
params
param mac0 ${net0/mac}
param arch ${arch}
param username ${username}
param password ${password}
param sessionJoin 1
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
param sysuuid ${uuid}
:fog.sysinfo
kernel bzImage32 loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=127000 web=https://xxx.xxx.xxx.xxx/fog/ consoleblank=0 rootfstype=ext4 storage=xxx.xxx.xxx.xxx:/images/ storageip=xxx.xxx.xxx.xxx loglevel=4 mode=sysinfo
imgfetch init_32.xz
boot || goto MENU
:bootme
chain -ar https://xxx.xxx.xxx.xxx/fog/service/ipxe/boot.php##params ||
goto MENU
autoboot

[george1421]

@hancocza OK that tells us that boot.php code is working on the fog server. That is the complete iPXE boot menu. So we know its not the fog server causing you pain.

So you haven’t changed the fog default.ipxe, boot.php is working because the menu is created when you call it from a browser.

so the question is what is being passed that is causing boot.php to return an invalid ipxe menu.

The next step is to usb boot again and get the error. Then inspect the apache access log, error log, and php-fpm error log files. These are typically in /var/log directory.

[george1421]

@george1421 Just wondering is this fog client computer you are trying to USB boot registered in FOG? If so delete the registration and see if it throws the same error.

[Sebastian Roth]

@hancocza Your picture shows the iPXE error http://ipxe.org/err/1c0de8 - see this is definitely talking about TLS being the issue. Now having a closer look at the iPXE menu output you posted from “boot.php” I see that i does a chain https://... right at the end as well as console --picture https://... and others which all will likely fail if you don’t have iPXE binaries compiled with your certificate in it.

Those URLs are generated and the http(s) part is derived from the client request itself. So if a client asks for http:// all the URLs in the bootmenu will also be http:// - but we generate an apache config to redirect HTTP to HTTPS and therefore you end up with https:// URLs in the iPXE boot menu. Either you disable the forced HTTPS redirect or you look into compiling correct iPXE binaries. The later should be real easy using the script as I suggested.

[hancocza]

@Sebastian-Roth I ran the recompiling script last friday. At the end it says it needs to go through the FOG installation again, so i did. After that, it still gave the same error. Should I be allowing it to go through the FOG installation again or does that basically reset the iPXE binaries again?

[hancocza]

@george1421 said in Not able to TFTP boot. Invalid Argument Error:

@george1421 Just wondering is this fog client computer you are trying to USB boot registered in FOG? If so delete the registration and see if it throws the same error.

This happened on both laptops that were registered as well as a few that were not.

I also checked all of the logs after attempting to boot, and it didn’t update any of those logs. I assume that means it’s in line with what Sebastian said about the TLS error.

[Sebastian Roth]

@hancocza said in Not able to TFTP boot. Invalid Argument Error:

Should I be allowing it to go through the FOG installation again or does that basically reset the iPXE binaries again?

When you run the buidlipxe.sh Script it compiles new binaries including the correct SSL cert and puts them in the “installer directory”. From now on you can run the FOG installer as often as you want, it should always install those HTTPS-enabled iPXE binaries!