IPXE Issues

[jaoyer]

This is going to be kind of long and I apologize in advance for the lengthy post, but I have been tearing my hair out the last week trying to get this to work.

Background: We have been running a FOG server (.32) for the last 6 years and it has been rock solid for us. A few months ago we got some newer computers in that I tried to image using our .32 server. It did not work (tried multiple kernels to get them to boot into the imaging process), so I thought that I would give the newest version a shot. At the time the newest version was 1.30. I installed 1.3.0-RC-10 on Ubuntu 14.04 after trying to get the stable version to image correctly (another long story). When trying to boot using the default undionly.kpxe bootfile, everything would boot fine until it would try to load default.ipxe. The problem it seems is that it is looking on our old FOG Server (10.40.22.66) for default.ipxe and failing, as shown below.

I ended up using undionly.kpxe.INTEL boot file in order to image. When I use undionly.kpxe.INTEL it will not boot load default.ipxe on a consistent basis (maybe every third or fourth time). I have noticed the number of “dots” on the Configuring line will be longer when it works. The first picture is of it failing and has 5 “dots”, the second is of it working with 13 “dots”

I assume that when the it fails with 5 dots that it is looking at our old server and not finding default.ipxe, and when it works it is looking at our new server (10.40.22.67) and loading default.ipxe.

My question is this:
Is there a way to keep our existing .32 server going until I update the Fog Client on the the rest of our computers to point to our new server? Is there anyway I can hardcode our new server’s ip address in the ipxescript and create a custom kpxe file and just bounce back and forth between our old and new server using dhcp options until I migrate everything over?

I took a look at the ipxescript located here: https://sourceforge.net/p/freeghost/code/HEAD/tree/trunk/src/ipxe/src/ipxescript

I copied that script into rom-o-matic.eu’s ipxe image page and changed the “chain” line to reflect our new server (chain tftp://10.40.22.67/default.ipxe), but it failed with a params error: command not found; Could not boot: Exec format error.

Looking at the ipxescript, I assume that the undionly.kpxe is finding my old server as “next-server”, and would like to change this to hardcode my new server. We got some new computers in last week that I would like to image with the new server and want to get it so it consistently boots to our new server.

My knowledge of this stuff is pretty limited, so I could use any help that I can get. Thanks in advance.

[Wayne Workman]

@jaoyer said in IPXE Issues:

As far as the updating issue going from 1.30RC10 to 1.40 (detailed in my 2nd response), should I create new thread?

Yes.

[Tom Elliott]

@jaoyer Did you run the installer as sudo ./installfog.sh?

I might suggest rerunning the installer after gaining root first through sudo -i

[jaoyer]

@Tom-Elliott

I think that fixed it. I do have split scope dhcp setup on two servers, and only changed the options on one of them. I changed them in both and it seems to be working fine now.

Feel free to call me an idiot :smile:

As far as the updating issue going from 1.30RC10 to 1.40 (detailed in my 2nd response), should I create new thread?

[Tom Elliott]

@jaoyer So is that the global options for options 66/67? Is it possible your lan is using “scoped” options to point lans where to get their options 66/67?

[jaoyer]

@Tom-Elliott

Option 66 is set to the new server
Option 67 I have set to either undionly.kpxe or undionly.kpxe.INTEL (I’ve tested both).

Cat returns:

#!ipxe
cpuid --ext 29 && set arch x86_64 || set arch i386
params
param mac0 ${net0/mac}
param arch ${arch}
param platform ${platform}
param product ${product}
param manufacturer ${product}
param ipxever ${version}
param filename ${filename}
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
:bootme
chain http://10.40.22.67/fog/service/ipxe/boot.php##params

[Tom Elliott]

So option 66/67 of the DHCP server are set to point at the New fog server?

The Old fogserver’s IP Address in the 66 option would cause the problem. YOu would need to update the option 66 to point at the new fog server.

What’s the contents of:

cat /tftpboot/default.ipxe

[Junkhacker]

@jaoyer whatever is serving out options 66/67 dhcp options is providing the next-server address.

[jaoyer]

@Junkhacker

Copying the file over has worked consistently the last 5 times I booted a test machine, so I am fairly certain that that works. However, I am still concerned why it is pointing at the old server. It is my intention to decommission our old server at some point, and am afraid that it will still look to the old server for the default.ipxe file. To test this, I took our old fog server down so that it was completely shut off and booted my test machine . It was still looking for the the default.ipxe file on our old server even though it was down and it eventually timed out.

https://drive.google.com/open?id=0B6yrqHh_joc2bWdYWkZobTlHSmV3TXQyNk9IcklnRG5YLTZv

Is there any cache or anything that I have to clear out on our switches?

How does the “next-server” part of the script work? Is it something as simple as giving the new server an address that comes before the old server?

[Junkhacker]

@jaoyer those worked. any luck with default.ipxe?

[jaoyer]

@Junkhacker

3rd time’s the charm…try these:

Picture 1: Failing using the old server address
https://drive.google.com/open?id=0B6yrqHh_joc2d20tWEZzdWcwMVBhckk4V1dZb0Z4MlVvMkpr

Picture 2: Failing with undionly.kpxe.INTEL with 5 “dots”
https://drive.google.com/file/d/0B6yrqHh_joc2aENyMnJtRE16aFE/view?usp=sharing

Picture 3: Working undionly.kpxe.INTEL
https://drive.google.com/open?id=0B6yrqHh_joc2WnBhWUdEemFIQVNqZ1dybzk4RTlOeFFmSENn

[Junkhacker]

@jaoyer

Your client does not have permission to get URL /v8XwHa7EvGmlAc-9NqToHtVtsfdfcbDoKR-Q-y0feAsv0D-ahsZHipQqqP77P5y2BJ1WeycTgJyMFs4SpaqKDG64f-YfxHhxPAY6=w1920-h941-rw from this server.(Client IP address: <retracted>)

Forbidden That’s all we know. 

[jaoyer]

Sorry, should have included that. One of our Windows servers handle dhcp.

I tried to update to 1.40 and it went through most of the installation ok, but it failed at the very end connecting to the database and then the webui wouldn’t load (The page isn’t working. Currently unable to handle this request. HTTP ERROR 500). I was reading about php5 vs php7 and assumed there was something in the upgrade from 5 to 7 that caused the webui not to work. I took a snapshot of our server before upgrading so was able to rollback to 1.30RC 10.

That was going to be my next issue that I was going to post. Just for kicks and grins I created a brand new Fog Server and installed 1.40. The install went fine, the webpage loaded , but I still had the same issue booting to default.ipxe with the undionly.kpxe boot file.

I have not migrated all of our images and database yet. That is on my list of things to do this summer. I wanted to get the new server working first. Once I get this working, I do not believe we would need the old server.

Sorry about the images. I can see them in my post, but I am not sure how to make them show up for everyone else. Here are links to them. Can you see these?

Picture 1: Failing using the old server address
https://lh4.googleusercontent.com/v8XwHa7EvGmlAc-9NqToHtVtsfdfcbDoKR-Q-y0feAsv0D-ahsZHipQqqP77P5y2BJ1WeycTgJyMFs4SpaqKDG64f-YfxHhxPAY6=w1920-h941-rw

Picture 2: Failing with undionly.kpxe.INTEL with 5 “dots”
https://lh5.googleusercontent.com/iJWYg7W-TOIvJDVcCRDWuGBFcQRXxDrxpyFf3WfgxGJxiZl8cVd-PVGSSLde3PSz15sHYOZeq9DhzT4z6YIFs-n0YsobzVgoISkH=w1920-h941-rw

Picture 3: Working undionly.kpxe.INTEL
https://lh5.googleusercontent.com/TepDSUE2fgfP3X-73oCdM3RhF9frvxi32xkRI1vTmAzG7ocUjHEeuBxYnjeEEp0PiJ3vTeoVdWs9wEHFSDy4OQRZOoG2fFnMnNba=w1920-h941-rw

I will try copying the default.ipxe file to the old server this afternoon and report back.

Thanks!

[Quazz]

Does your FOG server also function as DHCP server?

[Junkhacker]

first thing i’m going to recommend is that you upgrade to fog version 1.4.0. so many many things have been fixed or improved.

have you migrated your images and database info from the old server to the new? if you get the new server working, will you need the old one at all?

quick and dumb solution for a lot of your issues: put a copy of the default.ipxe file from the new server in the /tftpboot folder of your FOG 0.32 server so that it can always be found and loaded. that file just loads the next file from http. it doesn’t technically need to be on the new server.

oh, and btw, your images didn’t load