Host seems reachable by fog, but is not reaching PXE fog menu

thebrennan47

Server

FOG Version: 1.4.3
OS: Ubuntu LTS 16.04

Client

Service Version:
OS: Windows 7

Description

We are currently testing an upgrade to a new version of fog. I feel like I am getting close to up and running but have hit a few snags.

I have added a storage node, and it successfully connects to fog. (Graphs etc are showing up in the UI).

I have also added a local switch with a single host. On the host I have verified that PXE boot is the first option.

However, when I queue an upload task and manually reboot, I am getting nothing on the PXE boot screen. The single host is showing up as green in the UI, but I cannot ping it manually when logging into the fog server.

I have also verified the storage node is ping-able.

I am sure I am just missing something simple. Any ideas?

george1421

Just to be clear you have a working fog environment and then you upgraded and it broke or it never was able to function to begin with.

Why do you have a storage node and a fog master node? Are they on the same subnet?

On the network where the client computer is, what device is your dhcp server? What exactly do you have configured for dhcp option 66 {next-server} and dhcp option 67 {boot-file}?

thebrennan47

We have a new class of hardware and the version we have in production right now (Fog version 0.32) is not compatible. We are creating another server setup in parallel to test before we attempt to move everything over.

The storage node is currently set to also be the master node yes. We currently only have one. Once I get this working we plan on expanding further. The fog server is not currently on the same subnet. It is in a completely different building — I didn’t realize that might make a difference. I can definitely try that if it does

We are trying to use the storage node as the dhcp server. Below is the dhcpd.conf file contents

ddns-update-style none;

option domain-name "xyz.com";
option domain-name-servers ###.###.##.#,###.###.##.#;
option subnet-mask 255.255.255.0;

default-lease-time 2400;
max-lease-time 4800;

log-facility local7;

subnet 192.168.1.0 netmask 255.255.255.0 {
        range 192.168.1.50 192.168.1.100;
        option broadcast-address 192.168.1.255;
        option routers 192.168.1.1;
        include "/etc/dhcp/worker-reservations.conf";
        filename "pxelinux.0";
}

george1421

@thebrennan47 said in Host seems reachable by fog, but is not reaching PXE fog menu:

The storage node is currently set to also be the master node yes. We currently only have one. Once I get this working we plan on expanding further. The fog server is not currently on the same subnet. It is in a completely different building — I didn’t realize that might make a difference. I can definitely try that if it does
We are trying to use the storage node as the dhcp server. Below is the dhcpd.conf file contents

You keep saying / using contradicting terms (storage node). So for this discussion you only have one fog server (ignoring the fact that you have a 0.32 system). You are trying to use the FOG server as a dhcp server.

george1421

@thebrennan47 Just for clarity FOG has 2 install modes. The first is a Normal FOG server. This is a full installation of FOG. There is a storage node install too. That is a slave node to the FOG server. That storage node setup does not have a local database and relies on the FOG server or master node’s database.

george1421

@thebrennan47 Looking at your dhcp configuration, your first issue is that FOG no longer uses syslinux for booting (pxelinux) but uses iPXE. You need to change your boot file to undionly.kpxe for bios systems and ipxe.efi for uefi systems. Let me get the recommended dhcp configuration for FOG 1.3.0+

[Edit] Look at example #1 here: https://wiki.fogproject.org/wiki/index.php?title=BIOS_and_UEFI_Co-Existence#Example_1

thebrennan47

@george1421 I have made the change you suggested, however I am still missing something it seems like. I noticed that when I re-run installfog.sh on the storage node, it marks DHCP configuration as ‘skipped’. I imagine this may be the problem. Is there a way to change this to set up the storage node up as a dhcp server?

We are trying to set this up in the same way our current system is set up, which is basically going to be a fog server plus 3-4 storage nodes. Each storage node handles the local area connections for its ~15 workers

george1421

@thebrennan47 You can go in an tweak the .fogsettings file then rerun the installer. All originally selected settings are stored there.

Sebastian Roth

@thebrennan47 To me this sounds like you are trying to setup your new server as storage node only. This won’t work I’m afraid. As George already mentioned you need to select storage or master node when installing. Only the later will give you a full FOG server including DHCP, TFTP, NFS and such…

thebrennan47

@sebastian-roth @george1421

As sebastian suggested, I did a new installation and selected full install instead of just a storage node, which allowed me to select the options to configure DHCP.

After a couple of manual tweaks, my clients/hosts are now successfully being assigned IP addresses. So that is good news!

However it looks like the full installation does not have an option to hook up automatically to my fog server (Which is not this machine). I can do it manually, but I don’t know of another way to obtain the management password for the UI. Is there a way to do this? I believe this is the last step before I have this all hooked up.

I hacked up a quick diagram of our setup if that helps at all.

thebrennan47

@george1421 @Sebastian-Roth

Okay, so it looks like the management password is the same for all storage nodes? So I think I have gotten past that, unless there is something that the install script does that I am missing with the manual add. I attempted to do a capture operation, and I am still having an issue. Do you guys see anything wrong with my dhcpd.conf file?

Quick Edit: After some playing around it looks like the client machines are being assigned an IP correctly, but are not able to access the internet. It doesn’t seem like a DNS issue neccesarily because I cannot ping individual IP addresses either. I can only successfully ping the DHCP server. (IPv4 forwarding has also been enabled) So it seems like there is still something wrong, but I am getting close. Any help would be appreciated, thanks!

option space PXE;
option PXE.mtftp-ip code 1 = ip-address;
option PXE.mtftp-cport code 2 = unsigned integer 16;
option PXE.mtftp-sport code 3 = unsigned integer 16;
option PXE.mtftp-tmout code 4 = unsigned integer 8;
option PXE.mtftp-delay code 5 = unsigned integer 8;
option arch code 93 = unsigned integer 16;
use-host-decl-names on;
ddns-update-style interim;
ignore client-updates;

# option definitions common to all supported networks...
option domain-name "xxxx.xxxx.xxxx.com";
option domain-name-servers ###.###.12.30,###.###.12.8;

# Specify subnet of ether device you do NOT want service.
# For systems with two or more ethernet devices.
# subnet 136.165.0.0 netmask 255.255.0.0 {}
subnet 192.168.1.0 netmask 255.255.255.0{
    option subnet-mask 255.255.255.0;
    range dynamic-bootp 192.168.1.10 192.168.1.100;
    option broadcast-address 192.168.1.255;
    default-lease-time 21600;
    max-lease-time 43200;
    option routers 192.168.1.1;
    next-server 192.168.1.3;
    include "/etc/dhcp/worker-reservations.conf";
    class "Legacy" {
        match if substring(option vendor-class-identifier, 0, 20) = "PXEClient:Arch:00000";
        filename "undionly.kkpxe";
    }
    class "UEFI-32-2" {
        match if substring(option vendor-class-identifier, 0, 20) = "PXEClient:Arch:00002";
        filename "i386-efi/ipxe.efi";
    }
    class "UEFI-32-1" {
        match if substring(option vendor-class-identifier, 0, 20) = "PXEClient:Arch:00006";
        filename "i386-efi/ipxe.efi";
    }
    class "UEFI-64-1" {
        match if substring(option vendor-class-identifier, 0, 20) = "PXEClient:Arch:00007";
        filename "ipxe.efi";
    }
    class "UEFI-64-2" {
        match if substring(option vendor-class-identifier, 0, 20) = "PXEClient:Arch:00008";
        filename "ipxe.efi";
    }
    class "UEFI-64-3" {
        match if substring(option vendor-class-identifier, 0, 20) = "PXEClient:Arch:00009";
        filename "ipxe.efi";
    }
    class "SURFACE-PRO-4" {
        match if substring(option vendor-class-identifier, 0, 32) = "PXEClient:Arch:00007:UNDI:003016";
        filename "ipxe7156.efi";
    }
    class "Apple-Intel-Netboot" {
        match if substring(option vendor-class-identifier, 0, 14) = "AAPLBSDPC/i386";
        option dhcp-parameter-request-list 1,3,17,43,60;
        if (option dhcp-message-type = 8) {
            option vendor-class-identifier "AAPLBSDPC";
            if (substring(option vendor-encapsulated-options, 0, 3) = 01:01:01) {
                # BSDP List
                option vendor-encapsulated-options 01:01:01:04:02:80:00:07:04:81:00:05:2a:09:0D:81:00:05:2a:08:69:50:58:45:2d:46:4f:47;
                filename "ipxe.efi";
            }
        }
    }
}

george1421

@thebrennan47 Well IMO, I wouldn’t do it the way you did it (yes circle talking at its best).

I would still leave the storage nodes setup exactly as they were. The only thing you was missing is the dhcp server service, which can be added manually.

What you have is a hybrid solution that is not supported directly by the FOG Project, but that doesn’t mean it won’t work.

A storage node can pxe boot target computers, no problem. NFS is setup because it can deploy images no problem. The issue you will have is capturing images, that can only be done by the master node (not a storage node), unless you know a trick. The only thing missing is isc-dhcp which can be added from your distro’s repository. If you view the fog wiki page you can find a great example of the config file needed: https://wiki.fogproject.org/wiki/index.php?title=BIOS_and_UEFI_Co-Existence#Example_1

Now the only caveat with this setup is the caveat with any master node, storage node configuration. The storage nodes and the clients must be able to reach the FOG Master node or the storage node will become useless. The target computers only need to reach the master node if the FOG Client is installed. If the fog client isn’t installed in the reference image then only the storage nodes must be in contact with the master node.

Again, this setup configuration is not a supported configuration by the developers. But that doesn’t mean it won’t work as designed.

BTW: Your picture made it quite clear what you were doing, well done.

george1421

@thebrennan47 Sorry I had your post open from earlier and did not see your latest post.

Did I help you configure this setup? I know I helped someone in the last week or two with a similar setup where each storage node managed a single deployment bench. I only ask just to be sure I’m on the right track because I know it worked (routing) when I was done.

If I didn’t, to turn your storage node into a router, you need to enable the ip_forward kernel parameter, and then make it persistent with sysctl config file.

IN ADDITION, you have to remember that routing is a two way street. In your drawing you didn’t draw an ISP router at each location. If there is one there, you need to create a static route on your ISP router to describe the subnet that exists beyond your FOG-router. The default route for your FOG-router should be your ISP router. The default route for your target computers should be the deployment network interface on your fog server. This will give you the two way routing. At your HQ site, if your target computers need to communicate with the FOG Master node, you may need to create a static route on your ISP router to define each subnet at each remote location. This is a pure routing issue and not related to FOG imaging.

thebrennan47

Hey thanks @george1421 !

I was actually temporarily pulled off of this project for a few weeks – which is why this thread is so old. I don’t believe I am the one you are referring to if it was a couple of weeks ago.

Good news — I made some edits to the ip tables and it looks like the connection is now working in both directions. So it looks like I am getting very close! Thanks for the help with that

I once again tried a capture, but I got the following error. Any ideas?

Also, I looked at the file in a web browser and it contains the following:

#!ipxe
set fog-ip
set fog-webroot
set boot-url http://${fog-ip}/${fog-webroot}
cpuid --ext 29 && set arch x86_64 || set arch i386
goto get_console
:console_set
goto MENU
:alt_console
goto MENU
:get_console
console --picture http:///fog/service/ipxe/ --left 100 --right 80 && goto console_set || goto alt_console
:MENU
menu
item --gap Host is NOT registered!
item --gap -- -------------------------------------
choose --default fog.local --timeout 0 target && goto ${target}
:bootme
chain -ar http:///fog/service/ipxe/boot.php##params ||
goto MENU
autoboot

Also, the following is the TFTP setting in the Web UI for my Fog master server: /var/www/fog//service/ipxe/

george1421

@thebrennan47 Well now after looking at that picture my neck hurts…

There is a whole bunch of craziness in that picture.

What is host 192.168.1.1?
What is host 10.0.80.49?
And the last issue is the line with http///fog/service/ipxe in it. That should have an ip address

I see it tftp booting from 192.168.1.1 then connecting to 10.0.80.49 (which might be a fog server) but then the 10.0.80.49 gives it bad IP address. That should be the IP address of a fog server or a storage node and not be blank.

thebrennan47

@george1421 apologies for the crooked picture. Also I can clear up those ip addresses for you

10.0.88.49 is the DHCP/TFTP/Storage Node on interface #1
192.168.1.1 is the router address for interface #2

This seems correct I think? The weird thing is the file/directory does exist when I browse to it via a web browser
(http://10.0.88.49/fog/service/ipxe/boot.php)

george1421

@thebrennan47 It might help if you updated your picture with IP addresses but I’m going to guess what I see.

The FOG server management console is 10.0.80.49??

I could understand if 192.168.1.1 is on the deployment LAN for a storage node. Because dhcp would tell it to get the iPXE file from 192.168.1.1. The file its getting is default.ipxe. Inside the storage node 192.168.1.1 there is a file called /tftpboot/default.ipxe and it should point to the fog master server. That is where the boot.php is coming from. boot.php will determine what storage node to boot bzImage from. That is where its falling down.

Sebastian Roth

@thebrennan47 Good you posted the iPXE code output here. To me this looks as if FOG_WEB_HOST is not set, see set fog-ip on the second line is also empty. Please check in the FOG settings -> FOG configuration page on the web interface.

thebrennan47

@george1421 @Sebastian-Roth

10.0.88.49 was the DHCP / Storage Node. The fog management console is hosted at 10.0.88.42. I updated the default.ipxe file to point to the correct server and that fixed it! Thanks for the help with that

After a reboot on my client, I now get into the fog upload page and the process begins; but quickly terminates do to a mount failure for the /images folder. I am guessing it is a permissions issue? I am currently looking into but I figure you guys may have seen this one before so I wanted to ask real quick. Sorry for the spam of questions on this thread… I am getting really close to getting this working though it looks like

george1421

@thebrennan47 Boy, you are just full of good news.

OK what is 10.2.102.49? You keep throwing new IP addresses at this poor pxe booting client?

Host seems reachable by fog, but is not reaching PXE fog menu

Server

Client

Description

247

12.0k

17.3k

155.2k