Posts made by mkstreet

@Sebastian-Roth I will try this in about ten days when we are back in session. Thanks!!

@george1421 I have forwarded the first wish to the network area. In am waiting a response.

@george1421

OK. Two questions.

1. Instead of using DNSMASQ, if I attempt to have the network changed (i.e. 172.16.1.1 and/or 10.0.253.1), what exactly needs to be changed? I will need to explain what I want in almost step by step fashion to non-native English speakers who control the network.

2. Is there an alternative where I disconnect from the network just while I am loading the lab? Then, I am guessing, my Fog Server will need to have isc-dhcp running (instead of dnsmasq)…?

@george1421

1. What device is your pxe boot client?
   ** Inside my lab, we have about 36 Lenovo ThinkPad model M72z
   We have been using and loading these with Fog for about 2 years up until recently.

2. Do you have a different brand/model of device you could use?
   ** Yes. I have “regular” desktop PC’s which are outside the lab.
   I have just tried to network boot one of those. The case says its ACER Veriton.
   I see that it connects to 10.0.253.23 and then gets the same PXE-E78 error as the boxes inside the lab. I have attached a pcap of this here:
   http://s000.tinyupload.com/?file_id=48129942273982229370

3. You did mention that you moved this FOG server from an isolated network to this new network. What device was supplying the DHCP services for that isolated network?
   ** In brief, my department (LAB and offices) had our own Internet connection that was separate from the rest of the campus. Although this seems very odd, the ISP (which was essentially the state enterprise telecom entity) provided DHCP services from their central office about 2 kms away. I know it’s strange but that is what it was. This was the network configuration when we initially began using Fog in about March 2014. Once we had Fog initially setup and had gotten some experience with Fog, we had few/none network issues. We knew even less about that network/DHCP setup. From March 2014 to about August 2015, we very actively used Fog and loaded our entire lab at least ten times, and had separate images for our (non-Lab) office desktop PC’s that we managed with Fog too. From August 2015 until now, we have not used Fog much as we didn’t need to update things. About a month ago, my department gave up our separate connection to the state telecom provider, and joined the campus0wide LAN/network that the rest of the campus had just implemented (a different provider too than the state telecom we had been using). Now, we want to update and reload our lab. This was when we discovered that connecting to the Fog server and loading boxes, which we had reached the point of just-turn-on-and-take-for-granted, no longer could find the Fog server. This is what led to the path of attempting dnsmasq and voila here we are today.

@Sebastian-Roth

(Reply Part 2)

I’ve disabled 10.0.253.24 (Ubu 14.04, Fog 3121) and started up 10.0.253.23 (Ubu 16.04, Fog 1.3)

You are right that this box, which I just setup yesterday, hadn’t been made static. I’ve changed that now. After making the changes to the intefaces file, I rebooted the machine.

Here is the interfaces file, before and after:

before

ubucomlab@ubucomlab-ThinkCentre-M72z:/etc/network$ cat orig_interfaces 
# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopback
ubucomlab@ubucomlab-ThinkCentre-M72z:/etc/network$ 

after

ubucomlab@ubucomlab-ThinkCentre-M72z:/etc/network$ cat interfaces
# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopback

auto enp1s0
iface enp1s0 inet static
address 10.0.253.23
netmask 255.255.255.0
gateway 10.0.253.1
dns-nameservers 110.164.252.222 8.8.8.8

ifconfig output:

ubucomlab@ubucomlab-ThinkCentre-M72z:/etc/network$ ifconfig
enp1s0    Link encap:Ethernet  HWaddr f8:0f:41:a0:04:59  
          inet addr:10.0.253.23  Bcast:10.0.253.255  Mask:255.255.255.0
          inet6 addr: fe80::fa0f:41ff:fea0:459/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5100 errors:0 dropped:8 overruns:0 frame:0
          TX packets:2983 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:5067659 (5.0 MB)  TX bytes:433865 (433.8 KB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:760 errors:0 dropped:0 overruns:0 frame:0
          TX packets:760 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:117502 (117.5 KB)  TX bytes:117502 (117.5 KB)

ubucomlab@ubucomlab-ThinkCentre-M72z:/etc/network$ 

status for isc-dhcp

ubucomlab@ubucomlab-ThinkCentre-M72z:/etc/network$ sudo service isc-dhcp-server status
● isc-dhcp-server.service
   Loaded: not-found (Reason: No such file or directory)
   Active: inactive (dead)
ubucomlab@ubucomlab-ThinkCentre-M72z:/etc/network$ 

I’ve just tried to boot the host box and got the same PXE-E78 error.
Here is a pcap from that:
http://s000.tinyupload.com/?file_id=00695804426205933233

I’ve commented out the dchp-no-override in the ltsp.conf and restarted dnsmasq.
I get the PXE-E78 error.
Here is the pcap for this:
http://s000.tinyupload.com/?file_id=03577227175846412978

Here is my ltsp.conf

ubucomlab@ubucomlab-ThinkCentre-M72z:/etc/dnsmasq.d$ cat ltsp.conf
# Don't function as a DNS server:
# port=0

# Log lots of extra information about DHCP transactions.
log-dhcp

# Dnsmasq can also function as a TFTP server. You may uninstall
# tftpd-hpa if you like, and uncomment the next line:
# enable-tftp

# Set the root directory for files available via FTP.
tftp-root=/tftpboot

# The boot filename, Server name, Server Ip Address
dhcp-boot=undionly.kpxe,10.0.253.23,10.0.253.23

# rootpath option, for NFS
#dhcp-option=17,/images

# kill multicast
#dhcp-option=vendor:PXEClient,6,2b

# Disable re-use of the DHCP servername and filename fields as extra
# option space. That's to avoid confusing some old or broken DHCP clients.
#
# MKS 07-Oct-2016
#dhcp-no-override

# PXE menu.  The first part is the text displayed to the user.  The second is the timeout, in seconds.
pxe-prompt="Press F8 for boot menu", 3

# The known types are x86PC, PC98, IA64_EFI, Alpha, Arc_x86,
# Intel_Lean_Client, IA32_EFI, BC_EFI, Xscale_EFI and X86-64_EFI
# This option is first and will be the default if there is no input from the user.
pxe-service=X86PC, "Boot from network", undionly, 10.0.253.23

# A boot service type of 0 is special, and will abort the
# net boot procedure and continue booting from local media.
#pxe-service=X86PC, "Boot from local hard disk", 0

# If an integer boot service type, rather than a basename is given, then the
# PXE client will search for a suitable boot service for that type on the
# network. This search may be done by multicast or broadcast, or direct to a
# server if its IP address is provided.
# pxe-service=x86PC, "Install windows from RIS server", 1

# This range(s) is for the public interface, where dnsmasq functions
# as a proxy DHCP server providing boot information but no IP leases.
# Any ip in the subnet will do, so you may just put your server NIC ip here.
# Since dnsmasq is not providing true DHCP services, you do not want it
# handing out IP addresses.  Just put your servers IP address for the interface
# that is connected to the network on which the FOG clients exist.
# If this setting is incorrect, the dnsmasq may not start, rendering
# your proxyDHCP ineffective.
dhcp-range=10.0.253.23,proxy

# This range(s) is for the private network on 2-NIC servers,
# where dnsmasq functions as a normal DHCP server, providing IP leases.
# dhcp-range=192.168.0.20,192.168.0.250,8h

# For static client IPs, and only for the private subnets,
# you may put entries like this:
# dhcp-host=00:20:e0:3b:13:af,10.160.31.111,client111,infinite

On this box, I did try to check iptables and got this:

ubucomlab@ubucomlab-ThinkCentre-M72z:/etc/dnsmasq.d$ iptables -L
modprobe: ERROR: could not insert 'ip_tables': Operation not permitted
iptables v1.6.0: can't initialize iptables table `filter': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.
ubucomlab@ubucomlab-ThinkCentre-M72z:/etc/dnsmasq.d$ 

And for netstat:

ubucomlab@ubucomlab-ThinkCentre-M72z:/etc/dnsmasq.d$ netstat -an | grep 4011
udp        0      0 0.0.0.0:4011            0.0.0.0:*                          
ubucomlab@ubucomlab-ThinkCentre-M72z:/etc/dnsmasq.d$ 

@Sebastian-Roth

Yesterday, I did send a pcap from 10.0.253.24 (Ubu 14.04/Fog 3121).
This is my original Fog setup. It is setup with static IP.
Here is the link to yesterdays pcap from that box:
http://s000.tinyupload.com/?file_id=29844646319354557181

In addition, I created another pcap today for Tom Elliott from that box . Here is the link to that pcap:
http://s000.tinyupload.com/?file_id=06163152893454480790

I will now go check the settings on 10.0.253.23 (Ubu 16.xx / Fog 1.3 release candidate 11).
I suspect you are correct that this one is not static, but I will check that now and post information about that box (and pcap).

@Tom-Elliott

RE: WOL.
Yes, I meant Wake On LAN by WLON.
This used to work but doesn’t work now.
Yesterday, I offered this information and the pcap as additional information that might help uncover something.

RE: dchp-no-override
For DHCP-NO-OVERRIDE, I commented this out in ltsp.conf and restarted dnsmasq … on 10.0.253.24 (Ubuntu 14.04, Fog 3121) and it had no effect.

I captured a pcap of this:
http://s000.tinyupload.com/?file_id=06163152893454480790

Should I put this ltsp.conf option back or leave commented out?

@george1421

It’s a good question about isc-dhcp. I haven’t ever been using it as far as I know.
But I checked this morning after booting the lab

For 10.0.253.24 (Ubun 14.04, Fog 3121), I get the following:

compteach@iepcomlabsrv:~$ sudo service isc-dhcp-server status
isc-dhcp-server stop/waiting
compteach@iepcomlabsrv:~$
compteach@iepcomlabsrv:~$
compteach@iepcomlabsrv:~$ sudo service isc-dhcp-server stop
stop: Unknown instance:
compteach@iepcomlabsrv:~$
compteach@iepcomlabsrv:~$ sudo service isc-dhcp-server status
isc-dhcp-server stop/waiting
compteach@iepcomlabsrv:~$

@george1421

To experiment, I used the new Fog 1.3 install to contact that same PC with a hardware inventory request from Fog.

I noticed that…

1. The WLON did not happen. In the past, WLON worked.
2. When I manually turned that PC, the same results about finding the Fog server and press F8. When I did, I got the same PXE-E78 error and in Fog the active task showed the hardware inventory as still in progress.

I thought that this would be a way to attempt communication that did not involve TFTP boot.

I created a pcap file using the same command you gave me before
(sudo tcpdump -w output.pcap port 67 or port 68 or port 69 or port 4011)
but I don’t know if the port filters on this are suitable to capture needed info for the WLON/hardware inventory tasks…

This pcap file is at:
http://s000.tinyupload.com/?file_id=61578123182931059079

@george1421

On another box, I installed Ubuntu 16.04 LTS and Fog 1.3.0-RC-11.

I shut down the dnsmasq on the 10.0.253.24 box and started dnsmasq on the new box with an ltsp.conf etc.

I get the same behavior.

The PC boots and finds the internal DHCP (171.xxxx) and gets to the new box whose IP is 10.0.253.23.

It prompts me for F8 to boot from the network. Then I get:

UD 10.0.253.23

Which times out with the PXE-E78 error.

I captured a new pcap for this in case this is helpful.
This pcap is at:
http://s000.tinyupload.com/?file_id=97921552308199994673

@george1421

OK. The problem with dns and resolv.conf seems ok now. I am able to do apt-get updates and ping external places such as google.com. Oddly, the resolv.conf just shows the loopback:

compteach@iepcomlabsrv:/etc$ cat resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.0.1
compteach@iepcomlabsrv:/etc$
compteach@iepcomlabsrv:/etc$ ping google.com
PING google.com (110.164.6.251) 56(84) bytes of data.
64 bytes from mx-ll-110.164.6-251.static.3bb.co.th (110.164.6.251): icmp_seq=1 ttl=55 time=2.59 ms
64 bytes from mx-ll-110.164.6-251.static.3bb.co.th (110.164.6.251): icmp_seq=2 ttl=55 time=2.58 ms
64 bytes from mx-ll-110.164.6-251.static.3bb.co.th (110.164.6.251): icmp_seq=3 ttl=55 time=2.75 ms
^C
--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 2.588/2.644/2.751/0.095 ms

I made the changes to the ltsp.conf
a) comment out the port=0
b) change the dhcp-boot
c) change pxe-service

How every there is no change in the behavior.

I captured a new pcap and posted it here:
http://s000.tinyupload.com/?file_id=29844646319354557181

And for completeness, here is my ltsp.conf:

compteach@iepcomlabsrv:/etc/dnsmasq.d$ cat ltsp.conf
# Don't function as a DNS server:
# MKS  06-Oct-2016
#port=0

# Log lots of extra information about DHCP transactions.
log-dhcp

# Dnsmasq can also function as a TFTP server. You may uninstall
# tftpd-hpa if you like, and uncomment the next line:
# enable-tftp

# Set the root directory for files available via FTP.
tftp-root=/tftpboot

# The boot filename, Server name, Server Ip Address
dhcp-boot=undionly.kpxe,10.0.253.24,10.0.253.24

# rootpath option, for NFS
#dhcp-option=17,/images

# kill multicast
#dhcp-option=vendor:PXEClient,6,2b

# Disable re-use of the DHCP servername and filename fields as extra
# option space. That's to avoid confusing some old or broken DHCP clients.
dhcp-no-override

# PXE menu.  The first part is the text displayed to the user.  The second is the timeout, in seconds.
pxe-prompt="Press F8 for boot menu", 3

# The known types are x86PC, PC98, IA64_EFI, Alpha, Arc_x86,
# Intel_Lean_Client, IA32_EFI, BC_EFI, Xscale_EFI and X86-64_EFI
# This option is first and will be the default if there is no input from the user.
pxe-service=X86PC, "Boot from network", undionly, 10.0.253.24

# A boot service type of 0 is special, and will abort the
# net boot procedure and continue booting from local media.
#pxe-service=X86PC, "Boot from local hard disk", 0

# If an integer boot service type, rather than a basename is given, then the
# PXE client will search for a suitable boot service for that type on the
# network. This search may be done by multicast or broadcast, or direct to a
# server if its IP address is provided.
# pxe-service=x86PC, "Install windows from RIS server", 1

# This range(s) is for the public interface, where dnsmasq functions
# as a proxy DHCP server providing boot information but no IP leases.
# Any ip in the subnet will do, so you may just put your server NIC ip here.
# Since dnsmasq is not providing true DHCP services, you do not want it
# handing out IP addresses.  Just put your servers IP address for the interface
# that is connected to the network on which the FOG clients exist.
# If this setting is incorrect, the dnsmasq may not start, rendering
# your proxyDHCP ineffective.
dhcp-range=10.0.253.24,proxy

# This range(s) is for the private network on 2-NIC servers,
# where dnsmasq functions as a normal DHCP server, providing IP leases.
# dhcp-range=192.168.0.20,192.168.0.250,8h

# For static client IPs, and only for the private subnets,
# you may put entries like this:
# dhcp-host=00:20:e0:3b:13:af,10.160.31.111,client111,infinite
#dhcp-host=f8:0f:41:a0:04:75,net:allow
#dhcp-ignore=#allow

@george1421

Hmmm… If I understand this correctly, then I cannot disable dhcp relay within 10.0.253.1 as other hardware in this subnet but outside my lab would still need dhcp service for other purposes.
And, as you say, this path is getting messy.

As for changing the main dhcp option 66, I could try to request this. Would this affect only my subnet or our whole facility? My lab is about 90% of my subnet, but the main dhcp is servicing the whole campus which is comprised of several subnets… If the option 66 will affect others outside my area, then it is hard for me to do.

I am setting up the new version of FOG etc under VirtualBox. I think I will complete that, as it is a new clean install. I will see if this clean start resolves anything, as opposed to this – attempting to add DNSMASQ to an existing setup that (was) working.

@george1421

compteach@iepcomlabsrv:~$ netstat -an|grep 4011
udp        0      0 0.0.0.0:4011            0.0.0.0:*
compteach@iepcomlabsrv:~$

@george1421

I did some experimenting and noticed this anomaly.

The Fog server and the host I want to load are on two switches which are connected together.
When I attach the Ethernet cable to the outside LAN / Internet to one of those switches, then I get the behavior as noted. Meaning the DHCP gets answered but when I press F8 to network boot, I get the PXE-E78 error.

When I disconnect that cable to the outside, then the PC I want to load can no longer find the DHCP. So something external to the LAB must be helping get the initial requests to 10.0.253.24 ?

I found this…

compteach@iepcomlabsrv:/etc$ sudo iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

@george1421

For the /etc/dnsmasq.conf, the last lines are NOT commented out as shown here:

# For debugging purposes, log each DNS query as it passes through
# dnsmasq.
#log-queries
# MKS 04-Oct-2016
log-queries

# Log lots of extra information about DHCP transactions.
#log-dhcp
# MKS 04-Oct-2016
log-dhcp

# Include another lot of configuration options.
#conf-file=/etc/dnsmasq.more.conf
#conf-dir=/etc/dnsmasq.d
# MKS 04-Oct-2016
conf-dir=/etc/dnsmasq.d

I am not familiar with “iptables firewall” – where should I look?

@george1421

Ok I changed /etc/network/interfaces and did a reboot (to be sure my changes were implemented):

compteach@iepcomlabsrv:/etc/network$ cat interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto em1
iface em1 inet static
address 10.0.253.24
netmask 255.255.255.0
gateway 10.0.253.1
dns-nameservers 110.164.252.222 8.8.8.8

However, I still seem to have unknown hosts and the resolv.conf just has the loopback address:

compteach@iepcomlabsrv:/etc/network$ sudo service dnsmasq status
 * Checking DNS forwarder and DHCP server dnsmasq                                                                                                                       * (running)
compteach@iepcomlabsrv:/etc/network$
compteach@iepcomlabsrv:/etc/network$
compteach@iepcomlabsrv:/etc/network$ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.0.1
compteach@iepcomlabsrv:/etc/network$ ping google.com
ping: unknown host google.com
compteach@iepcomlabsrv:/etc/network$

Do I need to do anything with the /etc/dnsmasq.conf file ?

After sudo service dnsmasq stop, then /etc/resolv.conf has

compteach@iepcomlabsrv:/etc$ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 110.164.252.222
nameserver 8.8.8.8

But when I start … sudo service dnsmasq start then the /etc/resolv.conf has, as you said,:

compteach@iepcomlabsrv:/etc$ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.0.1

I am not sure how to fix this because the /etc/resolv.conf is dynamic and overwritten (as the files own comments state). Do I list the internal DNS in the /etc/network/interfaces ?

compteach@iepcomlabsrv:/etc/network$ cat interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto em1
iface em1 inet dhcp

RE: LTSP.CONF
I’ve removed the two lines at the end of ltsf.conf.
And did a sudo service dnsmasq restart
the boot results are the same… meaning I get the PXE-E78 error…etc

RE: BIOS.
I looked in the BIOS settings. It says Boot Mode Auto, and Boot Priority Legacy First.
I haven’t changed these BIOS settings in probably over a year and it was working before with this FOG server…

I’ve just noticed too when I tried to do sudo apt-get update that it seems the 10.0.253.24 cannot resolve hostnames because the apt-get update fails because it cannot locate the update mirror.
Further, something like ping google.com cannot find the host either.

But if I do…
sudo service dnsmasq stop

Then everything is golden. Can apt-get update, can ping google.com, etc.

I should make clear that before this network change that led to me attempting DNSMASQ,
this Fog server was working fine. So, is my problem a Fog configuration setting that needs to change to recognize working with DNSMASQ?

In other words, DNSMASQ is newly added to a Fog server that had been working before…