Host Startup; Booting into LVM Disk Fails

Wayne Workman

What is the target hardware? What version is it’s firmware? Make and Model please. What boot file are you using for DHCP options 066/067 next-server and filename.

dholtz-docbox

@Wayne-Workman : Haha, that’s funny. Reading that, I am like… I think that person was me!

That resolved the issue when I deployed the captured LVM image to the machine. Unfortunately, when LVM installs, I get these wonky boot options in my BIOS and then it’s like the system has no idea how to boot anymore from SAN. I have discussed it with my team, and we are going to nix LVM next release. It’s just not something we can do this release, as it was something I discovered going through FOG and becoming more intimate with the system at this level.

As far as what /dev/sda is, it’s the data drive. I guess the way this machine was setup - hardware wise - its first drive is the data drive and the second drive is the system drive. Upon discovering this, I asked why, since it only seems to complicate matters. The answer I received was that they tried mounting them different, but Linux always wanted to install them as such - mounting the smaller drive to /dev/sdb instead of /dev/sda. This is kind of where I stepped in and said that doesn’t seem right, can’t we just swap the SATA cables or define it differently when we setup the partitions? It hit an expertise-wall after that, as no one has the domain knowledge for any of this - which is a part of why I am so eager to ask so many questions. Knowing what I know now, I am sure we can handle this ourselves, but we know so little surrounding this that I am cobbling it together while building an insane amount of other tools and tech in parallel.

I had issues copying the drive as EXT4 in the past too. The swap drive would always be mis-aligned I believe, and the system would create a new one and then I’d have fstab issues always. I guess at the end of the day, I don’t feel I have successfully captured and deployed an image, given all the little things. Now I just wonder what I am doing right and doing wrong, given all the successes and failures I have had.

Let me dig up the information you requested. It might be a tomorrow-thing as I am going on 12-hours already and need to get home for misc. life things.

-Dustin

dholtz-docbox

PS> I think what trips me up is, I am just trying to find one happy path, and I don’t think that has been revealed yet to me (with our hardware, at least). It’s mind boggling to me the things that do work, when they work, and the things that do not work, and for reasons I am not equipped-enough to handle yet. Even now, understanding as much as I do, I circle back and ask simple questions, such as, why does the swap partition fail to image when deploying an EXT formatted drive. Or, why can I set the system up for LVM, capture and deploy the disk, and only then am I able to boot into ‘/dev/sdb’ with the settings I provided; however, fails to do a simple SANBOOT.

Also, here’s an image of the boot drives post-installation of Ubuntu…
bios_boot

dholtz-docbox

Yeah, I have to head home for now. I will get this information ASAP tomorrow or tonight if I can.

-Dustin

Wayne Workman

@dholtz-docbox said in Host Startup; Booting into LVM Disk Fails:

I had issues copying the drive as EXT4 in the past too. The swap drive would always be mis-aligned I believe, and the system would create a new one and then I’d have fstab issues always.

This is known about. Here’s lots of info on that, and options too:
https://forums.fogproject.org/topic/8549/fog-1-3-rc8-fails-to-apply-swap-uuid-on-image-deploy

Tom Elliott

@dholtz-docbox San boot releases to the first found HDD, or sda in your case. Exit should release to bios which then finds the disk with the right boot active flag. I’d recommend trying exit first. If this doesn’t work then try grub, though it defaults to first HDD. The best grub would be first found windows but these systems aren’t windows systems.

dholtz-docbox

@Tom-Elliott : That’s what I started to think last night, so that makes sense. I wanted EXIT to work as you described, but it throws a chainloading error each time. I am actually going to… just install the drive as EXT4. I mean… /dev/sdb is just /dev/sdb, one way or the other, right? I am hoping the system works fine, and then I can just do single disk targeting the second drive, hoping that it is more reliable.

Wayne Workman

@dholtz-docbox said in Host Startup; Booting into LVM Disk Fails:

I wanted EXIT to work as you described, but it throws a chainloading error each time.

A chainloading error is better than the other stuff you were describing. Let’s focus on that, please provide a photo of the error, and the MAC address of the computer you tried on, and the FOG Server’s IP address.

dholtz-docbox

No problem. Let me re-install the system with LVM - I was trying other things in the interim. I will provide an image as soon as that is complete.

Wayne Workman

@dholtz-docbox said in Host Startup; Booting into LVM Disk Fails:

Let me re-install the system with LVM

Nah, please go with Ext4. We will explore the EXIT option with that setup.

dholtz-docbox

Unfortunately, I have to move forward with the LVM installation for now. Given the time until our next release, it is too risky to greenlight the change to EXT. That said, I had the same issue with both SANBOOT and EXIT w/ the EXT install, so the problem is similar I feel - wild guess.

I was also hopeful that installing EXT would eliminate the extra boot options in my BIOS. It seems that was not the case, so maybe they are related to Server Edition instead? I just find it odd that, when installing Ubuntu to the disk, that I can no longer boot from the disk and must boot from one of those Ubuntu options. Which one also has relevance, as the other one appears to have no operating system on it. Perhaps it’s the swap?

As far as the other information…

host mac => 08:60:6e:fa:05:af
server ip => 10.1.10.42

and here’s an image of the boot output…
boot_pxe

Wayne Workman

@dholtz-docbox Ok, in your browser click on this link and give us a copy of the output:

10.1.10.42/fog/service/ipxe/boot.php?mac=08:60:6e:fa:05:af

It appears you have your DHCP server setup for FOG, and you also have dnsmasq setup for FOG? This isn’t necessary. Choose one or the other, I’d suggest using the full DHCP server.

And what boot file are you using again? I don’t think you listed that earlier. You can quickly find this just by looking at what you have set in DHCP option 067.

dholtz-docbox

@Wayne-Workman :

#!ipxe
set fog-ip 10.1.10.42
set fog-webroot fog
set boot-url http://${fog-ip}/${fog-webroot}
exit

dholtz-docbox

As far as whether I have DHCP setup for FOG and DNSMasq, I should only have DNSMasq? Is there a way to validate this? The .fogsettings for the server are as follows…

## Start of FOG Settings
## Created by the FOG Installer
## Version: 1.3.0-RC-11
## Install time: Fri 30 Sep 2016 05:26:11 PM EDT
ipaddress='10.1.10.42'
interface='eth0'
submask='255.255.255.0'
routeraddress='10.1.10.1'
plainrouter='10.1.10.1'
dnsaddress='#   No dns added'
username='fog'
password='{password}'
osid='2'
osname='Debian'
dodhcp='N'
bldhcp='0'
dhcpd=''
blexports='1'
installtype='N'
snmysqluser='root'
snmysqlpass=''
snmysqlhost='localhost'
installlang='0'
donate='0'
storageLocation='/images'
fogupdateloaded=1
docroot='/var/www/'
webroot='/fog/'
caCreated='yes'
startrange=''
endrange=''
bootfilename='undionly.kpxe'
packages='apache2 bc build-essential cpp curl g++ gawk gcc gzip htmldoc lftp libapache2-mod-php5 libc6 libcurl3 m4 mysql-client mysql-server net-tools nfs-kernel-server openssh-server php5 php5-cli php5-curl php5-fpm php5-gd php5-json php5-ldap php5-mcrypt php5-mysqlnd php-gettext sysv-rc-conf tar tftpd-hpa tftp-hpa vsftpd wget xinetd zlib1g '
noTftpBuild=''
notpxedefaultfile=''
sslpath='/opt/fog/snapins/ssl/'
backupPath='/home/'
php_ver='5'
php_verAdds='-5.6'
sslprivkey='/opt/fog/snapins/ssl//.srvprivate.key'
## End of FOG Settings

Then I have the following dnsmasq.conf…

# Don't function as a DNS server:
port=0

# Log lots of extra information about DHCP transactions.
log-dhcp

# Set the root directory for files available via FTP.
tftp-root=/tftpboot

# The boot filename, Server name, Server Ip Address
dhcp-boot=undionly.kpxe,,10.1.10.42

# Disable re-use of the DHCP servername and filename fields as extra
# option space. That's to avoid confusing some old or broken DHCP clients.
dhcp-no-override

# PXE menu.  The first part is the text displayed to the user.  The second is the timeout, in seconds.
pxe-prompt="Booting FOG Client", 0

# The known types are x86PC, PC98, IA64_EFI, Alpha, Arc_x86,
# Intel_Lean_Client, IA32_EFI, BC_EFI, Xscale_EFI and X86-64_EFI
# This option is first and will be the default if there is no input from the user.
pxe-service=X86PC, "Boot to FOG", undionly
pxe-service=X86-64_EFI, "Boot to FOG UEFI", ipxe

dhcp-range=10.1.10.1,proxy

Edit> I would be led to believe that only DNSMasq should be handled because of…

dodhcp='N'
bldhcp='0'

… no?

dholtz-docbox

You had me thinking…

One of the other VM’s on the network I used, at one point, to do an installation, where it setup using FOG’s DHCP. Is it possible something is still lingering on this machine? I am going to shut it off and give it a whack, in the mean time…

Edit> Nope. No difference.

Wayne Workman

@dholtz-docbox said in Host Startup; Booting into LVM Disk Fails:

… no?

Read what your screenshot says.
“Duplicate option 66 (next server) from DHCP proxy and DHCP server.”

This means your DHCP server is configured for FOG, and dnsmasq is configured for FOG. dnsmasq will refuse to run on a box that is also serving DHCP, so these two things are on two different systems. You don’t need both, and a full DHCP server is superior by far.

Use wireshark with the bootp filter to determine where the other DHCP server is, and do some ipconfig /release /renew commands to get them to respond - but you should already know where it is really.

Wayne Workman

@dholtz-docbox said in Host Startup; Booting into LVM Disk Fails:

@Wayne-Workman :

#!ipxe
set fog-ip 10.1.10.42
set fog-webroot fog
set boot-url http://${fog-ip}/${fog-webroot}
exit

So, here’s the output my server gives for an unregistered host (the mac you gave), and the output of a registered host I have here. I’m on 1.3.0 RC-14.

#!ipxe
set fog-ip 10.2.1.11
set fog-webroot fog
set boot-url http://${fog-ip}/${fog-webroot}
cpuid --ext 29 && set arch x86_64 || set arch i386
iseq ${platform} efi && set key 0x1b || set key 0x01
iseq ${platform} efi && set keyName ESC || set keyName CTRL + A
prompt --key ${key} --timeout 4000 Booting... (Press ${keyName} to access the menu) && goto menuAccess || sanboot --no-describe --drive 0x80
:menuAccess
login
params
param mac0 ${net0/mac}
param arch ${arch}
param platform ${platform}
param username ${username}
param password ${password}
param menuaccess 1
param debug 1
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
:bootme
chain -ar http://10.2.1.11/fog/service/ipxe/boot.php##params

#!ipxe
set fog-ip 10.2.1.11
set fog-webroot fog
set boot-url http://${fog-ip}/${fog-webroot}
cpuid --ext 29 && set arch x86_64 || set arch i386
iseq ${platform} efi && set key 0x1b || set key 0x01
iseq ${platform} efi && set keyName ESC || set keyName CTRL + A
prompt --key ${key} --timeout 4000 Booting... (Press ${keyName} to access the menu) && goto menuAccess || sanboot --no-describe --drive 0x80
:menuAccess
login
params
param mac0 ${net0/mac}
param arch ${arch}
param platform ${platform}
param username ${username}
param password ${password}
param menuaccess 1
param debug 1
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
:bootme
chain -ar http://10.2.1.11/fog/service/ipxe/boot.php##params

Have you modified any files yourself or modified your iPXE boot menu at all?

dholtz-docbox

@Wayne-Workman :

Yeah, I was wondering about that line - it has been bothering me. I have never used wireshark, so let me grab it and look around.

You have a lot more output

I have not edited anything though; I have only run the installfog.sh script over the existing installation.

Let me go and determine who else is serving DHCP real quick.

Edit> Which machine should I be using wireshark on? I’m not quite sure how to use the tool in the requested manner.
Edit> Nevermind. Wireshark was starting up with one of its windows resized to its minimum height - the window which lists the packets. I am looking through it now.
Edit>
0_1476989636971_upload-8ae29e1f-ba2f-41dd-9538-05e72c0cd3d0

Am I right in looking at this, in that, both 10.1.10.1 and 10.1.10.42 are serving DHCP, where only, w/ DNSMasq, 10.1.10.1 should be serving it?

Edit> I believe that is right, reading into how DHCP works. There should only be one DHCP Offer, correct? Both 10.1.10.1 and 10.1.10.42 offering would indicate that 10.1.10.42 is setup, somehow, to serve its own DHCP? I guess my next question is, if my dnsmasq.conf is correct, what is lingering that is causing 10.1.10.42 to also serve DHCP.

george1421

@dholtz-docbox TBH: I did not read the entire thread here, but your dhcp process looks normal if you are running dnsmasq in dhcpPROXY mode. In that case your primary dhcp server is 10.1.10…42 and 10.1.10.1 is the dnsmasq.

Looking down a bit more 10.1.10.1 is giving the ack so that tells me that 10.1.10.1 is your dhcp server. That way I rad this pcap is that you DO have two dhcp servers on your subnet. If .10.1 is your defined dhcp server for your subnet, you need to understand what .10.42 is doing. It should not be issuing an Offer if it has nothing to offer the client sending the discover.

dholtz-docbox

@george1421 : I am not sure if our terminology is in sync, but to clarify… the router which handles DHCP is 10.1.10.1 and the FOG Server is 10.1.10.42. When you say “primary DHCP”, from whose point-of-view is that? My current assumption is that the primary DHCP would be the one serving IP’s; where the FOG Server would be the DNSMasq?

Further, I agree, 10.1.10.42 should NOT be offering anything. But when I look at Wireshark, 10.1.10.42 the one with a boot filename, where 10.1.10.1 has no real supporting information.

That said… does my dnsmasq.conf need to be flipped around, such that…

dhcp-boot=undionly.kpxe,,10.1.10.1

and…

dhcp-range=10.1.10.42,proxy

… or something like that? Curious if I had something flipped around incorrectly.

Host Startup; Booting into LVM Disk Fails

177

11.6k

17.1k

154.6k