Host Startup; Booting into LVM Disk Fails

[Wayne Workman]

http://askubuntu.com/questions/371049/how-are-dev-sda-and-dev-sdb-chosen

According to that, you might want to remove one of the disks for Ubuntu’s installation only, then add it back afterwards and see how that goes. And also as that link suggests, use UUIDs instead of /dev/sdX, follow the documentation.

https://help.ubuntu.com/community/UsingUUID

[george1421]

@dholtz-docbox outside of your design requirements, the one question in my mind is if you did switch the cables (as a test) does it pxe boot properly to the OS. That way we can just focus on getting the proper exit to work/boot to the second hard drive. I know I’ve seen it in the past change the boot drive, but I can’t remember what I did to do that until I can get my hands on a fog server.

[dholtz-docbox]

I am looking to have someone open the enclosure so I can swap the cables - darn special screwdriver that I don’t have. Until then, I am looking at how to handle how /dev/sda is assigned, and it looks like it comes down to udev rules? That’s what I am looking at right now. I know a lot of this is hitting a bridge I was looking to prolong, but I have been looking to more intimately setup the partitions. I guess this is the start of understanding how some of this is handled.

[george1421]

@dholtz-docbox Remember at this point in time you are NOT in any OS. When the iPXE menu exits you are still running the iPXE kernel. It knows nothing about your target OS. Its just passing (or trying to) pass off boot control some some boot device.

[Wayne Workman]

George is right on that.

[george1421]

@george1421 OK I have a hackish way to test this idea.

But first, I went through each of the exit modes and all pick the first hard drive. Looking into the code all of the exit modes are hard coded to boot to the first hard disk.

Sooo… as a test please edit this file /var/www/html/fog/lib/fog/bootmenu.class.php search for some fragment of “sanboot --no-describe --drive 0x80” it should be listed only once in that file. Change the 0x80 to 0x81 then file and save the class file. Be sure that you have the host set to sanboot and then use the browser trick I chatted to you to confirm that the sanboot is now sending to the 0x81 disk. Once confirmed try to boot that target computer again. If it works then we have to find the next steps, but we can prove the process works.

[dholtz-docbox]

That makes sense. I guess my thought here, is, before trying to boot through network, after installing Ubuntu, I run a set of udev rules, this remapping which disk /dev/sda and /dev/sdb are associated with.

Also, I can’t switch the cables around, one drive is SATA where the other is… like an onboard PCI-e disk. I wanted to handle it when installing Linux, but it looks like they have their own set of udev rules which always maps the larger drive to /dev/sda instead /dev/sdb. Which is why I was thinking, maybe we’ll just have our own we run as necessary to correct this, prior to building our golden image.

At the end of the day, the primary disk should be /dev/sda anyway, so I want to figure out how to get the smaller drive mapped to /dev/sda instead of /dev/sdb if I can.

Anyway, my time has run up for today. I will try a few more things tomorrow morning, specifically regarding udev rules first, most likely.

Thanks again for everyone’s help.

-Dustin

[dholtz-docbox]

@george1421 said in Host Startup; Booting into LVM Disk Fails:

@george1421 OK I have a hackish way to test this idea.

But first, I went through each of the exit modes and all pick the first hard drive. Looking into the code all of the exit modes are hard coded to boot to the first hard disk.

Sooo… as a test please edit this file /var/www/html/fog/lib/fog/bootmenu.class.php search for some fragment of “sanboot --no-describe --drive 0x80” it should be listed only once in that file. Change the 0x80 to 0x81 then file and save the class file. Be sure that you have the host set to sanboot and then use the browser trick I chatted to you to confirm that the sanboot is now sending to the 0x81 disk. Once confirmed try to boot that target computer again. If it works then we have to find the next steps, but we can prove the process works.

Haha, yeah… I tried that earlier. The disk just boot looped with no prompt, indefinitely, until I force-killed-power to the machine.

Edit> That was my immediate hack idea too, just to see if it would work
PS> Everything checked out, regarding your hack for reviewing the settings passed - 0x81 was assigned when tested.

[Wayne Workman]

@george1421 said in Host Startup; Booting into LVM Disk Fails:

Sooo… as a test please edit this file /var/www/html/fog/lib/fog/bootmenu.class.php

Along the same lines, you will find this at line 142 concerning grub that is easy to snipe:

'rootnoverify (hd0);chainloader +1`

Maybe change hd0 to hd1 and switch the host to grub and see what happens.

[dholtz-docbox]

@Wayne-Workman : No dice - good suggest. It just takes me to the GRUB4DOS terminal.

I feel at an impasse, as I know I can leverage FOG as the solution I am looking for. However, I don’t think we have figured out how to structure our partitions so that it works well with remote imaging. If that is the case, I think it will take us a little longer to finalize everything, but I will start looking into how to install Ubuntu the way we need it - where the OS disk is /dev/sda, and the data disk is /dev/sdb. It feels kind of silly, but I feel like a lot of issues would be cleared up by figuring that out first. I just wish the Ubuntu installer let me map them to sda or sdb, it always assigns them automatically. I wish I had more control over it, I guess.

-Dustin

[george1421]

@dholtz-docbox Can you do what I asked (I think, or at least I was thinking about it). Schedule a debug deploy to that target system. Then at the FOG Engine command prompt, key in lsblk and post the results here.

Second, when you installed ubuntu you are sure that you didn’t install the any type of boot loader onto that first drive?

Third, In the AMD bois for this, when you tell it to boot from the disk with the ubuntu on it. In bios is this listed as the second hard drive? So the bios is seeing it as disk 2 or is it seeing it as another number disk? (I’m not sure if number is base 0 or base 1 for this discussion).

I still plan on testing a similar setup with a dell 780 and 2 hard drives to see if I can boot off the second drive.

[dholtz-docbox]

@george1421 : Let me get started on the first, and I will return with the rest.

Edit> Output from debug…

[george1421]

@dholtz-docbox Well thats a bit unexpected.

sda is a unpartitioned ~1tb disk

sdb is a bit confusing since there are two small partitions and then a large partition (where the lvm probably is). The two small partitions is what I’m questioning. One I might understand since that is a boot partition since you are using lvm on sdb3.

sdc looks like maybe a 8GB flash drive?

Is this an accurate assessment?

[dholtz-docbox]

@george1421 said in Host Startup; Booting into LVM Disk Fails:

Second, when you installed ubuntu you are sure that you didn’t install the any type of boot loader onto that first drive?

When I install Ubuntu, I go through the guided installation of LVM, using the entire disk. From there I let it create and write the partitions. I only install Ubuntu on the second disk.

I haven’t done anything to the other disk yet - it’s still just a factory-default drive. I don’t have an option to format this disk when I go through the process, at least I don’t feel like I do. I am use to the desktop installer, I guess. That said, should I try to also format this drive?

@george1421 said in Host Startup; Booting into LVM Disk Fails:

Third, In the AMD bois for this, when you tell it to boot from the disk with the ubuntu on it. In bios is this listed as the second hard drive? So the bios is seeing it as disk 2 or is it seeing it as another number disk? (I’m not sure if number is base 0 or base 1 for this discussion).

I posted this earlier in the thread, let me dig it up. That said, I am pretty sure it’s…

1. NIC
2. ubuntu (not sure who this one is, but it’s the only one I can boot from)
3. ubuntu (not sure who these guys are)
4. sata drive (which should be /dev/sdb)

@george1421 said in Host Startup; Booting into LVM Disk Fails:

I still plan on testing a similar setup with a dell 780 and 2 hard drives to see if I can boot off the second drive.

Oh, that’s awesome! Thank you very much.

[dholtz-docbox]

@george1421 said in Host Startup; Booting into LVM Disk Fails:

@dholtz-docbox Well thats a bit unexpected.

sda is a unpartitioned ~1tb disk

sdb is a bit confusing since there are two small partitions and then a large partition (where the lvm probably is). The two small partitions is what I’m questioning. One I might understand since that is a boot partition since you are using lvm on sdb3.

sdc looks like maybe a 8GB flash drive?

Is this an accurate assessment?

Pretty spot-on

Let me take a snapshot of what the partitioner does to the disk…

Edit> Installer Album
Edit> This is how it looks after installing ubuntu, all I do is move the IBA option to the top for network boot priority

[george1421]

@dholtz-docbox can you move the values at option 3 and option 4 into the slot 1 and 2 positions?

With a quick registration and then a boot to to iPXE and then local hard drive booted the ubuntu boot loader correctly with the default sanboot.

[Wayne Workman]

This is just a wild idea I have that probably will not work but what if you set the primary hdd in fog to sdb for capturing and set another hosts primary disk to sda and deploy ? Would the deployed machine pass through the hdd exit process fog has?

[dholtz-docbox]

@george1421 : I feel I have tried this, but let me give it a go. I have tried so many permutations of these settings, it’s hard to say which combinations haven’t been checked, heh. Do you have the Host Primary Disk setup too?

[george1421]

@Wayne-Workman After I finish a project I’m working on I’ll document what I did, and it worked correctly. The system always booted ubuntu from a second disk even when I swapped the initial /dev/sda out with a virgin hard drive.

[george1421]

@dholtz-docbox said in Host Startup; Booting into LVM Disk Fails:

@george1421 : I feel I have tried this, but let me give it a go. I have tried so many permutations of these settings, it’s hard to say which combinations haven’t been checked, heh. Do you have the Host Primary Disk setup too?

I did nothing more than install a second hard drive in the 780, the original first hard drive had windows on it, installed ubuntu on the second hard drive changed the boot order for second hard drive first then nic booted and ubuntu booted, then swapped the nic and ubuntu disk, pxe booted, registered using quick registration, rebooted into the ipxe menu and then let it timeout to boot the hard drive. It booted ubuntu, from there (and just to make sure it didn’t install a boot loader on disk 1) I swapped disk 1 out with a virgin ssd, pxe booted, let the timeout happen and then it booted into ubuntu. I never touched the fog web gui, my default bios exit mode is sanboot in FOG.