Host Startup; Booting into LVM Disk Fails

[Tom Elliott]

I say this, because you’re seeing stuff populate after the boot-url portion. From what I can tell, it appears No menu is on?

[george1421]

@Tom-Elliott said in Host Startup; Booting into LVM Disk Fails:

I say this, because you’re seeing stuff populate after the boot-url portion. From what I can tell, it appears No menu is on?

Right no menus were on. I had him turn it on and he posted the results in chat. If you ignore everywhere this thread has gone, the issue is by their specific design his /dev/sda is empty (no partition) and /dev/sdb has ubuntu on it. If he changes the boot order in bios to not pxe the target computer boots correctly, but if it boots through iPXE sanboot is trying to exit to the blank disk (/dev/sda). This is outside of FOG imaging so far. Right now he is testing his reference image to boot through ipxe to OS boot.

[dholtz-docbox]

@Tom-Elliott : Yeah, I wasn’t having any luck trying anything from the other thread. I have been tossing around the idea, why are these drives setup this way and what can we do to change it. In my head, is it as easy as just swapping the SATA cables? I am not sure why the drives are the way they are, but I would not mind changing it if I can. I just haven’t looked into it yet - it just sounds like a lot of issues would be resolved if the OS was on the first disk?

[Wayne Workman]

http://askubuntu.com/questions/371049/how-are-dev-sda-and-dev-sdb-chosen

According to that, you might want to remove one of the disks for Ubuntu’s installation only, then add it back afterwards and see how that goes. And also as that link suggests, use UUIDs instead of /dev/sdX, follow the documentation.

https://help.ubuntu.com/community/UsingUUID

[george1421]

@dholtz-docbox outside of your design requirements, the one question in my mind is if you did switch the cables (as a test) does it pxe boot properly to the OS. That way we can just focus on getting the proper exit to work/boot to the second hard drive. I know I’ve seen it in the past change the boot drive, but I can’t remember what I did to do that until I can get my hands on a fog server.

[dholtz-docbox]

I am looking to have someone open the enclosure so I can swap the cables - darn special screwdriver that I don’t have. Until then, I am looking at how to handle how /dev/sda is assigned, and it looks like it comes down to udev rules? That’s what I am looking at right now. I know a lot of this is hitting a bridge I was looking to prolong, but I have been looking to more intimately setup the partitions. I guess this is the start of understanding how some of this is handled.

[george1421]

@dholtz-docbox Remember at this point in time you are NOT in any OS. When the iPXE menu exits you are still running the iPXE kernel. It knows nothing about your target OS. Its just passing (or trying to) pass off boot control some some boot device.

[Wayne Workman]

George is right on that.

[george1421]

@george1421 OK I have a hackish way to test this idea.

But first, I went through each of the exit modes and all pick the first hard drive. Looking into the code all of the exit modes are hard coded to boot to the first hard disk.

Sooo… as a test please edit this file /var/www/html/fog/lib/fog/bootmenu.class.php search for some fragment of “sanboot --no-describe --drive 0x80” it should be listed only once in that file. Change the 0x80 to 0x81 then file and save the class file. Be sure that you have the host set to sanboot and then use the browser trick I chatted to you to confirm that the sanboot is now sending to the 0x81 disk. Once confirmed try to boot that target computer again. If it works then we have to find the next steps, but we can prove the process works.

[dholtz-docbox]

That makes sense. I guess my thought here, is, before trying to boot through network, after installing Ubuntu, I run a set of udev rules, this remapping which disk /dev/sda and /dev/sdb are associated with.

Also, I can’t switch the cables around, one drive is SATA where the other is… like an onboard PCI-e disk. I wanted to handle it when installing Linux, but it looks like they have their own set of udev rules which always maps the larger drive to /dev/sda instead /dev/sdb. Which is why I was thinking, maybe we’ll just have our own we run as necessary to correct this, prior to building our golden image.

At the end of the day, the primary disk should be /dev/sda anyway, so I want to figure out how to get the smaller drive mapped to /dev/sda instead of /dev/sdb if I can.

Anyway, my time has run up for today. I will try a few more things tomorrow morning, specifically regarding udev rules first, most likely.

Thanks again for everyone’s help.

-Dustin

[dholtz-docbox]

@george1421 said in Host Startup; Booting into LVM Disk Fails:

@george1421 OK I have a hackish way to test this idea.

But first, I went through each of the exit modes and all pick the first hard drive. Looking into the code all of the exit modes are hard coded to boot to the first hard disk.

Sooo… as a test please edit this file /var/www/html/fog/lib/fog/bootmenu.class.php search for some fragment of “sanboot --no-describe --drive 0x80” it should be listed only once in that file. Change the 0x80 to 0x81 then file and save the class file. Be sure that you have the host set to sanboot and then use the browser trick I chatted to you to confirm that the sanboot is now sending to the 0x81 disk. Once confirmed try to boot that target computer again. If it works then we have to find the next steps, but we can prove the process works.

Haha, yeah… I tried that earlier. The disk just boot looped with no prompt, indefinitely, until I force-killed-power to the machine.

Edit> That was my immediate hack idea too, just to see if it would work
PS> Everything checked out, regarding your hack for reviewing the settings passed - 0x81 was assigned when tested.

[Wayne Workman]

@george1421 said in Host Startup; Booting into LVM Disk Fails:

Sooo… as a test please edit this file /var/www/html/fog/lib/fog/bootmenu.class.php

Along the same lines, you will find this at line 142 concerning grub that is easy to snipe:

'rootnoverify (hd0);chainloader +1`

Maybe change hd0 to hd1 and switch the host to grub and see what happens.

[dholtz-docbox]

@Wayne-Workman : No dice - good suggest. It just takes me to the GRUB4DOS terminal.

I feel at an impasse, as I know I can leverage FOG as the solution I am looking for. However, I don’t think we have figured out how to structure our partitions so that it works well with remote imaging. If that is the case, I think it will take us a little longer to finalize everything, but I will start looking into how to install Ubuntu the way we need it - where the OS disk is /dev/sda, and the data disk is /dev/sdb. It feels kind of silly, but I feel like a lot of issues would be cleared up by figuring that out first. I just wish the Ubuntu installer let me map them to sda or sdb, it always assigns them automatically. I wish I had more control over it, I guess.

-Dustin

[george1421]

@dholtz-docbox Can you do what I asked (I think, or at least I was thinking about it). Schedule a debug deploy to that target system. Then at the FOG Engine command prompt, key in lsblk and post the results here.

Second, when you installed ubuntu you are sure that you didn’t install the any type of boot loader onto that first drive?

Third, In the AMD bois for this, when you tell it to boot from the disk with the ubuntu on it. In bios is this listed as the second hard drive? So the bios is seeing it as disk 2 or is it seeing it as another number disk? (I’m not sure if number is base 0 or base 1 for this discussion).

I still plan on testing a similar setup with a dell 780 and 2 hard drives to see if I can boot off the second drive.

[dholtz-docbox]

@george1421 : Let me get started on the first, and I will return with the rest.

Edit> Output from debug…

[george1421]

@dholtz-docbox Well thats a bit unexpected.

sda is a unpartitioned ~1tb disk

sdb is a bit confusing since there are two small partitions and then a large partition (where the lvm probably is). The two small partitions is what I’m questioning. One I might understand since that is a boot partition since you are using lvm on sdb3.

sdc looks like maybe a 8GB flash drive?

Is this an accurate assessment?

[dholtz-docbox]

@george1421 said in Host Startup; Booting into LVM Disk Fails:

Second, when you installed ubuntu you are sure that you didn’t install the any type of boot loader onto that first drive?

When I install Ubuntu, I go through the guided installation of LVM, using the entire disk. From there I let it create and write the partitions. I only install Ubuntu on the second disk.

I haven’t done anything to the other disk yet - it’s still just a factory-default drive. I don’t have an option to format this disk when I go through the process, at least I don’t feel like I do. I am use to the desktop installer, I guess. That said, should I try to also format this drive?

@george1421 said in Host Startup; Booting into LVM Disk Fails:

Third, In the AMD bois for this, when you tell it to boot from the disk with the ubuntu on it. In bios is this listed as the second hard drive? So the bios is seeing it as disk 2 or is it seeing it as another number disk? (I’m not sure if number is base 0 or base 1 for this discussion).

I posted this earlier in the thread, let me dig it up. That said, I am pretty sure it’s…

1. NIC
2. ubuntu (not sure who this one is, but it’s the only one I can boot from)
3. ubuntu (not sure who these guys are)
4. sata drive (which should be /dev/sdb)

@george1421 said in Host Startup; Booting into LVM Disk Fails:

I still plan on testing a similar setup with a dell 780 and 2 hard drives to see if I can boot off the second drive.

Oh, that’s awesome! Thank you very much.

[dholtz-docbox]

@george1421 said in Host Startup; Booting into LVM Disk Fails:

@dholtz-docbox Well thats a bit unexpected.

sda is a unpartitioned ~1tb disk

sdb is a bit confusing since there are two small partitions and then a large partition (where the lvm probably is). The two small partitions is what I’m questioning. One I might understand since that is a boot partition since you are using lvm on sdb3.

sdc looks like maybe a 8GB flash drive?

Is this an accurate assessment?

Pretty spot-on

Let me take a snapshot of what the partitioner does to the disk…

Edit> Installer Album
Edit> This is how it looks after installing ubuntu, all I do is move the IBA option to the top for network boot priority

[george1421]

@dholtz-docbox can you move the values at option 3 and option 4 into the slot 1 and 2 positions?

With a quick registration and then a boot to to iPXE and then local hard drive booted the ubuntu boot loader correctly with the default sanboot.

[Wayne Workman]

This is just a wild idea I have that probably will not work but what if you set the primary hdd in fog to sdb for capturing and set another hosts primary disk to sda and deploy ? Would the deployed machine pass through the hdd exit process fog has?