iSCSI diskless boot failed

Beyondlimitation

Hi, I’m just learning ipxe, iSCSI.
Recently, i tried several times to boot iSCSI disk.
I have a nice working fog server, and i can pxe-boot.

With this server i tried this --> https://backreference.org/2013/12/23/diskless-iscsi-boot-with-pxe-howto/

Before explain the detail, check this.

autodraw 2020. 1. 8. 오후 9_49_47.png

I’m running everything on VMware ESXi 6.7 U3.
My Final goal is to boot iSCSI disk to boot Ubuntu server in ipxe VM.

fogserver VM

freenas VM is the iSCSI target.
Here are the details.

iSCSI disk, iscsi-boot detail

I chrooted into the iscsi disk and here is the detail.
All command are executed in chrooted environment.

/etc/iscsi/initiatorname.iscsi

## DO NOT EDIT OR REMOVE THIS FILE!
## If you remove this file, the iSCSI daemon will not start.
## If you change the InitiatorName, existing access control lists
## may reject this initiator.  The InitiatorName must be unique
## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.
InitiatorName=iqn.2019-01.local.freenas.client:client

/etc/default/grub

# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_HIDDEN_TIMEOUT=0
GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=10
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX="ISCSI_INITIATOR=iqn.2019-01.local.freenas.client:client ISCSI_TARGET_NAME=iqn.2019-01.local.freenas:iscsi-boot ISCSI_TARGET_IP=10.0.0.104 ISCSI_TARGET_PORT=3260 root=UUID=8aa17353-7496-4d6e-b300-ad72028906f0 ip=dhcp"

# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"

# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480

# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY="true"

# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"

result of blkid

/dev/sda1: UUID="0AC5-AF0E" TYPE="vfat" PARTLABEL="EFI System Partition" PARTUUID="e980ec5f-a6c7-4933-a912-5922f81f8599"
/dev/sda2: UUID="aba56f48-ca1e-4aa2-aff8-bb229a85505b" TYPE="ext4" PARTUUID="6a81f845-9f28-4d59-9c2f-630b8f65554b"
/dev/sdb1: UUID="928B-C33F" TYPE="vfat" PARTLABEL="EFI System Partition" PARTUUID="642cf4ed-10fe-44fd-956f-1736ed71c9d3"
/dev/sdb2: UUID="8aa17353-7496-4d6e-b300-ad72028906f0" TYPE="ext4" PARTUUID="15c9c4d2-c08b-47ed-be10-642e9f57dd18"
/dev/sdb3: UUID="cbc1696c-a1cf-4548-b3c6-a89005c9f296" TYPE="swap" PARTUUID="a3a01143-4556-4dcd-9959-7a3036418681"
/dev/loop0: TYPE="squashfs"
/dev/loop1: TYPE="squashfs"
/dev/loop2: TYPE="squashfs"
/dev/loop3: TYPE="squashfs"
/dev/loop4: TYPE="squashfs"
/dev/loop5: TYPE="squashfs"
/dev/loop6: TYPE="squashfs"
/dev/loop7: TYPE="squashfs"
/dev/loop8: TYPE="squashfs"
/dev/loop9: TYPE="squashfs"
/dev/loop10: TYPE="squashfs"
/dev/loop11: TYPE="squashfs"
/dev/loop12: TYPE="squashfs"
/dev/loop13: TYPE="squashfs"
/dev/loop14: TYPE="squashfs"
/dev/loop15: TYPE="squashfs"
/dev/loop16: TYPE="squashfs"
/dev/loop17: TYPE="squashfs"
/dev/loop18: TYPE="squashfs"
/dev/loop19: TYPE="squashfs"

And the result is this.

Boot from SAN device 0x80 failed: Operation canceled (http://ipxe.org/0b8080a0)
Unregistered SAN device 0x80
Could not boot: Operation canceled (http://ipxe.org/0b8080a0)
Could not boot: Operation canceled (http://ipxe.org/0b8080a0)

I can connect to iscsi target and mount it.
PXE server works great no issue.

But don’t know why it does not work, and i don’t know how to debug this…
Now what should i do now?

george1421

First let me say that this is not a FOG problem, but a problem created by using FOG for another purpose. It is just interesting enough of a problem that I’m interested in see it working.

The first thing I noticed in your iPXE menu configuration is that you have #ipxe, remove that. There is already one created by default in the iPXE menu. That second #ipxe may confuse iPXE command processor.

The second thing that stands out in the parameters is you listed the iqn but I don’t see how the command will find the freenas server. I need to look up the sanboot method of iPXE to be sure, but I think a host name is needed in addition to the iqn.

Last, just because you can load and boot the lun doesn’t mean the OS will see the lun. It would be roughly equivalent if you try to network boot windows but windows itself is looking for a local hard drive for windows and not the netboot share where windows was launched from. I realize you are not a native english speaker so if you need clarification please let me know.

I will research how to sanboot a system and comment back on this post. I am interested to see if its possible to work as you have designed.

george1421

creating a diskless iscsi boot lun ref: https://backreference.org/2013/12/23/diskless-iscsi-boot-with-pxe-howto/

ipxe sanboot call: sanboot --filename \EFI\redhat\grub.efi iscsi:192.168.0.2::::iqn.2010-04.org.ipxe.chipmunk:rhel6

I see your setup closely matches the link I provided above. I would say first get the #ipxe out of the parameters section and try to boot again.

Is the target computer a uefi or bios based computer? If its uefi make sure that secure boot is turned off.

edit: Final post here has missing step from top instructions: https://forums.linuxmint.com/viewtopic.php?t=248304

george1421

I’m currently building a VM with linux mint mate to give me an OS that I can copy over to an iscsi LUN. I’m going to follow the instructions in the links I provided to see how far I can go. I feel if I can get the iscsi lun created correctly I can boot it using iPXE. But there is no way for me to know until I try it.

Beyondlimitation

@george1421 Yeah, not native , but still able to read tech things written English.
Anyway, i tried several times and my VM boot grub but not OS. But it seems almost works.

I first check VM firmware to see if everything is UEFI with secure boot disabled.
And i found ipxe VM firmware was BIOS which is not right.
So i changed it to UEFI, remove #!ipxe and tried again.

Not work. But i got new error.
It said VM cannot sanboot to that iscsi, because server is not found.
I forgot freenas VM powering on so i tried again.

It still displayed error message but another new message.
https://ipxe.org/err/3c2220

According to note, ACPI table has a problem.
I read mailing list below.
https://lists.ipxe.org/pipermail/ipxe-devel/2017-April/005551.html

I also read this --> https://ipxe.org/cmd/sanboot

I assumed that ACPI table has a issue so i changed iPXE boot entry.

set root-path iscsi:freenas.local:tcp:3260:0:iqn.2019-01.local.freenas:iscsi
sanhook --no-describe --drive 0x81 ${root-path}
sanboot

Well it doesn’t work cuz i didn’t typed right.
So changed it and tried again.

Still error. But i didn’t see this --> https://ipxe.org/err/7f22208e

At that time, i didn’t thought about how ipxe finds grub…
So i read command document about sanboot again.
And the note below the page said “On a UEFI platform, some older operating systems (e.g. RHEL6) will install the bootloader with a non-default filename. You can use the --filename option (or the san-filename setting) to specify the correct bootloader filename.”

And i changed parameter.

set root-path iscsi:freenas.local:tcp:3260:0:iqn.2019-01.local.freenas:iscsi-boot
sanboot --no-describe --filename \EFI\ubuntu\grubx64.efi ${root-path}

Booted again and it booted grub, but failed to boot os.
I think os installation has some issue… But it successfully booted grub.

Here are what i learned.
When computer firmware is UEFI, use --filename argument to notify where the grub is.
ACPI table issue can be occur on vmware esxi vm.

george1421

@Beyondlimitation I’m questioning if you have the OS configured correctly on the SAN LUN. I started working on this last night (I’m in central USA) but I did not finish because it was a long day. My plan is to build a linux mint VM and make all of the iscsi configurations needed. Then from a second VM mount the first VM’s disk and the iscsi LUN and use DD to copy the first VM’s disk to the LUN. Once I have that then I will try SANBOOT from the FOG server.

The VM I created for the first linux mint is configured in uefi mode. I went with uefi mode because its easier to boot than MBR and I can if needed call out the specific uefi file to boot.

I don’t know the answer yet but in my head it should work this way.

george1421

Well I didn’t have success today. I was getting a different error message, but by testing different things in ipxe I can tell I’m connecting to the lun but its just not booting. I had a different idea on how to make a simple operating system to tell if its not booing because of the way I created the lun or its something else. I’m going to try that idea later. I’m not sure if the FOG version of iPXE contains everything needed for iscsi booting or not. So I need to ensure I have a good boot lun before debugging other things. I’m still working on it.

Quazz

I believe you have to modify the client OS to support being booted over iSCSI, though I have limited experience with it.

I could be wrong, but I think by default iSCSI isn’t included in the initramfs of Ubuntu.

Perhaps this https://apfelboymchen.net/gnu/notes/booting ubuntu iscsi.html could be helpful. (at least the client info)

All that said, NFS boot is likely to be a lot more straightforward to implement and I use that method quite often.

iSCSI diskless boot failed

242

11.7k

17.2k

154.7k