6 Storage Nodes: No open slots, there are 0 before me

cojohnson · Sep 29, 2017, 1:53 PM

Hi Guys.

I have a similar issue as @george1421 did in the post:
https://forums.fogproject.org/topic/10169/storage-node-no-open-slots-there-are-0-before-me

I have one master node set to handle 0 clients. and 6 nodes to handle 10 clients each. I am getting
No open slots. There are 0 before me. (In line for xxxx)

Any ideas / suggestions?

Tom Elliott · Sep 29, 2017, 2:05 PM

What version of FOG are you running?

The slots refer to an individual node, so from the sounds of things it’s working off the master node?

cojohnson · Sep 29, 2017, 6:03 PM

@tom-elliott Running Version 1.5.0-RC-9

Yes, it is defaulting to the master node only.

I also just verified that this only effect the “Legacy” devices booted with the undionly.kpxe, undionly.kkpxe and realtek.kpxe. Network booting a newer UEFI device with ipxe.efi and imaging the device works proper. The server sends it off to the storage nodes.

Sebastian Roth · Oct 2, 2017, 4:03 PM

@cojohnson said:

I also just verified that this only effect the “Legacy” devices booted with the undionly.kpxe, undionly.kkpxe and realtek.kpxe. Network booting a newer UEFI device with ipxe.efi and imaging the device works proper. The server sends it off to the storage nodes.

This sounds pretty strange to me and I can’t see why this should be the case. Don’t get me wrong, not saying that it can’t be, just I don’t get it yet. Can you please to me a favor: Select two of your hosts within the same subnet that ought to boot from the same storage node (whichever you like) - one legacy BIOS and one UEFI. Schedule a task for each of those and copy their MAC addresses from the FOG web UI host settings to a temporary text file. Then open the following URL in your browser while replacing x.x.x.x with the FOG server IP and put the MAC addresses instead of <macaddress>: http://x.x.x.x/fog/service/ipxe/boot.php?mac=<macaddress>

Open it for both hosts/MACs you selected and copy/paste the code you see in the browser here in the forums and let us know which one’s UEFI.

cojohnson · Oct 2, 2017, 4:03 PM

@sebastian-roth
They look identical (with the exception of hostname)

Legacy Client

#!ipxe
set fog-ip 10.21.0.105
set fog-webroot fog
set boot-url http://${fog-ip}/${fog-webroot}
cpuid --ext 29 && set arch x86_64 || set arch i386
goto get_console
:console_set
colour --rgb 0x00567a 1 ||
colour --rgb 0x00567a 2 ||
colour --rgb 0x00567a 4 ||
cpair --foreground 7 --background 2 2 ||
goto MENU
:alt_console
cpair --background 0 1 ||
cpair --background 1 2 ||
goto MENU
:get_console
console --picture http://10.21.0.105/fog/service/ipxe/bg.png --left 100 --right 80 && goto console_set || goto alt_console
:MENU
menu
colour --rgb 0x00567a 0 ||
cpair --foreground 1 1 ||
cpair --foreground 0 3 ||
cpair --foreground 4 4 ||
item --gap Host is registered as HP4540n-7297!
item --gap -- -------------------------------------
item fog.local Boot from hard disk
item fog.memtest Run Memtest86+
item fog.keyreg Update Product Key
item fog.deployimage Deploy Image
item fog.multijoin Join Multicast Session
item fog.quickdel Quick Host Deletion
item fog.sysinfo Client System Information (Compatibility)
choose --default fog.local --timeout 3000 target && goto ${target}
:fog.local
sanboot --no-describe --drive 0x80 || goto MENU
:fog.memtest
kernel memdisk initrd=memtest.bin iso raw
initrd memtest.bin
boot || goto MENU
:fog.keyreg
login
params
param mac0 ${net0/mac}
param arch ${arch}
param username ${username}
param password ${password}
param keyreg 1
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
param sysuuid ${uuid}
:fog.deployimage
login
params
param mac0 ${net0/mac}
param arch ${arch}
param username ${username}
param password ${password}
param qihost 1
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
param sysuuid ${uuid}
:fog.multijoin
login
params
param mac0 ${net0/mac}
param arch ${arch}
param username ${username}
param password ${password}
param sessionJoin 1
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
param sysuuid ${uuid}
:fog.quickdel
login
params
param mac0 ${net0/mac}
param arch ${arch}
param username ${username}
param password ${password}
param delhost 1
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
param sysuuid ${uuid}
:fog.sysinfo
kernel bzImage32 loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=127000 web=http://10.21.0.105/fog/ consoleblank=0 rootfstype=ext4 storage=10.21.0.105:/images/ storageip=10.21.0.105 loglevel=4 mode=sysinfo
imgfetch init_32.xz
boot || goto MENU
:bootme
chain -ar http://10.21.0.105/fog/service/ipxe/boot.php##params ||
goto MENU
autoboot

UEFI Client

#!ipxe
set fog-ip 10.21.0.105
set fog-webroot fog
set boot-url http://${fog-ip}/${fog-webroot}
cpuid --ext 29 && set arch x86_64 || set arch i386
goto get_console
:console_set
colour --rgb 0x00567a 1 ||
colour --rgb 0x00567a 2 ||
colour --rgb 0x00567a 4 ||
cpair --foreground 7 --background 2 2 ||
goto MENU
:alt_console
cpair --background 0 1 ||
cpair --background 1 2 ||
goto MENU
:get_console
console --picture http://10.21.0.105/fog/service/ipxe/bg.png --left 100 --right 80 && goto console_set || goto alt_console
:MENU
menu
colour --rgb 0x00567a 0 ||
cpair --foreground 1 1 ||
cpair --foreground 0 3 ||
cpair --foreground 4 4 ||
item --gap Host is registered as HPADAPTER15!
item --gap -- -------------------------------------
item fog.local Boot from hard disk
item fog.memtest Run Memtest86+
item fog.keyreg Update Product Key
item fog.deployimage Deploy Image
item fog.multijoin Join Multicast Session
item fog.quickdel Quick Host Deletion
item fog.sysinfo Client System Information (Compatibility)
choose --default fog.local --timeout 3000 target && goto ${target}
:fog.local
sanboot --no-describe --drive 0x80 || goto MENU
:fog.memtest
kernel memdisk initrd=memtest.bin iso raw
initrd memtest.bin
boot || goto MENU
:fog.keyreg
login
params
param mac0 ${net0/mac}
param arch ${arch}
param username ${username}
param password ${password}
param keyreg 1
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
param sysuuid ${uuid}
:fog.deployimage
login
params
param mac0 ${net0/mac}
param arch ${arch}
param username ${username}
param password ${password}
param qihost 1
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
param sysuuid ${uuid}
:fog.multijoin
login
params
param mac0 ${net0/mac}
param arch ${arch}
param username ${username}
param password ${password}
param sessionJoin 1
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
param sysuuid ${uuid}
:fog.quickdel
login
params
param mac0 ${net0/mac}
param arch ${arch}
param username ${username}
param password ${password}
param delhost 1
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
param sysuuid ${uuid}
:fog.sysinfo
kernel bzImage32 loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=127000 web=http://10.21.0.105/fog/ consoleblank=0 rootfstype=ext4 storage=10.21.0.105:/images/ storageip=10.21.0.105 loglevel=4 mode=sysinfo
imgfetch init_32.xz
boot || goto MENU
:bootme
chain -ar http://10.21.0.105/fog/service/ipxe/boot.php##params ||
goto MENU
autoboot

Sebastian Roth · Oct 2, 2017, 4:06 PM

@cojohnson Sorry I did not mention that there ought to be a task scheduled for both of the hosts. Possibly we’ll see a difference in the output then. So please schedule tasks for those two machines and open the URLs again.

cojohnson · Oct 4, 2017, 3:48 PM

@sebastian-roth

I just changed all of my FOG nodes to the “working” branch. In case this changes anything. I am having problems with image replication over FTP.

https://forums.fogproject.org/topic/10891/image-replication-not-working

Anyways, Here is the UEFI Client

#!ipxe
set fog-ip 10.21.0.105
set fog-webroot fog
set boot-url http://${fog-ip}/${fog-webroot}
kernel bzImage32 loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=127000 web=http://10.21.0.105/fog/ consoleblank=0 rootfstype=ext4 mac=9c:eb:e8:3b:01:0f ftp=10.21.0.105 storage=10.21.0.105:/images/ storageip=10.21.0.105 osid=9 irqpoll chkdsk=0 img=HPStreamG2andG3FY1718 imgType=n imgPartitionType=all imgid=45 imgFormat=0 PIGZ_COMP=-6 hostearly=1 type=down
imgfetch init_32.xz
boot

Here is the Legacy Client

#!ipxe
set fog-ip 10.21.0.105
set fog-webroot fog
set boot-url http://${fog-ip}/${fog-webroot}
kernel bzImage32 loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=127000 web=http://10.21.0.105/fog/ consoleblank=0 rootfstype=ext4 mac=b4:b5:2f:84:9d:46 ftp=10.21.0.105 storage=10.21.0.105:/images/ storageip=10.21.0.105 osid=5 irqpoll hostname=HP4540n-7297 chkdsk=0 img=HP45xxUniversal imgType=n imgPartitionType=all imgid=46 imgFormat=0 PIGZ_COMP=-6 hostearly=1 type=down
imgfetch init_32.xz
boot

Sebastian Roth · Oct 5, 2017, 4:21 AM

@cojohnson Why is OS ID set to different values - one is set to 5 the other is 9…?

Wayne Workman · Oct 5, 2017, 4:21 AM

@sebastian-roth likely different images for legacy & uefi. One would be MBR, the other GPT. Just a guess.

Sebastian Roth · Oct 5, 2017, 6:57 AM

@cojohnson Please post a picture of your images (so we see “HPStreamG2andG3FY1718” and “HP4540n-7297”) and storages lists here. My guess is that those images are configured to be on different storage groups and therefore the clients talk to different storage nodes…

Tom Elliott · Oct 11, 2017, 2:12 PM

@wayne-workman

OS ID 5 = Windows 7
OS ID 9 = Windows 10

The OS ID values have no bearing on “GPT/MBR” partition types, that’s all handled by the image itself. You can have a GPT based Windows 7, and an MBR based Windows 10 image of course. (Or vice-versa, or both the same type).

Tom Elliott · Oct 11, 2017, 2:19 PM

Seeing as the image os id’s are different, is it likely the image that’s os id 5 actually only available on the master node? It seems like to from what I can tell.

At the moment both image outputs are showing to 10.21.0.105, is this your master node or an alternate node?

Please remember, the queuing system is not 100% perfect. It attempts to switch to less busy nodes. It has no knowledge of which node will be fastest, of course, it just knows how busy one to another node is. Of course, if the image is unable to be found on other nodes, it can’t do anything with that node. If none of the nodes have the image, the only place it can look is the master node. If the master node has no queue-able slots, it can’t do anything. The first host to try to check in should be “there are 0 before me”, second would be “there is 1 before me”, etc… Of course this is timing dependent too, so it’s possible to have many hosts state that “there are 0 before me”. Normally this doesn’t pose much a problem though and the first to checkin first when a slot becomes available will begin the work. I have added a random timer to try to prevent timing issues as such from occurring however.

cojohnson · Oct 13, 2017, 11:26 AM

@sebastian-roth
I do not image Legacy equipment very often… I may not be able to test a solution for a while… Note, I did switch from the developers branch to the working branch just recently. I would like to test it on the “Working” branch

Sebastian Roth · Oct 18, 2017, 7:55 PM

@cojohnson Sorry for my late reply. We also need a picture of the Storage settings page.

Sebastian Roth · Dec 2, 2017, 1:05 PM

@cojohnson Any news on this?

cojohnson · Dec 4, 2017, 2:36 PM

@sebastian-roth Sadly No! I haven’t had the need to image numerous machines at the same time in the middle of the school year. I normally only need to do this over the summer.

Sebastian Roth · Dec 4, 2017, 7:36 PM

@cojohnson Well, just let us know when you need further help on this.

6 Storage Nodes: No open slots, there are 0 before me

144

12.1k

17.3k

155.4k