@george1421 I did not copy the one from the zip file as that one says it’s a sample. I had, however, previously copied the refind.conf file from the FOG server at the primary location and brought it down to the secondary location so that’s the refind.conf file it is using. I still have a .old version of the secondary FOG server’s original refind.conf file, if you think I should put it back.
EDIT: Ugh. Why does Windows have to suck so bad?
@george1421 I had already copied the 0.11.0 refind files to the FOG server but I did it again just for sanity’s sake. When that again failed to make a difference I decided to try resetting BIOS again. Again, the first time it worked, but when I rebooted it again I noticed it booted straight into Windows so I went into BIOS and changed the NIC back to first priority because it had reversed the priorities by itself. Then it counted down in the FOG menu, began to load Windows and then the screen went black and the monitor went to sleep. It’s still sitting there like that.
How do I set the global efi exit mode? Is that under FOG Configuration -> iPXE General Configuration -> Boot Exit Settings -> Exit To Hard Drive Type (EFI)? If that is where it is, it is already set to REFIND_EFI. If that’s what it’s supposed to be then no, I am not seeing the REFIND menu.
@JJ-Fullmer Creating a bootable USB with rEFInd 0.12.0 (the latest listed here) and trying to boot from that resulted in immediate rejection and attempting to load Windows and another black screen.
Creating a bootable USB with rEFInd 0.11.0 and booting from that immediately brought up the rEFInd menu without any issues.
@george1421 Unfortunately the BIOS reset did not behave. At first it appeared as if it was going to work. The machine booted into Windows after the FOG menu but when it was rebooted again, it once more started goofing off. Also, it seems these motherboards have an annoying habit of automatically changing their first boot priority back to the local Windows boot manager randomly.
I tried your hack just now. I tried adding that line to the beginning of the default.ipxe file and nothing changed, so I made a backup of that file and included only those two lines. The result was:
tftp://192.168.9.1/default.ipxe... ok
builtin/platformstring = efi
Chainloading failed, hit 's' for the iPXE shell; reboot in 10 seconds
192.168.9.1 is the IP of the FOG server. I will now replace the default.ipxe file with the backup.
EDIT: To answer another of your questions, it appears that my DHCP server is just a CISCO 1900 series router. CISCO1941/K9 I think.
@george1421 Alright. I am at the secondary location now. I’m going to try resetting and reconfiguring BIOS on one of these rigs first and see how that goes.
@george1421 I can remote to the secondary FOG server via SSH through the RDP into the mining manager there, but unfortunately there is no one on-site there to do any PXE booting of the rigs. I am the only one administering this mine at either location.
A strange new development, however: out of the blue, for no discernible reason whatsoever, a couple of the MSI rigs at the primary location randomly started having this issue. I guess they decided to reboot for some reason and when they wouldn’t come back online I plugged a cart into one of them and it was boot looping like the ones down at the secondary location. On a whim, I reset BIOS to defaults and reconfigured it and it worked. The same for the other one. I guess that’s another thing I can try at the secondary location. If that fixes the problem…
EDIT: I think I’m going to head down to the secondary location shortly here. There’s nothing else I can do from here.
@george1421 I thought the FOG server at the primary location was running at least FOG 1.5.7 but it appears to be running 1.5.6.
I don’t know how to find what DHCP options 66 and 67 are. Honestly, I don’t even know what that means.
I’m at the primary location right now. Unfortunately, the weather took a serious turn for the worse (as in we went from zero snow to like 4 inches overnight and it’s still snowing) so I’m not sure if I will make it down to the secondary location today. If that is the case I won’t be able to try your hack or tell you what the DHCP server is. Though if I do make it down there, I will try to post what I can. Otherwise that will have to wait until next week.
I’m not particularly familiar with wireshark, I’m afraid.
@Sebastian-Roth
$ ls -al /var/www/
total 20
drwxr-xr-x 4 root root 4096 Oct 14 19:58 .
drwxr-xr-x 14 root root 4096 Oct 14 19:53 …
drwxr-xr-x 11 www-data www-data 4096 Oct 22 19:49 fog
drwxr-xr-x 4 root root 4096 Oct 19 18:17 html
-rw-r–r-- 1 www-data www-data 52 Oct 14 19:58 index.php
$ ls -al /var/www/fog/
total 412
drwxr-xr-x 11 www-data www-data 4096 Oct 22 19:49 .
drwxr-xr-x 4 root root 4096 Oct 14 19:58 …
drwxr-xr-x 2 www-data www-data 4096 Oct 14 19:58 api
drwxr-xr-x 2 www-data www-data 4096 Oct 14 19:58 client
drwxr-xr-x 2 www-data www-data 4096 Oct 14 19:58 commons
-rw-r–r-- 1 www-data www-data 370070 Oct 14 19:58 favicon.ico
lrwxrwxrwx 1 www-data www-data 13 Oct 14 19:58 fog -> /var/www/fog/
drwxr-xr-x 2 www-data www-data 4096 Oct 14 19:58 fogdoc
drwxr-xr-x 3 root root 4096 Oct 22 19:50 html
-rw-r–r-- 1 www-data www-data 572 Oct 14 19:58 index.php
drwxr-xr-x 13 www-data www-data 4096 Oct 14 19:58 lib
drwxr-xr-x 10 www-data www-data 4096 Oct 14 19:58 management
drwxr-xr-x 3 www-data www-data 4096 Oct 14 19:58 service
drwxr-xr-x 2 www-data www-data 4096 Oct 14 19:58 status
$ ls -al /var/www/html/
total 28
drwxr-xr-x 4 root root 4096 Oct 19 18:17 .
drwxr-xr-x 4 root root 4096 Oct 14 19:58 …
drwxr-xr-x 7 root root 4096 Oct 19 18:17 admin
lrwxrwxrwx 1 root root 13 Oct 14 19:58 fog -> /var/www/fog/
-rw-r–r-- 1 root root 10918 Oct 14 19:53 index.html
drwxr-xr-x 2 root root 4096 Oct 19 18:17 pihole
It appears that I have both /var/www/fog/service/ipxe and /var/www/html/fog/service/ipxe directories and their contents appears to be identical. Is there a symlink or something. I didn’t even know there was a /var/www/html/fog/service/ipxe directory but the date of the files I changed matches that in /var/www/fog/service/ipxe.
@george1421 They are two completely independent FOG servers.
@george1421 There are only two locations. The primary location and the secondary location. The secondary location is running FOG 1.5.9 on Ubuntu Server 20.04, however several of the refind files have been replaced with those from the primary location. The primary location is running Ubuntu Server 18.04 but I don’t know precisely what version of FOG is running. Is there a way to tell inside the FOG dashboard or is there some other way to tell? I can get you the FOG version at the primary location tomorrow.
The FOG server at the Primary location has FOG running on a 250GB SSD I think, with the default /images on the same drive for images. There are only three images stored and there is plenty of room. The secondary location has a 1TB HDD mounted to /images which doesn’t even have 200GB of data on it including images for both the MSI and Biostar machines and more than half of that is because the MSI image isn’t resized. The images are only about 30GB when the partitions are resized.
I will try your hack tomorrow.
@george1421 They are all in UEFI mode and are basically identical in every respect from hardware to the OS image that it’s running to the rigs at the primary location. Completely interchangeable.
I also have rigs with Biostar motherboards that otherwise have exactly the same hardware as the MSI machines. These rigs are not experiencing these issues. I’m not sure if that renders your DHCP questions moot, but I can get you that information tomorrow if it’s still necessary.
@george1421 I’m not really sure what you mean. I don’t know what the refind menu is.
What’s happening is that when I try to boot these particular mining rigs with MSI motherboards (which work fine at my primary location with the same hardware configuration but the FOG server is running older software) from PXE, it detects media and properly boots into the FOG menu where it gives the option to boot from the local drive by default or to register, delete or image the host. When I select the first option or the let the timer run down, it then begins to load Windows, as I can see the circling Windows loading indicator, but then the screen goes black and it eventually reboots and does the whole thing over again. And then keeps doing that.
When I remove the network card from the list of available boot options (or for that matter even set it to second priority after the local drive) it will then boot into Windows just fine. The worst that happens is it sometimes requires a few minutes to auto-repair.
When I tried changing the UEFI exit type for one of the misbehaving hosts to either HDD or WINDOWS, all that happened was that the timer in the FOG menu would just keep resetting and it would never boot past that until I either changed BIOS to not boot from the NIC or deleted and re-registered the host in FOG.
So far replacing the refind files with older ones has not worked, but I will try swapping the refind.efi for the refind_x64.efi and see if that helps. I have to admit though, that I have my doubts, since the refind.efi file from the primary location did not solve the problem.
@george1421 So refind.efi is just refind_x64.efi renamed? Interesting. I’m not sure it will make a difference though, as I have already copied the refind.efi file from the primary location to the secondary and it didn’t fix anything but I will try it anyway. Thanks.
@Goll420 How did you go about switching to 0.11.0? I’m concerned that perhaps I did something wrong.
@JJ-Fullmer Well, that didn’t work. I am at a loss.
I downloaded the bin-0.11.0 zip file and copied the refind_x64.efi file to the /var/www/fog/service/ipxe/ directory (there was no /var/www/fog/html/service/ipxe) and that made no difference. There was no refind.efi file in the zip. When that failed, I copied all of the relevant refind files (refind.conf, refind.efi, refind_x64.efi and refind_ia32.efi) from the FOG server at the primary location into the /var/www/fog/service/ipxe directory at the secondary location. I actually had to put them all on a USB stick and drive to the secondary location to do this because gmail would not let me email them to myself. That also made no difference. I then created a /var/www/fog/html/service/ipxe/ directory to put everything in in the hopes that would solve the problems. It did not.
Then I downloaded the .deb file and ran that. It also didn’t fix anything.
When all of that failed to accomplish anything useful, I then tried to mess around with a few different UEFI exit types (I tried FIRST_FOUND_HDD and FIRST_FOUND_WINDOWS) for a couple of the rigs. All that resulted in was the inability to boot past the FOG menu. The countdown would simply keep restarting. Even after setting the UEFI exit type back to default, the only way I could get it to boot past that menu was to delete and re-register the host.
I feel like I’m beating my head against a wall. I wonder if the gnuefi download might help?
@Goll420 Awesome. That’s good to know. Thanks. I will be getting 0.11.0 set up today. I am optimistic that it will solve my issues.
@JJ-Fullmer That does indeed help. I think will probably end up switching to rEFInd 0.11.0, if that seems to be what’s working for you for the majority of machines. How do I tell what version of rEFInd is currently being used? Just for my own curiosity I’d like to check the primary location.
It should be pretty easy to get this set up and test it from my office at the primary location tomorrow.
@Sebastian-Roth I’m not sure exactly that it is rEFInd that isn’t working. That’s just what one of the guys on the reddit thread suggested. He also asked about the hardware.
I have basically two different hardware configurations for my mining rigs. They’re either using a certain model of MSI motherboard or they’re using a certain model of Biostar motherboard. Other than that, the hardware is all the same.
Also, yes, they are all booting UEFI.
As far as I can tell, the exit types at both locations are just left at default (which is to say there isn’t one selected). I can try different exit types.
Both locations have a combination of both hardware configurations. At the primary location with the older Ubuntu and FOG software, it’s all working just dandy. At the secondary location with Ubuntu 20.04 and FOG 1.5.9, the MSI hosts seem to be having this problem.
It’s also strange because when I first registered the hosts on FOG at the secondary location, they booted into Windows just fine. It was only after I captured an image and subsequently tried to deploy it to a few of the hosts that this problem started happening.
Hey guys. I seem to be having a new problem with FOG. When I have my hosts configured to boot from NIC at first priority, they will boot to PXE and then when no input happens they will skip that and attempt to load Windows. This is where the problem starts. Instead of loading Windows, they will either just freeze solid on a black screen or go into a boot loop and keep booting PXE, attempting to load Windows and then rebooting. When I disable PXE booting, the machines boot into Windows without a problem.
I have posted about this on reddit here. One user there has suggested something about non-signed kernels having issues chainloading to a signed Windows kernel.
This problem does not happen at my primary mine location where the FOG server is running an older version of Ubuntu and an older version of FOG. That same user on reddit suggested the possibility of changing the kernel at the secondary location to the same one used at the primary location. Is this a feasible solution and if so, how do I go about doing that? If not, is there some other way to fix this issue? I need to be able to image these machines remotely and cannot be on-site to manually reconfigure BIOS back and forth between PXE and local disk boot whenever I need to image them.
Thanks.
@Sebastian-Roth So it turns out most of my problems were in fact caused simply by old software. I installed Ubuntu 20.04 and FOG 1.5.9 and it solved all the database updating issues.
Now I just have to solve some DNS problems that I’m pretty sure are not FOG related.
Thanks mate.