@vpt said in Unable to use an NFS share to store images:

With this setup, I get an error message during capturing that states that access is denied when attempting to mount the share to the /images directory. I have verified that the share on the QNAP is wide open to anyone, but I still get an access denied.

Its probably because FOS Linux connects to the storage node using NFS and the NFS server (QNAP in this case) has squash_root enabled (meaning that it rejects root mounting the NFS share). I have some KBs in the tutorials forum that talk about synology NAS and using a windows server as a FOG storage node (not recommended) that mentions this setting too.

Hmm… I find it interesting that its trying to load a non-fog init (virtual hard drive). Why is it referencing fog/images/init.gz Secondly, the init for fog should be .xz compressed and not .gz compressed. Off the top of my head, I don’t know of the FOG kernel will be able to expand a .gz compressed file since natively it only understand .xz.

But the error basically says the kernel could not connect to the virtual hard drive.

@AlexPDX said in kernel panic-not syncing: VFS: unable to mount root fs on unknown block(1,0):

I followed the instalation steps exactly

I find this hard to understand. If you followed the installation steps exactly then it would not try to load fog/image/init.gz. Where did you get the instructions to install fog 1.5.6?

Can you go to this url with a web browser and then post the output here. http://192.168.192.111/fog/service/ipxe/boot.php?mac=00:00:00:00:00:00 and also http://192.168.192.111/fog/service/ipxe/boot.php?mac=f8:b1:56:d9:b8:ab

@AlexPDX OK, can you show me the host definition for the system Windows10-CLEAN?

Also I have to ask you why you have this item in the FOG iPXE menu?

item FOG_1.5.5 FOG Server

kernel bzImage loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=127000 web=http://192.168.192.111/fog/ consoleblank=0 rootfstype=ext4 storage=192.168.192.111:/images/ storageip=192.168.192.111 loglevel=4
imgfetch init.xz
boot || goto MENU

This is set as default but it doesn’t “do” anything. I know it boots the fog kernel, but its not a standard menu item.

Back to what you posted there is nothing in the iPXE menu to call fog/images/init.gz that I can see.

Lets start with the host definition screen shot, then if that doesn’t give us a clue, I’ll have you schedule a capture or deploy (whatever you were doing to that computer when it threw that error), but don’t pxe boot the target computer. I’ll want you to run that second link. As long as the computer mac address is the same it should give us a little better picture. When a task is running, that call to boot.php will be very short.

@Nabeel Did you configure your dhcp server to supply dhcp options 66 and 67? For dhcp option 66 that needs to be the IP address of your fog server. For dhcp option 67 that should be undionly.kpxe for bios based computers and ipxe.efi for UEFI based systems.

If you need to support both systems in your environment then once you prove that the above options work you can review this article: https://wiki.fogproject.org/wiki/index.php/BIOS_and_UEFI_Co-Existence#Using_Windows_Server_2012_.28R1_and_later.29_DHCP_Policy

@androsszit So just to be clear, deploy image from the iPXE menu doesn’t expand the disk correctly, but registering the host then and then from the web ui deploying the image to it works?

I know this is asking a bit more from you, but if you do a iPXE menu full registration at the end of the full registration it asks you if you want to deploy the image, if you answer yes to this does it expand the disk correctly? This is the first I’ve heard about a different reaction from the iPXE Deploy image and from the web ui deploy image.

@Sebastian-Roth I also did testing of iPXE Deploy image vs the web ui Deploy image and the instructions (kernel parameters passed) to FOS are identical. So at least what FOS knows about there should be no difference in the way it deploys the image.

@siara Lets have you rerun the fog installer script. It almost sounds like the fog program code didn’t get installed correctly.

@KennethJ said in Image sharing between sites:

Our network is setup in a way that makes it impossible for the remote-node to contact the master-node

Top down is fine as long as the master node can make http calls as well as ftp to the remote site. Remember that ftp is a two way street the master will open a command channel to the remote FTP server, and the remote FTP server will open a data channel back to the master FOG server (ftp client) to send the file over. This should be allowed through your site to site links since ftp is a well known communications protocol that has these 2 channels. If this is a limitation for your network design, you will loose a little flexibility but you could use rsync (outside of FOG) to send files from the master node to the remote storage nodes you will loose the ability to control when an image is replicated but you can achieve this top down communications.

There are FOG replicator logs in /opt/fog/logs that might give you an understanding why replication isn’t working too. I would begin with the logs then see where that leads you.

@Sebastian-Roth said in Deployment and Quick Delete Host not working from iPXE menu:

ave you seen UEFI clients behaving differently on the iPXE menu/login/params stuff yet

No I haven’t. This is a unique case. So both Deploy Image (quick image) and Quick Delete function by sending a single parameter (delhost or qihost) to boot.php. Its the same for the registration commands too. But registration works. I haven’t looked at the boot.php flow but what might make FOS fall through both qihost and delhost to end up executing the compat test code.

Possibly we could set a global kernel parameter of isdebug=yes then get this menu to appear by picking delhost or qihost. At them menu break out and confirm that the kernel parameters sent from FOG are correct. If they are correct then we could focus on the FOS code (/bin/fog) to why it might fail through to compat test mode. When testing is done then isdebug=yes should be removed so FOG runs correctly.

You have to remember that the FOG Client runs as SYSTEM, and SYSTEM has full authority on the LOCAL COMPUTER. SYSTEM does not have access or authority to any external resources. If you want the batch file to work that way you will need to map a drive (using a proper domain account) within the batch file to the remote share and then run the installer from that mapped drive. The alternative method is to deploy the Authy package as an executable if everything is in a compressed executable or use a snapin package (zip file) to include all of your files to be copied to the target computer.

Yes you can delete the image, FOG will not care about the missing image until you go to redeploy to those hosts again. I have not tried this, but I suspect you will get an invalid image ID if you try to redeploy those hosts without first assigning a new image to them. But the only time you will have a problem in the future is when you are deploying to these systems and the image is unlinked.

@jflippen how are your storage nodes setup? Are you using the location plugin to direct clients to specific storage nodes? -OR- are they setup just for load sharing (i.e. first one available gets the imaging job)?

I see from your picture that your fog server is at 10.59.10.12 and the storage server you are using is at 10.59.181.12. Are they on different subnets? What subnet is the pxe booting client on when attempting to connect to 10.59.181.12? Do you have any screening/firewall routers in between that might be blocking the NFS mount?

@KennethJ said in Image sharing between sites:

Have tried a bit now and it seems the FTP port is closed by default on all our networks

Is ssh port open? While FOG doesn’t currently support it, you may need to use rsync in place of FOG Replication to sync your image files. You will loose some capability with this option, but you might bypass the ftp port blockage.

@sehume Well this is an interesting issue…

The next steps are to see what the pxe booting client is seeing, because something is going on there. I have a suspicion, but lets test it.

1. Manually register this host from the picture.
2. Schedule a debug capture or deploy (don’t care). Before you schedule the task, tick the debug checkbox, then schedule the task.
3. PXE boot the target computer. After a few screens of text where you have to clear with the enter key you should be dropped to a linux command prompt. You may get the same error message about connecting to the fog server, just press enter like you did before.
4. At the FOS Linux command prompt, on the target computer, key in ip addr show confirm that eth0 has an IP address. Please post a clear picture of the output of that command here.
5. Ping the fog server from the target computer. Post the results of the test.
6. Next we are going to do the http test from the target computer. Key in the following command curl -Ikfs http://10.245.220.30/fog/management/index.php --connect-timeout 5
   The success response will start out with

HTTP/1.1 200 OK

Lets see what info those tests provide.

@frankv24 said in tftp server issue:

when adding capture filter 67 and 68 no data was found for .203

Just to be clear, if you setup a filter for .203 in addition to udp ports 67 and 68, then that is the problem. The dhcp process uses broadcast messages to communicate. Since they are broadcast messages all client computers on the 10.15 subnet will hear this communication. That is why as long as the witness computer is on the 10.15 subnet and listening it should hear the dhcp communications. At the point of the dhcp communications, the target computer doesn’t know its IP address.

@faresalandlos Well at this point I don’t think its a firewall issue, but to answer your question for ubuntu this is the command to disable the firewall sudo ufw disable

So what do I think:

1. You need to understand why nfs is not starting on that system.
2. your exports are right and once nfs is running the when you run the showmount command you should see what is posted in the /etc/exports file. The exports file is configured correctly.
3. Your /images/dev directory is not complete. You are missing the post init scripts directory.
   Your directory should look like this:

[root@sonic ~]# ls -la /images/dev
total 428
drwxrwxrwx  5 fog  fog    4096 Jun 24 13:11 .
drwxrwxrwx 18 fog  root   4096 May 28 11:11 ..
-rwxrwxrwx  1 fog  fog       0 Mar 16  2016 .mntcheck
drwxrwxrwx  2 fog  root   4096 Oct  4  2018 postinitscripts

You may need to just rerun the fog installer and it should fix this file path.

4. For your nfs service there are a number of sub services like portmapper and such that nfs depends on. Does running sudo systemctl status nfs.service give you any clue to why nfs is not starting?

@jmason If you can still duplicate this random switching of device and linux dev node names we could use your help trying to debug this. We have the idea to query the nvme controller directly to see what its seeing as to order vs what linux is detecting as the order. We need a system that has 2 nvme drives in it that are exhibiting the switching of nvme drives issue. Do you have time to test this?

@Sebastian-Roth

@Sebastian-Roth I agree we need a system with a dual nvme drive so we can compare. I have a pdf of what all the short names really mean that I’ll link in too. I’m suspecting the field you pointed to will tell us how many and then there are other fields that will tell use more details about the drive. Also in the nvme list command I’m interested in seeing if the name space field changes if there are more drives as well as the interaction when these disks change ordinal position in respect to what udev assigns them.

I’m sure your fog server is setup correctly. I want to focus on your target computer. More specifically the network connection. I feel this is a spanning tree issue. As a quick test, do you have a (dumb) unmanaged switch? The dumber the better, like a cheap $30 amazon switch? If you do insert that between your building switch and your pxe booting computer. See if that resolves your issue.

If it does then you will need to get your networking group to look into that building switch. They need to change the STP (spanning tree protoocol) from standard STP to one of the fast STP protocols like RSTP, MSTP, fast-STP, port-fast or whatever your switch vendor calls it.