FOG Kernel Update Errors
-
Hello everyone,
In trying to update the kernel on my FOG server, I was greeted with a credentials error. I apologize, but I did not capture that error and am unable to get back to that state after attempting repairs. I may have made this whole situation a lot worse, I’m not sure.
In trying to troubleshoot the credentials issue, I started updating passwords to match the login for the GUI. The FOG GUI login is just the default with a username of fog and a password of password. That account could not login to the server with those credentials, however, so I updated the server’s account to match that. I then updated the Storage Management area to match that password and also changed it in the TFTP settings to match there. I then changed it in the .fogsettings config file as well.
Now, when trying to update my kernel, I get the error:
Type: 2, File: /var/www/html/fog/lib/fog/fogftp.class.php, Line: 707, Message: ftp_put(): Could not create file., Host: [IP Address], Username: fogI noticed that many of those passwords that I was updating appeared to either be randomly generated or possible done in FOGCrypt? I’m not sure tbh.
My FOG version is 1.4.4 SVN Revision 6077.
The OS on this server is Ubuntu 16.04.3 LTS.All has been working fine until today when trying to image a new Dell Precision machine, which couldn’t get through FOG Registration, so we started with the Kernel update as the next step to resolve that issue. And as a result, I have created more problems than I’ve solved.
If we can get the Kernel update errors resolved, I think I can go from there.Thanks in advance for any support!
EDIT: Redacted the IP Address; completely missed that it was included.
-
Firstly, ah sure you are in a pickle now. Changing passwords at will is not a good place to start. Lets start with this tutorial to get things back in sync: https://forums.fogproject.org/topic/11203/resyncing-fog-s-service-account-password
Hint NEVER change the linux user
fog(in FOG versions 1.5.5 and earlier) orfogprojectif 1.5.6 and later. That service account is managed by FOG. You will break things pretty quickly if you change the fog set password on that account.Now to the point fog was working yesterday and all of a sudden it stops working with no changes by you: https://forums.fogproject.org/topic/10006/ubuntu-is-fog-s-enemy is probably the root of your issue.
-
Well just to point out briefly that FOG didn’t break at any point, it just simply didn’t work for a new machine. It still worked great for other machines no problem. It’s simply a brand new model that we don’t normally work with, which is why I assumed the kernel needed to be updated – we had done that a few times in the past, as we’ve been using FOG for many years here, and it typically resolves our problem.
I’ll follow your tutorial to get things back up and running and report back. Also pointing out that I’m on FOG 1.4.4, so not sure if that service account bit is relevant in my case.
-
Apologies for the double-post, but the tutorial @george1421 linked suggests that I copy/paste the password from the .fogsettings file. In my first post, I noted that I changed the password in that field already as well. Note that this is from instructions on the Wiki when changing passwords, so I figured this was the procedure (and in previous versions of FOG, it was). Now that this has changed, perhaps we need to take a different approach (and update the Wiki please!).
Considering that the password has been updated in Ubuntu, .fogsettings, the Storage Management page, and in TFTP, is there another way to restore this?
-
@kenneth-sisco said in FOG Kernel Update Errors:
Also pointing out that I’m on FOG 1.4.4, so not sure if that service account bit is relevant in my case.
Yep still relevant. That account has been in FOG since the early days. The name was changed because there were bad instructions out on the internet that says to create the linux user
fogand then install FOG with that account.I noted that I changed the password in that field already as well.
Yeah you did it up good then. Start off then by updating the .fogsettings file with a new secure password that you make up, then follow the tutorial again to fix things.
If that linux user
fog’s password is changed or out of sync with reality everything will work fine until you try to capture an image. It will fail with error messages similar to the tutorial. You can also see this if you look on the fog server linux console in the /images/dev directory. When everything is working correctly there should be no directories in that path that resemble ethernet mac address. If there are the FTP mv process has failed with bad credentials.First get everything back in sync password wise then we can work through any error messages you see.
-
Thanks @george1421 . I followed the tutorial to do as you stated. There is now a matching password in .fogsettings, TFTP settings, and in the Storage Node configuration. I re-ran the installer with no errors at all.
I logged back into the GUI and attempted a Kernel Update again, and I am welcomed with the following error:
Type: 2, File: /var/www/html/fog/lib/fog/fogftp.class.php, Line: 707, Message: ftp_put(): Could not create file., Host: [IP Address], Username: fogEDIT: Redacted the IP Address; completely missed that it was included in the error.
-
Apologies again for the double-post, but I’ve been researching further on this error and the Wiki points to this being a permissions issue. I have checked the following items:
/tftpboot is set to 777
/images is set to 777These are the only two directories that I am aware of that are used for the FTP side of things.
I also opted to check the TFTP Server settings under Fog Settings in the GUI and it notes the PXE Kernel Directory being /var/www/fog//service/ipxe/. I checked the permissions on that folder, and they appear to be limited to a www-data group? I’m not sure if the FOG service account is joined to that group by default or if I need to update permissions there? I didn’t see anything about it in the Wiki.
-
@kenneth-sisco Ok now where its stopped is its trying to write the kernel files to /var/www/html/fog/service/ipxe directory. If I remember correctly there was a bug in 1.4.4 where the directory permissions did not let the linux user
fog(or apache I can’t remember which) write into that directory. While there is a slight security risk in this if you issue the commandchmod -R 777 /var/www/html/fog/service/ipxethat will mask the directory permission issue and allow the ftp download to write to that directory. The security risk is that you are setting world read-write access on that directory.If you still can’t get the download to work, you can update the kernel by hand, but lets see where fixing the permissions on the kernel directory gets us.
-
@george1421 Ah thank you – it looks like I got to the same point on that one. I just attempted the command you noted:
sudo chmod -R 777 /var/www/html/fog/service/ipxe
This completed without error, but now I do the command to check:
ls -l /var/www/html/fog/service/ipxeI get that the directory doesn’t exist. I changed your command to instead be a direct copy/paste from the TFTP Server settings in FOG Settings instead. This location is: /var/www/fog/service/ipxe (note the lack of the /html/ directory in the middle). The commands seemed to work in that case.
Tested the Kernel update again and SUCCESS! Looks like we’re in business.
Side note: I don’t really care about security on this box, as it’s shut down 90% of the time. We only power it on when we need it because we don’t want to worry about hardening it.
Thank you for the help. I’m going to do a test run with a client and see how it goes!
-
@kenneth-sisco said in FOG Kernel Update Errors:
/var/www/fog/service/ipxe
Well that’s great you have it sorted out.
Yeah some time after 1.4.4 the path was standardized across all linux distros because you had the debian variants in /var/www/fog/service/ipxe and the rhel variants in /var/www/html/fog/service/ipxe it was too hard to give clear directions to fog admins so the developers merged the paths together.
-
Tested the client and it doesn’t look like the kernel update fixed the problem. I installed PC 32-bit and PC 64-bit kernels (both version 4.19.36, last updated May 1, 2019).
So getting to the crux of the issue I was trying to solve, I am getting the error on the client noted in the screenshot at the end of this post.
Now I know that the network and PXE booting are fine, as I can actually PXE boot to the FOG menu without a problem and it gets an IP Address through that on the subnet that I expect. Once I get to the FOG menu, I select “Full Host Registration and Inventory.” I am then welcomed with the below error message. I believed this to be related to the kernel, which was why I was doing the kernel update in the first place. Was I wrong?
Also, if you need me to create a separate thread for this issue, let me know. I’ll mark this as solved and start a new one no problem.
-
@kenneth-sisco Yeah based on the title this issue is solved. Lets start a new thread with this “new” issue. The developers don’t really like us to mix threads because it confuses the future readers.
But in the mean time as you start a new thread. Kill this current capture/deploy task and then reschedule the task with the debug check box ticked then schedule the task. PXE boot the target computer. You will see several screens of text that you will need to clear with the enter key. After the pages have completed you will be dropped to the FOS Linux command prompt. At the FOS linux command prompt key in
ip addr showandlspci -nn|grep -i netand post the results in the new thread. Also include in that new thread the hardware manufacturer and model of computer that has this issue. -
Great! Let’s mark this one as solved. In short, the issue for the kernel update was related to the permissions on the /var/www/fog/service/ipxe folder, which was a confirmed bug in version 1.4.4.
I’ll move the other issue to a new thread. Cheers!
