Unable to update Kernel after upgrade

[george1421]

A couple of things here.

1. The linux service account was changed from fog to fogproject because there were bad instructions on the internet that said to create a user account called fog and install FOG using that account. The FOG installer takes over that account changing its password and effectively locking out the user. And/Or people would reset the password for the linux account fog and then just make a mess of things.

2. It sounds like the fogproject account doesn’t have write access to the /images directory. FOG uploads the captured image to the /images/dev share then logs into the FOG server (from the target computer) using the FTP program and uses the FTP server to move the uploaded files to the /images directory from the /images/dev directory. This is the point where the upload is failing in your message. You should see a similar error message on the target computer during image capture.

So for number 1 run through this tutorial and ensure the user ID is fogproject and the password matches the password found in the /opt/fog/.fogsettings file. https://forums.fogproject.org/topic/11203/resyncing-fog-s-service-account-password

For the second issue what do you get when you key in these two commands. Post the resluts from both here.

ls -la /

ls -la /images

[Jpolk91]

@george1421 thanks for your reply! I didn’t realize how late it was when I posted so sorry for the slow response!

1. That makes total sense, I really need to be better about keeping up on whats happening. I saw in that link that .fogsettings is hidden (I forgot to check for hidden files initially) so that explains why there is a fogsettings1 in that folder and why it has fog as the un/pw. The password was correct in .fogsettings however and I pasted it everywhere else (and made sure I could log in as that user) but I went ahead and reran the installer again, same error after.

2. here is the output from those commands. Excuse the mess in /images I need to clean that out pretty badly the only two I really use are:

drwxrwxrwx  2 fogproject root   4096 Apr 25  2019 GenericDesktop
drwxrwxrwx  2 fogproject root   4096 Apr 25  2019 GenericLaptop4-22-19

ls -la /

total 136
drwxr-xr-x  27 root       root  4096 Jan  3 09:05 .
drwxr-xr-x  27 root       root  4096 Jan  3 09:05 ..
drwxr-xr-x   2 root       root 12288 Jan  2 12:13 bin
drwxr-xr-x   4 root       root  4096 Jan  2 12:20 boot
drwxrwxr-x   2 root       root  4096 Jun 14  2017 cdrom
drwxr-xr-x  19 root       root  4240 Jan  2 14:20 dev
drwxr-xr-x 139 root       root 12288 Jan  3 09:05 etc
drwxr-xr-x   9 root       root  4096 Jan  3 09:03 home
drwxrwxrwx  31 fogproject root  4096 Jun  6  2019 images
lrwxrwxrwx   1 root       root    32 Jun 14  2017 initrd.img -> boot/initrd.img-4.4.0-79-generic
drwxr-xr-x  24 root       root  4096 Feb  5  2019 lib
drwxr-xr-x   2 root       root  4096 Jan  2 12:08 lib64
drwx------   2 root       root 16384 Jun 14  2017 lost+found
drwxr-xr-x   3 root       root  4096 Jun 14  2017 media
drwxr-xr-x   2 root       root  4096 Jun 14  2017 mnt
drwxr-xr-x   3 root       root  4096 Jun 14  2017 opt
dr-xr-xr-x 308 root       root     0 Jan  2 13:18 proc
drwx------   8 root       root  4096 Feb  5  2019 root
drwxr-xr-x  37 root       root  1100 Jan  3 09:05 run
drwxr-xr-x   2 root       root 12288 Jan  2 12:15 sbin
drwxr-xr-x   2 root       root  4096 Feb  5  2019 snap
drwxr-xr-x   3 root       root  4096 Jun 14  2017 srv
dr-xr-xr-x  13 root       root     0 Jan  2 13:18 sys
drwxr-xr-x   6 fogproject root  4096 Feb  5  2019 tftpboot
drwxr-xr-x   6 root       root  4096 Jan  3 09:05 tftpboot.prev
drwxrwxrwt  27 root       root 12288 Jan  3 09:09 tmp
drwxr-xr-x  11 root       root  4096 Jun 14  2017 usr
drwxr-xr-x  15 root       root  4096 Jun 14  2017 var
lrwxrwxrwx   1 root       root    29 Jun 14  2017 vmlinuz -> boot/vmlinuz-4.4.0-79-generic

ls -la /images

total 124
drwxrwxrwx 31 fogproject root   4096 Jun  6  2019 .
drwxr-xr-x 27 root       root   4096 Jan  3 09:05 ..
drwxrwxrwx  2 fogproject root   4096 Mar 21  2018 1040g3
drwxrwxrwx  2 fogproject dhq-it 4096 Jun 13  2017 6470b
drwxrwxrwx  2 fogproject root   4096 Nov  3  2017 650G1
drwxrwxrwx  2 fogproject root   4096 Sep  7  2017 650g2
drwxrwxrwx  2 fogproject dhq-it 4096 Jun 13  2017 6550b
drwxrwxrwx  2 fogproject root   4096 Oct 20  2017 6560b
drwxrwxrwx  2 fogproject root   4096 Oct 19  2017 6570b
drwxrwxrwx  2 fogproject dhq-it 4096 Jun 13  2017 6710
drwxrwxrwx  2 fogproject dhq-it 4096 Jun 13  2017 6730b2017
drwxrwxrwx  2 fogproject root   4096 Oct 18  2017 6730BLoaner
drwxrwxrwx  2 fogproject dhq-it 4096 Jun 13  2017 7900us
drwxrwxrwx  2 fogproject dhq-it 4096 Jun 13  2017 7900us121316
drwxrwxrwx  2 fogproject dhq-it 4096 Jun 14  2017 8000us
drwxrwxrwx  2 fogproject root   4096 Jul 21  2017 800g2mini
drwxrwxrwx  2 fogproject root   4096 Feb  5  2019 800g4mini2-5-19
drwxrwxrwx  2 fogproject dhq-it 4096 Jun 13  2017 8200SFF
drwxrwxrwx  2 fogproject root   4096 Sep  7  2017 8200SFFupdated
drwxrwxrwx  2 fogproject dhq-it 4096 Jun 13  2017 8200US
drwxrwxrwx  2 fogproject root   4096 Aug 23  2017 850g3
drwxrwxrwx  2 fogproject dhq-it 4096 Jun 13  2017 DC7800
drwxrwxrwx  6 fogproject root   4096 Jun  6  2019 dev
drwxrwxrwx  2 fogproject root   4096 Sep 12  2017 EliteOne800AIO
drwxrwxrwx  2 fogproject root   4096 Jun 26  2017 Everfi
drwxrwxrwx  2 fogproject root   4096 Jun  6  2019 GeneralDesktop-800g4mini06-19
drwxrwxrwx  2 fogproject root   4096 Apr 25  2019 GenericDesktop
drwxrwxrwx  2 fogproject root   4096 Apr 25  2019 GenericLaptop4-22-19
-rwxrwxrwx  1 fogproject root      0 Jun 14  2017 .mntcheck
drwxrwxrwx  2 fogproject root   4096 Jun 14  2017 postdownloadscripts
drwxrwxrwx  2 fogproject root   4096 Feb  2  2018 ProBook6470b
drwxrwxrwx  2 fogproject dhq-it 4096 Jun 13  2017 Probook650G1

[george1421]

@Jpolk91 OK with the fogproject user set correctly then it must be a permission issue. Before we go resetting permissions at will lets confirm the issue.

From a windows computer use the command line ftp program to connect to your FOG server. use the login of fogproject and the password gleaned from the .fogsettings file.

Then execute these commands in the ftp session.

cd /images/dev
mkdir test1
rmdir test1
cd /images
mkdir test1
rmdir test1
exit

That is basically the permissions fog uses when capturing images. If you can’t perform the steps above then run these commands.

To reset the ownership of the files in the /images share you can run this command sudo chown -R fogproject:root /images

And then to reset the permissions (its a bit lax but will for sure fix any permission issue) sudo chmod -R 777 /images The -R means directories and all files below and 777 is read/write for everyone.

[Sebastian Roth]

@Jpolk91 While George is pointing out the right configs to look at I think the issue is not about upload/download an image but only the error message initially posted about kernel update, right??

So I’d suggest you follow @george1421’s advice on testing the FTP connection but instead of using /images you want to test this:

cd /var/www/html/fog/service/ipxe
mkdir backup
put testfile.bin

The last line simply means uploading a binary testfile.

[george1421]

@Sebastian-Roth said in Unable to update Kernel after upgrade:

While George is pointing out the right configs to look at I think the issue is not about upload/download an image but only the error message initially posted about kernel update, right??

Ugh, I needed to “read” what it said vs what normally is wrong. Thanks Sebastian for slapping me back to reality, its still 2019 right…

There is/was a bug in FOG 1.4.4 where the permissions were not set correctly on /var/www/html/fog/service/ipxe to allow downloading of the kernels via the web gui. You could do this manually as root or sudo as root but not from the web gui. I would have thought a later fog installer script would have fixed that issue??

Lets get the output on the ipxe directory using the same commands we used on /images:
ls -la /var/www/html/fog/service
ls -la /var/www/html/fog/service/ipxe

We need to make sure that apache has write access into the ipxe directory for the web ui kernel updater to work correctly.

[Jpolk91]

@george1421 @Sebastian-Roth I’m back, This one took forever to do and I was interrupted several times. one road block to be aware of having people do this in the future is windows FTP.EXE has a password character limit of 32 so if the auto-gen password is longer than that (mine is) it gets Truncated you can see this in action using FTP -d on windows and looking at the commands its actually sending, I don’t know what genius decided that was the limit but I can’t figure out how to get around it That being said…

@george1421 FTP testing:
I was able to get ftp’d into the fog server using Filezilla and can create and remove folders in both /images and /images/dev and then see the changes locally on the server. Here is the filezilla “log” history logged in as fogproject

Status:    Logged in 

Status:    Retrieving directory listing... 

Status:    Calculating timezone offset of server... 

Status:    Timezone offset of server is 0 seconds. 

Status:    Directory listing of "/home/fogproject" successful 

Status:    Retrieving directory listing of "/"... 

Status:    Directory listing of "/" successful 

Status:    Retrieving directory listing of "/images"... 

Status:    Directory listing of "/images" successful 

Status:    Creating directory '/images/New directory'... 

Status:    Retrieving directory listing of "/images/New directory"... 

Status:    Directory listing of "/images/New directory" successful 

Status:    Retrieving directory listing of "/images"... 

Status:    Directory listing of "/images" successful 

Status:    Retrieving directory listing of "/images"... 

Status:    Directory listing of "/images" successful 

Status:    Retrieving directory listing of "/images/dev"... 

Status:    Directory listing of "/images/dev" successful 

Status:    Creating directory '/images/dev/New directory'... 

Status:    Retrieving directory listing of "/images/dev/New directory"... 

Status:    Directory listing of "/images/dev/New directory" successful 

Status:    Retrieving directory listing of "/images/dev"... 

Status:    Directory listing of "/images/dev" successful 

Permissions
ls -la /var/www/html/fog/service

total 184
drwxr-xr-x  3 www-data   www-data 4096 Jan  3 09:03 .
drwxr-xr-x 10 www-data   www-data 4096 Jan  3 09:05 ..
-rw-r--r--  1 www-data   www-data  675 Jan  3 09:03 alo-bg.php
-rw-r--r--  1 www-data   www-data  632 Jan  3 09:03 autologout.php
-rw-r--r--  1 www-data   www-data  551 Jan  3 09:03 auto.register.php
-rw-r--r--  1 www-data   www-data 1178 Jan  3 09:03 av.php
-rw-r--r--  1 www-data   www-data  582 Jan  3 09:03 blame.php
-rw-r--r--  1 www-data   www-data  568 Jan  3 09:03 capone.php
-rw-r--r--  1 www-data   www-data 1041 Jan  3 09:03 checkcredentials.php
-rw-r--r--  1 www-data   www-data  642 Jan  3 09:03 debug.php
-rw-r--r--  1 www-data   www-data  724 Jan  3 09:03 dircleanup-dirs.php
-rw-r--r--  1 www-data   www-data  646 Jan  3 09:03 displaymanager.php
-rw-r--r--  1 www-data   www-data 1574 Jan  3 09:03 getversion.php
-rw-r--r--  1 www-data   www-data  594 Jan  3 09:03 greenfog.php
-rw-r--r--  1 www-data   www-data 1289 Jan  3 09:03 grouplisting.php
-rw-r--r--  1 www-data   www-data 8331 Jan  3 09:03 hostinfo.php
-rw-r--r--  1 www-data   www-data 1190 Jan  3 09:03 hostnameloop.php
-rw-r--r--  1 www-data   www-data  735 Jan  3 09:03 hostname.php
-rw-r--r--  1 www-data   www-data 1289 Jan  3 09:03 imhttps://stackoverflow.com/questions/55045291/azure-ftp-login-via-command-line-batch-file-failsagelisting.php
-rw-r--r--  1 www-data   www-data 2498 Jan  3 09:03 inventory.php
drwxr-xr-x  2 fogproject www-data 4096 Jan  3 09:03 ipxe
-rw-r--r--  1 www-data   www-data  620 Jan  3 09:03 jobs.php
-rw-r--r--  1 www-data   www-data  847 Jan  3 09:03 locationcheck.php
-rw-r--r--  1 www-data   www-data 1349 Jan  3 09:03 locationlisting.php
-rw-r--r--  1 www-data   www-data  559 Jan  3 09:03 man.hostexists.php
-rw-r--r--  1 www-data   www-data  575 Jan  3 09:03 mc_checkin.php
-rw-r--r--  1 www-data   www-data  580 Jan  3 09:03 Post_Stage2.php
-rw-r--r--  1 www-data   www-data  588 Jan  3 09:03 Post_Stage3.php
-rw-r--r--  1 www-data   www-data  578 Jan  3 09:03 Post_Wipe.php
-rw-r--r--  1 www-data   www-data  551 Jan  3 09:03 Pre_Stage1.php
-rw-r--r--  1 www-data   www-data 1292 Jan  3 09:03 printerlisting.php
-rw-r--r--  1 www-data   www-data  625 Jan  3 09:03 PrinterManager.php
-rw-r--r--  1 www-data   www-data  561 Jan  3 09:03 Printers.php
-rw-r--r--  1 www-data   www-data 2150 Jan  3 09:03 progress.php
-rw-r--r--  1 www-data   www-data  845 Jan  3 09:03 register.php
-rw-r--r--  1 www-data   www-data  739 Jan  3 09:03 servicemodule-active.php
-rw-r--r--  1 www-data   www-data 1772 Jan  3 09:03 snapcheck.php
-rw-r--r--  1 www-data   www-data 1309 Jan  3 09:03 snapinlisting.php
-rw-r--r--  1 www-data   www-data  626 Jan  3 09:03 snapins.checkin.php
-rw-r--r--  1 www-data   www-data  632 Jan  3 09:03 snapins.file.php
-rw-r--r--  1 www-data   www-data 4056 Jan  3 09:03 Test.php
-rw-r--r--  1 www-data   www-data  642 Jan  3 09:03 updates.php
-rw-r--r--  1 www-data   www-data  667 Jan  3 09:03 usercleanup-users.php
-rw-r--r--  1 www-data   www-data  601 Jan  3 09:03 usertracking.report.php

ls -la /var/www/html/fog/service/ipxe

total 54692
drwxr-xr-x 2 fogproject www-data     4096 Jan  3 09:03 .
drwxr-xr-x 3 www-data   www-data     4096 Jan  3 09:03 ..
-rw-r--r-- 1 fogproject www-data     1966 Jan  3 09:03 advanced.php
-rw-r--r-- 1 fogproject www-data    16272 Jan  3 09:03 bgdark.png
-rw-r--r-- 1 fogproject www-data    21280 Jan  3 09:03 bg.png
-rw-r--r-- 1 fogproject www-data     1139 Jan  3 09:03 boot.php
-rw-r--r-- 1 fogproject www-data  8389280 Jan  3 09:03 bzImage
-rw-r--r-- 1 fogproject www-data  7808288 Jan  3 09:03 bzImage32
-rw-r--r-- 1 fogproject www-data   234697 Jan  3 09:03 grub.exe
-rw-r--r-- 1 fogproject www-data      592 Jan  3 09:03 index.php
-rw-r--r-- 1 fogproject www-data 18865084 Jan  3 09:03 init_32.xz
-rw-r--r-- 1 fogproject www-data 19689768 Jan  3 09:03 init.xz
-rw-r--r-- 1 fogproject www-data    25340 Jan  3 09:03 memdisk
-rw-r--r-- 1 fogproject www-data  1839104 Jan  3 09:03 memtest.bin
-rw-r--r-- 1 fogproject www-data    29719 Jan  3 09:03 refind.conf
-rw-r--r-- 1 fogproject www-data   262592 Jan  3 09:03 refind.efi
-rw-r--r-- 1 fogproject www-data   201600 Jan  3 09:03 refind_ia32.efi
-rw-r--r-- 1 fogproject www-data   208776 Jan  3 09:03 refind_x64.efi

@Sebastian-Roth FTP testing
I am able to access /var/www/html/fog/service/ipxe
the backup directory already exists
I am unable to create a file in that folder. from the filezilla log

Status:    Logged in 

Status:    Starting upload of C:\Users\[my windows profile]\Desktop\test.bin 

Command:    CWD /var/www/fog/service/ipxe/backup 

Response:    250 Directory successfully changed. 

Command:    TYPE I 

Response:    200 Switching to Binary mode. 

Command:    PASV 

Response:    227 Entering Passive Mode (10,104,1,104,252,158). 

Command:    STOR test.bin 

Response:    553 Could not create file. 

Error:    Critical file transfer error 

So i’m assuming it is a permission error somewhere in the ipxe folder, i’ll wait and let you guys tell me what to actually run so I don’t screw it up.

[Sebastian Roth]

@Jpolk91 said in Unable to update Kernel after upgrade:

ls -la /var/www/html/fog/service/ipxe

Maybe I am blind but I can’t see directory backup here!! Later in your post you say “the backup directory already exists”?!

CWD /var/www/fog/service/ipxe/backup

Watch out /var/www/fog and /var/www/html/fog - one should be just a link to the other directory. Please run the following commands and post output here:

ls -al /var/www/
ls -al /var/www/html/

[Jpolk91]

“Watch out /var/www/fog and /var/www/html/fog”

@Sebastian-Roth I was definately in /var/www/fog earlier (i must have missed the html part, sorry)
its strange if they’re supposed to be a link though because I have
“/var/www/fog/service/ipxe/backup”
but
“/var/www/html/fog/service/ipxe/backup” doesn’t exist

heres those commands
ls -al /var/www/

total 16
drwxr-xr-x  4 root     root     4096 Jun 26  2017 .
drwxr-xr-x 15 root     root     4096 Jun 14  2017 ..
drwxr-xr-x 11 www-data www-data 4096 Jun 26  2017 fog
drwxr-xr-x  3 root     root     4096 Jan  3 09:03 html

ls -al /var/www/html/

total 24
drwxr-xr-x  3 root     root      4096 Jan  3 09:03 .
drwxr-xr-x  4 root     root      4096 Jun 26  2017 ..
drwxr-xr-x 10 www-data www-data  4096 Jan  3 09:05 fog
-rw-r--r--  1 root     root     10701 Jun 26  2017 index.html

[Tom Elliott]

@Jpolk91 So it seems that you have a double install going on.

Can you do:

mv /var/www/fog/ /var/www/fog_old
ln -s /var/www/html/fog /var/www/fog
chown -R www-data:www-data /var/www/html/fog /var/www/fog
chown -R fogproject:www-data /var/www/html/fog/service/ipxe
chmod -R 777 /var/www/html/fog/service/ipxe

Then retry the kernel update.

[Jpolk91]

@Tom-Elliott

1. yep, that did it. I’ve got 5.1.16 (NVME fix) installed which is what I wanted since this machine has a (i think) newer NVME drive and figured it may help, unfortunately my image is still hanging so all that and still work to do but no worries there I’ll go make another post on that in the appropriate forum if I can’t get it going! [Edit 3:45pm theres already a post re: elitebook 830g6! horray! gonna go dig there]

2. Is there anything else I may need to do or be aware of re: the double install or you think I should be good?

3. also I know it would be hard to guess with what we’ve shared and my assumption is I did something stupid without realizing it (I really need to find time to learn more about linux in general especially since I need to keep this server alive) but is there a “common” reason these issues may happen so I can avoid it in the future?

4. thanks so much to you @Sebastian-Roth and @george1421 you guys are all fantastic here! Absolutely top rate.

[george1421]

@Jpolk91 said in Unable to update Kernel after upgrade:

yep, that did it. I’ve got 5.1.16 (NVME fix) installed which is what I wanted since this machine has a (i think) newer NVME drive and figured it may help, unfortunately my image is still hanging so all that and still work to do but no worries there I’ll go make another post on that in the appropriate forum if I can’t get it going!

Yes please create a new post for this. There is a potential fix for this not related to the kernel. We do like to keep the problems and solutions to match the forum subject line so not to confuse future readers. The solution “depending on what the actual problem is” could be as simple as a kernel parameter change. Having pictures of errors will also help us understand the context of the error over just having the error message.

[Sebastian Roth]

@Jpolk91 Great to hear this has fixed the issue for you!

Is there anything else I may need to do or be aware of re: the double install or you think I should be good?

Not that I know of from the top of my head. While I cannot promise you anything I would leave things as is and wait if any issue comes up.

At some point you might need to think about updating your Ubuntu install but 16.04 LTS is still fine at the moment. No rush.

also I know it would be hard to guess with what we’ve shared and my assumption is I did something stupid without realizing it (I really need to find time to learn more about linux in general especially since I need to keep this server alive) but is there a “common” reason these issues may happen so I can avoid it in the future?

I don’t think it is something you did. If I remember correctly there was an issue with this link in an older version. As you updated FOG from 1.4.x there is a chance that old version has messed it up. Sorry.

Learning Linux is a live long task. Even for me, working on Linux every day for many years. Just keep on playing with it and find yourself getting more and more comfortable. A great way is setting up a test environment in a VM where you can snapshot, test and re-install without much trouble…

[Jpolk91]

@Sebastian-Roth

At some point you might need to think about updating your Ubuntu install but 16.04 LTS is still fine at the moment. No rush.

Yeah thats definately on my to do list!

I don’t think it is something you did. If I remember correctly there was an issue with this link in an older version. As you updated FOG from 1.4.x there is a chance that old version has messed it up. Sorry.

no problem! just wanted to be sure to not repeat it if it were my mistake haha

Learning Linux is a live long task. Even for me, working on Linux every day for many years. Just keep on playing with it and find yourself getting more and more comfortable. A great way is setting up a test environment in a VM where you can snapshot, test and re-install without much trouble…

Yeah, I think I may try and work out of a VM every so often. Tried it before but the timing was unfortunate and I got literally buried at work so I never went back to it but it may be time to give it another go. thanks again for all you’re help! I’ll be back on Monday haha