FOG Project

    • Register
    • Login
    • Search
    • Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search

    Solved Database update failure after image capture

    FOG Problems
    3
    18
    250
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      MikeBC last edited by

      ![alt text](Screenshot2.jpg image url)phpfpmerrorlog.jpg Screenshot1.jpg

      Please could I get some assistance to resolve this issue. Some brief context of how the issue happened is, after extended power outrages and the server shutting down due to UPS dying. I ran yum updates and the problem went away previously.
      This time it did not work. I ran yum updates and updated Fog to version 1.5.9 from 1.5.7 and this has not solved the issue. I can deploy images without error/issue only capturing is an issue presently, which I need to urgently resolve. The error happens after image has captured and database needs to update.
      Server is running Centos 7, Gnome 3.28.2

      I have ensured the user credentials in TFTP under Fog settings and the Storagemanagement-Storage Nodes are identical.

      I would really appreciate a fix/solution so as to mitigate further problems when power outages happen again in future.

      1 Reply Last reply Reply Quote 0
      • M
        MikeBC @george1421 last edited by

        @george1421 Thank you. I will delete them shortly to avoid any further issues.

        1 Reply Last reply Reply Quote 0
        • george1421
          george1421 Moderator @MikeBC last edited by

          @MikeBC In your last picture of /images/dev directory. There are several directories there with mac looking addresses. These are botched uploads. You can delete the directories and its contents they are just taking up space.

          rm -rf /images/dev/082e5f1dface
          rm -rf /images/dev/64006a486167
          rm -rf /images/dev/ecf4bb70a96c
          

          unless you are currently capturing an image you should never have mac style named directories in /images/dev

          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

          M 1 Reply Last reply Reply Quote 0
          • M
            MikeBC last edited by

            Looks there was finally as solution. I ran yum update -y today and after a server reboot the image capture with database update worked 100%. I must thank @Sebastian-Roth for assisting me in the chat bubble running various scripts and looking many possible fixes to try remedying the issue experienced. The support received was fantastic, thank you kindly.

            1 Reply Last reply Reply Quote 0
            • M
              MikeBC @Sebastian Roth last edited by Sebastian Roth

              @Sebastian-Roth

              [root@imagingout ~]# ls -al /images/dev
              total 0
              drwxrwxrwx.  6 fogproject root 130 Oct  6 08:20 .
              drwxrwxrwx. 10 fogproject root 186 Oct  6 08:21 ..
              drwxrwxrwx   2 fogproject root 190 Mar 20  2020 082e5f1dface
              drwxrwxrwx   2 fogproject root 222 Jan 14  2020 64006a486167
              drwxrwxrwx   2 fogproject root   6 Apr 30 15:39 ecf4bb70a96c
              -rwxrwxrwx.  1 fogproject root   0 Oct 11  2019 .mntcheck
              drwxrwxrwx.  2 fogproject root  34 Oct 11  2019 postinitscripts
              [root@imagingout ~]# cat /etc/exports
              /images *(ro,sync,no_wdelay,no_subtree_check,insecure_locks,no_root_squash,insecure,fsid=0)
              /images/dev *(rw,async,no_wdelay,no_subtree_check,no_root_squash,insecure,fsid=1)
              [root@imagingout ~]#
              
              george1421 1 Reply Last reply Reply Quote 0
              • S
                Sebastian Roth Moderator last edited by Sebastian Roth

                @MikeBC Seems like it’s not creating the upload directory or maybe it does but in a different place. Please run these commands (again) and post output here:

                ls -al /images/dev
                cat /etc/exports
                

                We can also switch to chat for faster turnaround - chat bubble in the right top corner.

                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                M 1 Reply Last reply Reply Quote 0
                • M
                  MikeBC @Sebastian Roth last edited by

                  @Sebastian-Roth I ran the capture twice once to the /images/Client directory and then to the /images/ directory both times same error occurred that I have been experiencing. I have attached output.
                  Screenshot from 2020-10-06 08-29-59.png

                  1 Reply Last reply Reply Quote 0
                  • S
                    Sebastian Roth Moderator last edited by

                    @MikeBC Looking through the logs again I saw this at the bottom end:

                    [Mon Oct 05 10:29:844700 2020] [proxy_fcgi:error] [pid 55573] (70007)The timeout specified has expired: [client 10.0.3.129:45100] AH01075: Error dispatching request to :
                    

                    I can’t make sense of this as you don’t seem to have many clients or others sending in requests that would cause I high load on Apache and PHP-FPM. Searching the forum a bit I found this similar sounding topic that seems to never have been solved: https://forums.fogproject.org/topic/14563/snapin-problem-on-fog-1-5-9-rc1-and-1-5-9-rc2

                    The other error messages we see from today are related to the FTP commands used to rename /images/dev/f8bc1265c913 to it’s destination in /images/.... I really wonder if /images/dev/f8bc1265c913 really exists at this point?? It should be created when the machine runs the task to temporarily store the uploaded image files. Please run ls -al /images/dev/f8bc1265c913 and post output here.

                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                    M 1 Reply Last reply Reply Quote 0
                    • M
                      MikeBC @Sebastian Roth last edited by

                      @Sebastian-Roth I ran the task at about 10H30 or so. I wasn’t in front of the PC at the time to verify error on completion. I will run it again shortly and provide error message again.

                      1 Reply Last reply Reply Quote 0
                      • S
                        Sebastian Roth Moderator last edited by

                        @MikeBC What time did you run the task? I only see a few log messages for today. Which error did you get on the screen this time?

                        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                        M 1 Reply Last reply Reply Quote 0
                        • M
                          MikeBC @Sebastian Roth last edited by

                          @Sebastian-Roth I have attached logs as requested
                          Screenshot from 2020-10-05 10-48-23.png
                          Screenshot from 2020-10-05 10-47-49.png
                          Screenshot from 2020-10-05 10-48-10.png

                          1 Reply Last reply Reply Quote 0
                          • S
                            Sebastian Roth Moderator last edited by

                            @MikeBC We need to figure out why you get HTTP 503. As you can obviously use the FOG web UI to schedule new tasks it doesn’t seem to be a general Apache/PHP issue. So please schedule another task and let the host run the task. Meanwhile open a root shell on your FOG server and run tail -f /var/log/php-fpm/* /var/log/httpd/error_log (note there is a space between the two) to see what messages come in when the task finishes and shows the error message.

                            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                            Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                            Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                            M 1 Reply Last reply Reply Quote 0
                            • M
                              MikeBC @Sebastian Roth last edited by

                              @Sebastian-Roth I have supplied output as requested

                              Screenshot6.jpg

                              1 Reply Last reply Reply Quote 0
                              • S
                                Sebastian Roth Moderator last edited by

                                @MikeBC Please run find /var/log -name "*php*" as root and post output here.

                                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                M 1 Reply Last reply Reply Quote 0
                                • M
                                  MikeBC @Sebastian Roth last edited by

                                  @Sebastian-Roth incidentally yes they did. However, I did delete that host and used a separate machine to test and problem with the 503 service unavailable still remains/persists. I also tried the dash and same 503 error came up.

                                  1 Reply Last reply Reply Quote 0
                                  • S
                                    Sebastian Roth Moderator last edited by

                                    @MikeBC said in Database update failure after image capture:

                                    The image path listed is to a Client folder for client specific images.

                                    Ok, got me. I never used sub-directories in /images and didn’t even know this would work.

                                    Then we should look into why you get the 503 Service unavailable and No Active Task found for Host. Did those two messages happen on the same machine one after the other?

                                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                    M 1 Reply Last reply Reply Quote 0
                                    • M
                                      MikeBC @Sebastian Roth last edited by

                                      @Sebastian-Roth Thank you for assisting. The image path listed is to a Client folder for client specific images. So some context to this. We have client specific images captured to a Client folder and then we have oem images for a wide variety of OEM vendors relating to makes and models in an oem folder. I also have a Test folder R&D/development purposes. I will try the dash, but I have captured over 300 images using this method with the slash. I only capture as needed when required. We focus on deployment of captured imaged.

                                      Regarding the hosts query, my techs boot via pxe to deploy as such apart from the server running I don’t have any other hosts running, if I have this answered this correctly. We do have up to 50 different PC’s having an image being deployed at a time, not sure if this the load issue? I have a separate server as a back-up that runs fine with both capture and deployment, that is running version 1.5.8

                                      We don’t use a proxy server for our network. Here is the version detail on login page:
                                      Latest version: 1.5.9
                                      Latest development version: 1.5.9.5

                                      Also, I have updated kernel: bzImage version: 5.6.18 & bzImage32 version: 5.6.18

                                      I have uploaded output of commands requested.

                                      Screenshot3.jpg Screenshot4.jpg Screenshot5.jpg

                                      1 Reply Last reply Reply Quote 0
                                      • S
                                        Sebastian Roth Moderator last edited by

                                        @MikeBC There seem to be a few things coming together in your case. We’ll try to look into those one by one. The first and most important one is the name you gave the image Client/Skillspro_Win10...! It contains a slash. The slash is used on Linux systems as directory separator and it does cause trouble of you use it. We’ve had someone else just a week ago and we are looking into preventing this from the FOG web UI so people just cannot use a slash in the image name. So as a quick fix you edit the image and change name and path to something without / - maybe just use a simple dash (-) instead.

                                        Please run the following commands and post output here:

                                        ls -al /images
                                        ls -al /images/dev
                                        mount
                                        df -h
                                        

                                        The first picture showing a 503 Service Unavailable might mean that you have many fog-clients sending requests and putting too much load on your server. How many hosts with fog-client installed do you have?

                                        All the ftp_rename and ftp_put warnings can be ignored because they happen because of the slash in your image name/path.

                                        The series of ftp_login(): login incorrect messages can be ignored as well I would think. My guess is you played with the credentials at some point when trying to solve this issue. As we see ftp_rename error after that again I am fairly sure you have the credentials correct again.

                                        The PHP warnings Invalid argument supplied... stem from some code I added not long ago that pulls the current version information from github.com. Though I have tested this stuff on my setup it seems like it doesn’t work in all cases. Do you see the version information on the FOG web UI login page? Do you use a proxy server in your network?

                                        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                                        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                                        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                                        M 1 Reply Last reply Reply Quote 0
                                        • 1 / 1
                                        • First post
                                          Last post

                                        182
                                        Online

                                        10.2k
                                        Users

                                        16.3k
                                        Topics

                                        149.9k
                                        Posts

                                        Copyright © 2012-2020 FOG Project