• Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login
  • Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login

Web interface slowdown and FOG Client authentication issues

Scheduled Pinned Locked Moved Solved
FOG Problems
3
14
787
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • J
    j_d
    last edited by j_d Jan 19, 2019, 4:06 PM Jan 19, 2019, 10:05 PM

    Just remembered I forgot to check in yesterday. It was a busy day.

    Turns out I didn’t rerun the FOG install script after copying over the SSL files. I ran it and it seems to have fixed it on some (probably most?) hosts but not all the ones I checked. It must’ve fixed it on most because the web interface is good again (not always, read below).

    Also deployed an image that was made when the previous FOGserver was still in use, and the FOGservice on that machine works perfectly again. So worst case, I re-image all the ones with authentication errors still. However, I noticed during the imaging, the web interface was once again very slow and giving me 503 errors. Completely fine after it finished.

    So it looks like the original problem might be solved and it was my own dumb mistake. I’ll have to check next week to be sure though.


    @Fernando-Gietz said in Web interface slowdown and FOG Client authentication issues:

    FOG version in the new centOS 7 server?
    How many clients are in the host table of fog database?
    How many clients are power on simultaneously?
    PHP version?
    Is a virtual server?

    version 1.5.5
    500-600 clients at least 80-90% of those powered on
    Didn’t the FOG installer install PHP? If so, that’s the version.
    Yes, it’s running on a Hyper-V instance

    Speaking of the amount of clients. A while back, before the issue that caused me to post this thread, I had to double the client check-in time to 600 seconds instead of 300 and completely disabled pinging hosts, to help speed up the web interface. Is there anything I can do so I don’t have to lower those features? It’s slightly inconvenient.

    1 Reply Last reply Reply Quote 0
    • S
      Sebastian Roth Moderator
      last edited by Jan 20, 2019, 7:00 AM

      @j_d said in Web interface slowdown and FOG Client authentication issues:

      So worst case, I re-image all the ones with authentication errors still.

      As I said, doing a “Reset Encryption Data” for all your hosts and they should’ve all synced back in. But now that you went the other way round I don’t think it’s still worth it.

      However, I noticed during the imaging, the web interface was once again very slow and giving me 503 errors. Completely fine after it finished.

      How many clients did you image? Multicast or unicast? This might be interesting to investigate some more but it’s probably not easy to figure out. That said, I have worked on improving “FOG speed” since 1.5.5 was released. So it would be interesting to see if those changes would help in your situation. Take a snapshot of your Hyper-V VM, pull the latest changes from github, checkout dev-branch and re-run the installer:

      cd /path/to/fogproject/
      git pull
      git checkout dev-branch
      cd bin/
      ./installfog.sh -y
      

      This will install PHP version 7.2 as well (CentOS still had PHP 5.6 in FOG version 1.5.5) which speeds up the whole web UI too. So code changes and PHP 7.2 should make a major difference for you.

      Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

      Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

      1 Reply Last reply Reply Quote 0
      • J
        j_d
        last edited by Jan 25, 2019, 2:45 PM

        Is resetting encryption data supposed to help at this point? It doesn’t seem to fix those clients that are still not talking to the server (they still have the same issue in fog.log as the one in the original post). I can’t easily check how many clients are and aren’t working right.
        I’ve also tried uninstalling and reinstalling FOGservice on a client and that didn’t help.
        Deploying a completely new image is the only thing I can do but again I don’t know how to check which ones are still broken.

        @Sebastian-Roth said in Web interface slowdown and FOG Client authentication issues:
        […]Take a snapshot of your Hyper-V VM, pull the latest changes from github, checkout dev-branch and re-run the installer[…]

        That has actually made things a lot snappier. I can now set client polling time back to 300 seconds.

        1 Reply Last reply Reply Quote 0
        • S
          Sebastian Roth Moderator
          last edited by Sebastian Roth Jan 25, 2019, 10:24 AM Jan 25, 2019, 4:16 PM

          @j_d said in Web interface slowdown and FOG Client authentication issues:

          Is resetting encryption data supposed to help at this point?

          Hmmm, my fault. At first I thought it would but looking at it again I see why it doesn’t. I only saw the part of the story where you had a huge load but I missed that you had installed a fresh FOG server.

          So why does it matter? On installing a FOG server a unique CA (certificate authority) certificate/key pair and a webserver certificate/key pair is being generated. A fresh fog-client install grabs the CA certificate from the FOG server and stores this within the Windows cert store. Now you have the CA certificate from your old server stored on all the clients which are all now trying to connect to the new server which has a different CA certificate.

          So you wanna grab the following files from your old server and put in the same place on your new server (don’t just overwrite but backup or move the new ones just in case):

          /opt/fog/snapins/ssl/CA/.fogCA.key
          /opt/fog/snapins/ssl/CA/.fogCA.pem
          /opt/fog/snapins/ssl/.srvprivate.key
          /var/www/html/fog/management/other/ssl/srvpublic.crt
          /var/www/html/fog/management/other/ca.cert.pem
          /var/www/html/fog/management/other/ca.cert.der
          

          Hint: The later two are copies of the first file - CA cert - available to the clients in two different formats, you definitely need those in place if you add new fog-clients later on.

          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

          1 Reply Last reply Reply Quote 0
          • J
            j_d
            last edited by Jan 28, 2019, 4:42 PM

            I have previously copied over the files from /opt/fog/snapins/ssl/, but I didn’t know there were other files in /var/www/html/fog/management/other/ that needed copying. This section in the wiki doesn’t mention it.
            On a related note, Would it be a good idea to have a link to an article on migration on the wiki’s main page? I didn’t even know migrating required any special steps until I ran into these problems.

            I will try copying those /var/www/html/fog/management/other/ files on Thursday. Will that require running installfog.sh again?

            Thanks for the help so far!

            1 Reply Last reply Reply Quote 0
            • S
              Sebastian Roth Moderator
              last edited by Jan 28, 2019, 7:04 PM

              @j_d said in Web interface slowdown and FOG Client authentication issues:

              This section in the wiki doesn’t mention it.

              I think it does: “IMPORTANT: Then re-run the installer.” mentioned just below the cp/scp commands. Sure it might be a but hidden in the gist of information.

              On a related note, Would it be a good idea to have a link to an article on migration on the wiki’s main page?

              There are a lot of things that we should add or update or rearrange in the wiki! But just not enough time to do so. I’d be awesome if people knowing FOG a bit would join in to help on things like documentation - request wiki write access here.

              Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

              Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

              1 Reply Last reply Reply Quote 0
              • J
                j_d
                last edited by Jan 31, 2019, 9:48 AM

                I have now also copied over these files from the old server to the new

                /var/www/html/fog/management/other/ssl/srvpublic.crt
                /var/www/html/fog/management/other/ca.cert.pem
                /var/www/html/fog/management/other/ca.cert.der
                

                I have rerun the installer using

                ./installfog.sh -y
                

                I have rebooted the server.
                … And I don’t think anything changed. FOGService on existing hosts still doesn’t authenticate with the FOGserver.
                Here’s another fog.log, fresh from one of the hosts.

                I have re-deployed an image that was captured months back using the old server, and that host will then authenticate just fine. Same with an image that was made with the new server.
                The problem is with hosts that have been running since before the server migration.
                I’d rather not re-image all those machines and I’m also worried some of the changes I’ve made so far are going to mess things up further down the line.

                1 Reply Last reply Reply Quote 0
                • S
                  Sebastian Roth Moderator
                  last edited by Feb 7, 2019, 9:08 PM

                  @j_d Sorry for my late reply. Sort of lost track of this over a busy week. We’ll probably best try to track this down by comparing the CA certificate hashes to see if those match. Do you know how to get certificate hashes both on Windows and Linux?

                  Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                  Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                  1 Reply Last reply Reply Quote 0
                  • S
                    Sebastian Roth Moderator
                    last edited by Feb 9, 2019, 8:42 PM

                    @j_d Here is some more information on how to check the certificate hashes/fingerprints. On your FOG server run:

                    openssl x509 -noout  -fingerprint -sha1 -inform pem -in /opt/fog/snapins/ssl/CA/.fogCA.pem 
                    openssl x509 -noout  -fingerprint -sha1 -inform pem -in /var/www/html/fog/management/other/ca.cert.pem
                    

                    Those two must definitely match. Note it down (first and last three blocks should be enough) and go to one of your clients that shows the issue. Here hit Win key + R and mmc. From the menu click File -> Add/Remove Snap-in -> Certificates -> Computer accont -> Local computer. Now when you have the certificate view navigate to Certificates (Local Computer) -> Trusted Root Certification Authorities -> Certificates and double click the one called and “FOG Server CA” -> Details tab -> field Thumbprint - see if that matches the output of the above openssl commands.

                    cert_win.jpg

                    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                    1 Reply Last reply Reply Quote 0
                    • S
                      Sebastian Roth Moderator
                      last edited by Feb 9, 2019, 8:57 PM

                      @j_d As well I may ask you to validate the certificate is matching the CA:

                      openssl verify -CAfile /var/www/html/fog/management/other/ca.cert.pem /var/www/html/fog/management/other/ssl/srvpublic.crt
                      

                      Should tell you OK.

                      From the client log you posted it clearly states to be a certificate/CA issue:

                      Middleware::Authentication ERROR: Certificate is not from FOG CA
                      

                      Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                      Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                      1 Reply Last reply Reply Quote 0
                      • 1 / 1
                      • First post
                        Last post

                      157

                      Online

                      12.0k

                      Users

                      17.3k

                      Topics

                      155.2k

                      Posts
                      Copyright © 2012-2024 FOG Project