• Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login
  • Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login

Database Stress when cloning on Big Environments

Scheduled Pinned Locked Moved Solved
Bug Reports
3
9
769
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • E
    EduardoTSeoane
    last edited by Nov 20, 2019, 9:40 AM

    Hi, @Sebastian-Roth .

    I’m here again, I don’t know where to put this thread, because this is not properly a bug.

    First at all, I have currently 16.500 computers registered on my fog system, with 60 StorageNodes, and growing, located on different buildings on different towns, we are 13 techies.

    I need to notify than the continuous writtings on the database, to report the clonation progress, causes a lot of not needed stress on the database, maybe is a good idea to change the notification system, to reduce stress and make the system more scalable.

    Two ideas:
    Easier: Notification on 33%, 66%, 90%, 100%.
    Best: Get out of the database that informatión and store only the clonation data need by the history.

    1 Reply Last reply Reply Quote 0
    • S
      Sebastian Roth Moderator
      last edited by Dec 1, 2019, 8:32 AM

      @EduardoTSeoane Thanks for reporting this. At first I thought there can’t be that many deployments at the same time to cause stress on the DB but with that many storage nodes I suppose it really can happen.

      I’d suggest you try something to see if that eases the stress. Add the following line to your post init script (if you already use a custom post init script just add it to that or simply add that line to `/images/dev/

      sed -i 's/usleep 3000000$/usleep 30000000/g' /bin/fog.statusreporter
      

      Status updates will then happen only every 30 seconds (3 secs default) and therefore 10 times less updates on the DB.

      Best: Get out of the database that informatión and store only the clonation data need by the history.

      I don’t think we want to get rid of the status reporting altogether because this is what updates the status bar that you see in the web UI task view and people not having physical access to the hosts rely on this status.

      Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

      Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

      E 1 Reply Last reply Dec 9, 2019, 9:38 AM Reply Quote 1
      • Q
        Quazz Moderator
        last edited by Nov 20, 2019, 10:11 AM

        I’d say, only update database itself if task fails or is completed.

        Keep webserver progress update ping as is.

        1 Reply Last reply Reply Quote 0
        • E
          EduardoTSeoane
          last edited by EduardoTSeoane Nov 20, 2019, 7:09 AM Nov 20, 2019, 12:43 PM

          Hehehe i Verified it again!!! @Quazz Task Deploy info on database is updated during progress on deploying at least, a few computers is not problem, but with 16500 and 13 techies working like slaves this can be a cause of database’s stress.

          At least taskPercentText, taskDataCopied, taskTimeRemaining, taskTimeElapsed, taskBPM on task table is updated during partition deploy.

          I think that this info can help to improve this great solution.

          1 Reply Last reply Reply Quote 0
          • S
            Sebastian Roth Moderator
            last edited by Dec 1, 2019, 8:32 AM

            @EduardoTSeoane Thanks for reporting this. At first I thought there can’t be that many deployments at the same time to cause stress on the DB but with that many storage nodes I suppose it really can happen.

            I’d suggest you try something to see if that eases the stress. Add the following line to your post init script (if you already use a custom post init script just add it to that or simply add that line to `/images/dev/

            sed -i 's/usleep 3000000$/usleep 30000000/g' /bin/fog.statusreporter
            

            Status updates will then happen only every 30 seconds (3 secs default) and therefore 10 times less updates on the DB.

            Best: Get out of the database that informatión and store only the clonation data need by the history.

            I don’t think we want to get rid of the status reporting altogether because this is what updates the status bar that you see in the web UI task view and people not having physical access to the hosts rely on this status.

            Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

            Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

            E 1 Reply Last reply Dec 9, 2019, 9:38 AM Reply Quote 1
            • E
              EduardoTSeoane @Sebastian Roth
              last edited by Dec 9, 2019, 9:38 AM

              @Sebastian-Roth yeah, I don’t want te get rid the satus reporting, maybe to do it on another way that don’t use the database, I know that my System maybe is oversized but I want to help to get better performance to all.

              I’m trying the solution on production, when i have some contrasted information I put a quote on this solution.

              1 Reply Last reply Reply Quote 0
              • S
                Sebastian Roth Moderator
                last edited by Dec 9, 2019, 1:44 PM

                @EduardoTSeoane I’d be very interested to hear if this quick fix helped to ease the DB load. We might consider make this value configurable through the web UI then.

                Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                E 1 Reply Last reply Dec 9, 2019, 2:05 PM Reply Quote 0
                • E
                  EduardoTSeoane @Sebastian Roth
                  last edited by Dec 9, 2019, 2:05 PM

                  @Sebastian-Roth said in Database Stress when cloning on Big Environments:

                  @EduardoTSeoane I’d be very interested to hear if this quick fix helped to ease the DB load. We might consider make this value configurable through the web UI then.

                  Good Idea. When I get some tests on production I’ll post it here.

                  1 Reply Last reply Reply Quote 0
                  • S
                    Sebastian Roth Moderator
                    last edited by Dec 19, 2019, 9:17 AM

                    @EduardoTSeoane bumping this topic as well… 😉

                    Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                    Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                    1 Reply Last reply Reply Quote 0
                    • S
                      Sebastian Roth Moderator
                      last edited by Jan 8, 2020, 8:53 PM

                      @EduardoTSeoane I am marking this as solved as I would think the quick fix should help in your environment and we haven’t heard anything negative back yet.

                      Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                      Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                      1 Reply Last reply Reply Quote 0
                      • 1 / 1
                      • First post
                        Last post

                      328

                      Online

                      12.0k

                      Users

                      17.3k

                      Topics

                      155.2k

                      Posts
                      Copyright © 2012-2024 FOG Project