• Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Register
    • Login

    Fog server keeps going down

    Scheduled Pinned Locked Moved Solved
    FOG Problems
    7
    103
    84.1k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Wayne WorkmanW
      Wayne Workman @szecca1
      last edited by

      @szecca1 I have no idea, I don’t have any VMWare experience… but probably the other @Developers and @Moderators could chime in on that.

      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
      Daily Clean Installation Results:
      https://fogtesting.fogproject.us/
      FOG Reporting:
      https://fog-external-reporting-results.fogproject.us/

      S 1 Reply Last reply Reply Quote 0
      • S
        szecca1 @Wayne Workman
        last edited by

        @Wayne-Workman Better question, why would this randomly just start happening. The server was working fine until recently

        Wayne WorkmanW 1 Reply Last reply Reply Quote 0
        • Wayne WorkmanW
          Wayne Workman @szecca1
          last edited by

          @szecca1 Ask your boss if a new VM has been created on that physical box. Ask if any changes on other VMs on that box happened around the time that the FOG server started having issues.

          And maybe just bump it to 6 gigs and see what happens tomorrow morning.

          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
          Daily Clean Installation Results:
          https://fogtesting.fogproject.us/
          FOG Reporting:
          https://fog-external-reporting-results.fogproject.us/

          S 1 Reply Last reply Reply Quote 0
          • S
            szecca1 @Wayne Workman
            last edited by

            @Wayne-Workman Ok i extended it to 6 gb and will update you tomorrow on if the server continues to do the same thing. My boss is out today so tomorrow I will discuss with him about any changes made and about dynamic RAM.

            Wayne WorkmanW 1 Reply Last reply Reply Quote 0
            • Wayne WorkmanW
              Wayne Workman @szecca1
              last edited by

              @szecca1 Can you please setup this crontab task? I wrote a script to monitor resource usage over time. This can prove invaluable in finding out WHEN and WHY the server is crashing.

              dt="$(date)"
              echo ----------------------------- >> /root/monitor.log
              echo $dt >> /root/monitor.log
              echo ----------------------------- >> /root/monitor.log
              /usr/bin/free -m >> /root/monitor.log
              echo ----------------------------- >> /root/monitor.log
              /usr/bin/top -n 1 >> /root/monitor.log
              echo ----------------------------- >> /root/monitor.log
              echo .............................. >> /root/monitor.log
              echo .............................. >> /root/monitor.log
              echo .............................. >> /root/monitor.log
              

              I put that script here:
              /root/monitor.sh

              Made it executable:
              chmod +x /root/monitor.sh

              Then added a crontab entry to run it every 15 minutes:
              crontab -e

              0,15,30,45 * * * * /root/monitor.sh
              

              Your output should look something like this:
              cat /root/monitor.sh

              -----------------------------
              Mon Nov 30 12:42:53 CST 2015
              -----------------------------
                            total        used        free      shared  buff/cache   available
              Mem:           3934         409        2066           0        1458        3197
              Swap:          2047           0        2047
              -----------------------------
              top - 12:42:53 up 29 min,  1 user,  load average: 3.68, 4.22, 3.76
              Tasks: 180 total,   3 running, 177 sleeping,   0 stopped,   0 zombie
              %Cpu(s): 29.8 us, 37.6 sy,  0.0 ni, 31.5 id,  0.2 wa,  0.0 hi,  0.9 si,  0.0 st
              KiB Mem :  4028768 total,  2123932 free,   411532 used,  1493304 buff/cache
              KiB Swap:  2097148 total,  2097148 free,        0 used.  3281768 avail Mem
              
                PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
                559 root      20   0   49864  10204   4808 S  50.0  0.3  14:39.07 systemd-udevd
               2423 root      20   0       0      0      0 D  18.8  0.0   2:08.52 kworker/0:2
              55794 root      20   0   49864   8820   3420 R  18.8  0.2   0:00.03 systemd-udevd
              55793 root      20   0   49864   8820   3420 S  12.5  0.2   0:00.02 systemd-udevd
                 35 root      20   0       0      0      0 R   6.2  0.0   1:25.50 kdevtmpfs
              55783 root      20   0  160796   4376   3688 R   6.2  0.1   0:00.01 top
              55795 root      20   0   49864   7324   1924 S   6.2  0.2   0:00.01 systemd-udevd
              55797 root      20   0   49864   7324   1924 S   6.2  0.2   0:00.01 systemd-udevd
                  1 root      20   0   54716   8416   5696 S   0.0  0.2   0:02.41 systemd
                  2 root      20   0       0      0      0 S   0.0  0.0   0:00.00 kthreadd
                  3 root      20   0       0      0      0 S   0.0  0.0   0:01.21 ksoftirqd/0
                  5 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/0:0H
                  7 root      20   0       0      0      0 S   0.0  0.0   0:16.32 rcu_sched
                  8 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcu_bh
                  9 root      20   0       0      0      0 S   0.0  0.0   0:17.35 rcuos/0
                 10 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcuob/0
                 11 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 migration/0
                 12 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 watchdog/0
                 13 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 watchdog/1
                 14 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 migration/1
                 15 root      20   0       0      0      0 S   0.0  0.0   0:00.01 ksoftirqd/1
                 17 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/1:0H
                 18 root      20   0       0      0      0 S   0.0  0.0   0:05.64 rcuos/1
                 19 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcuob/1
                 20 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 watchdog/2
                 21 root      rt   0       0      0      0 S   0.0  0.0   0:00.01 migration/2
                 22 root      20   0       0      0      0 S   0.0  0.0   0:00.01 ksoftirqd/2
                 24 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/2:0H
                 25 root      20   0       0      0      0 S   0.0  0.0   0:05.83 rcuos/2
                 26 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcuob/2
                 27 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 watchdog/3
                 28 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 migration/3
                 29 root      20   0       0      0      0 S   0.0  0.0   0:00.01 ksoftirqd/3
                 31 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/3:0H
                 32 root      20   0       0      0      0 S   0.0  0.0   0:06.48 rcuos/3
                 33 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcuob/3
                 34 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 khelper
                 36 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 netns
                 37 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 perf
                 38 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 writeback
                 39 root      25   5       0      0      0 S   0.0  0.0   0:00.00 ksmd
                 40 root      39  19       0      0      0 S   0.0  0.0   0:00.00 khugepaged
                 41 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 crypto
                 42 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kintegrityd
                 43 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 bioset
              -----------------------------
              ..............................
              ..............................
              ..............................
              

              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
              Daily Clean Installation Results:
              https://fogtesting.fogproject.us/
              FOG Reporting:
              https://fog-external-reporting-results.fogproject.us/

              S 1 Reply Last reply Reply Quote 1
              • cmlC
                cml Moderator
                last edited by cml

                You can set memory allocation in VMWare, in the VM properties go to the resources tab, and click on memory.

                You can either reserve all memory, or a set amount.

                0_1448972464611_2015-12-01 06_05_14-Fog server keeps going down _ FOG Project.png

                S 1 Reply Last reply Reply Quote 0
                • S
                  szecca1 @Wayne Workman
                  last edited by

                  @Wayne-Workman I just got into work today and this morning the fog server was still up and running after the memory upgrade to 6 GBs. I can still do that if you’d like because I would hate to see your hard word go to waste. Let me know if you want me to do that

                  Wayne WorkmanW 2 Replies Last reply Reply Quote 0
                  • S
                    szecca1 @cml
                    last edited by

                    @cml Thank you I saw this right before I upgraded the memory but didn’t know what to do and didn’t want to mess with something I didn’t understand. The memory upgrade seems to have fixed the issue for now and I will keep an eye on it but do you think I should do this anyways?

                    1 Reply Last reply Reply Quote 0
                    • Wayne WorkmanW
                      Wayne Workman @szecca1
                      last edited by Wayne Workman

                      @cml Thank yo ufor the screen shot! I only have Hyper-V & KVM experience, glad someone could give a VMWare screenshot!

                      @szecca1 Yes please go ahead and setup the crontab event. Because the server is still online today - we know your issue is memory related, and the output from the below scheduled script will help you in the future… Keep in mind - the 6 gigs of RAM may not have fixed it, it may only have lengthened the amount of time between crashes…

                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                      Daily Clean Installation Results:
                      https://fogtesting.fogproject.us/
                      FOG Reporting:
                      https://fog-external-reporting-results.fogproject.us/

                      S 1 Reply Last reply Reply Quote 0
                      • S
                        szecca1 @Wayne Workman
                        last edited by

                        @Wayne-Workman 0_1448980546116_upload-b996b95f-b042-49a1-bcdd-ec4580b0d045

                        This is what I am getting when I typed that path in. Am I doing something wrong?

                        Wayne WorkmanW 1 Reply Last reply Reply Quote 0
                        • Wayne WorkmanW
                          Wayne Workman @szecca1
                          last edited by Wayne Workman

                          @szecca1 The path is /root/ the filename is monitor.sh

                          You create the file. I made it from scratch. You can create it with Vi. Like vi /root/monitor.sh

                          I suppose you can put the script anywhere you like.

                          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                          Daily Clean Installation Results:
                          https://fogtesting.fogproject.us/
                          FOG Reporting:
                          https://fog-external-reporting-results.fogproject.us/

                          S 2 Replies Last reply Reply Quote 0
                          • S
                            szecca1 @Wayne Workman
                            last edited by

                            @Wayne-Workman Hey after I am done typing the script how do I save it?

                            1 Reply Last reply Reply Quote 0
                            • S
                              szecca1 @Wayne Workman
                              last edited by

                              @Wayne-Workman I’m sorry I am having a difficult time doing this, so I’m just going to skip it unless you have more instructions on how to do this.

                              Wayne WorkmanW 1 Reply Last reply Reply Quote 0
                              • Wayne WorkmanW
                                Wayne Workman @szecca1
                                last edited by Wayne Workman

                                @szecca1 https://wiki.fogproject.org/wiki/index.php/Vi

                                Also - use putty and just copy / paste. No need to type all that out.

                                In Putty, you can copy by selecting the text and then right clicking anywhere on the window.
                                In Putty, you can paste by simply putting the cursor where you want your text to go, and then right click anywhere on the window.

                                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                                Daily Clean Installation Results:
                                https://fogtesting.fogproject.us/
                                FOG Reporting:
                                https://fog-external-reporting-results.fogproject.us/

                                S 1 Reply Last reply Reply Quote 0
                                • S
                                  szecca1 @Wayne Workman
                                  last edited by

                                  @Wayne-Workman Thank you but clearly I did something wrong. This is my output when I followed your instructions
                                  0_1448989852542_upload-f2f4a8dc-1889-4681-9097-91d7083a4258

                                  Wayne WorkmanW 1 Reply Last reply Reply Quote 0
                                  • Wayne WorkmanW
                                    Wayne Workman @szecca1
                                    last edited by

                                    @szecca1 The cat command just outputs the contents of a file for viewing.

                                    You run executibles in Linux with ./

                                    Like ./installfog.sh ?

                                    Try to run the script you just wrote. Like this:
                                    cd /root
                                    ./monitor.sh
                                    then output the contents of the log using the cat command.
                                    cat /root/monitor.log

                                    You can also look at the log using vi…
                                    vi /root/monitor.log

                                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                                    Daily Clean Installation Results:
                                    https://fogtesting.fogproject.us/
                                    FOG Reporting:
                                    https://fog-external-reporting-results.fogproject.us/

                                    S 1 Reply Last reply Reply Quote 0
                                    • S
                                      szecca1 @Wayne Workman
                                      last edited by

                                      @Wayne-Workman I may have found my problem but not sure how to fix it
                                      0_1448990477493_upload-9f582c84-7e96-43a6-9566-86d924f5ff64

                                      Wayne WorkmanW 1 Reply Last reply Reply Quote 0
                                      • Wayne WorkmanW
                                        Wayne Workman @szecca1
                                        last edited by

                                        @szecca1 That’s not a big deal. What gets ran is what is stored on the disk.

                                        Can you run the script? Is there output in the log?

                                        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                                        Daily Clean Installation Results:
                                        https://fogtesting.fogproject.us/
                                        FOG Reporting:
                                        https://fog-external-reporting-results.fogproject.us/

                                        S 1 Reply Last reply Reply Quote 0
                                        • S
                                          szecca1 @Wayne Workman
                                          last edited by

                                          @Wayne-Workman No I tried to see if the file was in the /root folder and its not so i wanted to make sure that wasn’t a problem. I’ll just recreate it again

                                          Wayne WorkmanW S 2 Replies Last reply Reply Quote 0
                                          • Wayne WorkmanW
                                            Wayne Workman @szecca1
                                            last edited by

                                            @szecca1 You’re using Fedora right?

                                            Did you create the crontab event as root? the crontab -e command will create a cron event and run as the user who created the event, meaning, not created using root, no permission to the /root directory.

                                            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                                            Daily Clean Installation Results:
                                            https://fogtesting.fogproject.us/
                                            FOG Reporting:
                                            https://fog-external-reporting-results.fogproject.us/

                                            1 Reply Last reply Reply Quote 0
                                            • 1
                                            • 2
                                            • 3
                                            • 4
                                            • 5
                                            • 6
                                            • 4 / 6
                                            • First post
                                              Last post

                                            184

                                            Online

                                            12.0k

                                            Users

                                            17.3k

                                            Topics

                                            155.2k

                                            Posts
                                            Copyright © 2012-2024 FOG Project