Fog server keeps going down

[Tom Elliott]

@szecca1 then the only thing I can think of is the console is blanked out, not necessarily unusable. Do you have a keyboard that you can press the space or a key to see if anything returns a display?

[szecca1]

@Wayne-Workman Do you want me to do this?

[szecca1]

@Tom-Elliott Yes we tried entering something and nothing is returned at all

[szecca1]

@szecca1 I just walked in today and the server is not reachable. I figured before I reboot it I would see if you guys want me to do anything. I can ping the server and it responds but trying to putty into the server it just hangs and never connects.

[Wayne Workman]

@szecca1 Can you get to it through the hypervisor?

[szecca1]

@Wayne-Workman I apologize I was trying to get a hold of my boss to log onto vSphere. When I go to the console, it asks for localhost login and then gives an error:

(38484.938838) Out of memory: kill process 2739 (httpd) score 104 or sacrifice child
(38484.942704) killed process 2739 (httpd) total -vm:1247792kb, anon-rss: 412668kb, file-rss:0KB

[szecca1]

@Wayne-Workman I am holding off on rebooting for a little bit just so you guys can see that error message

[Wayne Workman]

@szecca1 It displays that error AT the login screen, or after?

[Wayne Workman]

@szecca1 Check the amount of free RAM.

free -m

http://www.cyberciti.biz/faq/linux-check-memory-usage/

What does top say for memory usage in httpd?

[szecca1]

@Wayne-Workman as soon as the localhost login screen comes up, I click and that error message comes up.

[Wayne Workman]

@szecca1 said:

@Wayne-Workman I apologize I was trying to get a hold of my boss to log onto vSphere. When I go to the console, it asks for localhost login and then gives an error:

(38484.938838) Out of memory: kill process 2739 (httpd) score 104 or sacrifice child
(38484.942704) killed process 2739 (httpd) total -vm:1247792kb, anon-rss: 412668kb, file-rss:0KB

1247792KB + 412668KB = 1660460KB = 1660.46MB = 1.24GB

How much RAM does the server have?

[szecca1]

@Wayne-Workman I can’t type anything in as the screen wont allow me to even log into the machine until I reboot it

[szecca1]

@Wayne-Workman We set up the server with 4096 MB

[szecca1]

@Wayne-Workman But it says the memory overhead is 88MB but not sure what that means

[Wayne Workman]

@szecca1 So it’s simple… the server ran out of RAM - or - has dynamic RAM assigned and VMWare tried to smoosh it down a little to make room for other VMs that have a need… I think.

I was just reading this: https://plumbr.eu/blog/memory-leaks/out-of-memory-kill-process-or-sacrifice-child

This seems to say the same thing: http://ubuntuforums.org/showthread.php?t=2070853

[szecca1]

@Wayne-Workman So you would recommend adding more Ram? What do you think I should up it to?

[Wayne Workman]

@szecca1 I couldn’t see how it’s actually using all of 4 gigs… Make sure in VMWare that fog’s ram is not set to dynamic (auto expanding and contracting).

if it is set to dynamic, you could push the memory weight higher (it’s priority).

You could try bumping it to 6 gigs… but I really really doubt that the 4 gigs it has is being used… I really think dynamic RAM is to blame.

[szecca1]

@Wayne-Workman How do I see if it is set to dynamic. All I am seeing is the amount of ram nothing saying dynamic or not!

[Wayne Workman]

@szecca1 I have no idea, I don’t have any VMWare experience… but probably the other @Developers and @Moderators could chime in on that.

[szecca1]

@Wayne-Workman Better question, why would this randomly just start happening. The server was working fine until recently