Posts made by LLamaPie

@LLamaPie Everything has been clean now for about a week. I would consider this at least resolved on our end. Still no answer about when it became compromised exactly. Our hyper-paranoid theory is it may have been a “time bomb”. This could have been on the server for months before popping up. Our long-term solution is keeping endpoint protection in place. I have nothing else to add but if I discover anything I will let everyone know.

@george1421 Yep, can do. I’ll keep you guys posted. So far it’s been fine since Sophos cleaned it.

@Tom-Elliott Yea the “coinminer config” that Sophos nuked + the 400% CPU usage makes me think it was being used to do some sort of mining.

@george1421 Nope, that is what is baffling us as well. The server is local only and locked down. No one outside the network should be able to access it.

It’s hard to say when it was compromised but we did notice the sudden spike in resource usage 1-2 weeks ago. The server is largely left alone as it does what it needs to do. Beyond running updates on occasion, no changes are made. I will keep an eye on things. I just know after Sophos cleaned up the issue yesterday it has been fine. It’s too soon to say for sure.

What I can possibly do is scan some of our long-term back ups to figure out how long it’s been infected. We will want to discard them anyway. I’ll see what I can do.

@Tom-Elliott Well Sophos found this:

@Tom-Elliott Yep, that is what I was worried about. Worst case I need to nuke the server and rebuild.

Running Fog Version: 1.5.10.15
Linux: Debian 12

Over the last week, we have noticed a massive spike in the CPU usage on our FOG Server VM. See the screenshot. I am unable to find what the process is or why it is using so much CPU.

www-data user appears to be part of the web server but .systmd doesn’t appear to relate to anything (at least that I can find). I will kill the process and it will just come back up shortly after. Killing it does not appear to affect fog either.

Does anyone have any clue what this is?

Obvious question:

Did you reboot your fog server yet and/or update it to the latest version and kernels?

@Sebastian-Roth Works! Thank you for the fix!

@Sebastian-Roth I attempted to update the dev-branch version and installed 1.5.10.4. Fog is saying the latest version is 1.5.10.5. I deleted the .git file and reran the command to download the fogproject.git once again. When I go to install the dev-branch it only installs version 1.5.10.4. Am I missing a step or is there a better place to locate the current version?

Answered my own question. I followed the instructions on the github page. I have 1.5.10.5 now installed. I will test this and let you know if that did the trick.

@flat4vw This is an old post but is probably still relevant:

https://forums.fogproject.org/topic/7322/install-update-your-database-schema-url-just-goes-to-normal-fog-ui/3

TL;DR: you are probably fine.

@rodluz I don’t think this is related. It appears to be an issue revolved around multiple network interfaces in the device. When I disabled WIFI in Bios everything works as expected.

@Sebastian-Roth I have confirmed disabling wifi in BIOS resolved this issue. So it does in fact have to do with multiple interfaces. Disabling wifi allowed me to enter the deploy image screen as normal. Of course, this is not a long term or preferable solution as you would need to go back into bios and turn wifi back on every time you image the device in this manner. At least we were able to confirm the problem.

@admiralshaw My work around which isn’t that awful is to note the last 4 digits of the Mac for the host. Then search that host in the fog server interface and manually assign the image and then start the deploy task there. The next PXE boot should go directly into the deployment. This is assuming your issue is the same as mine.

@LLamaPie There is multiple network interfaces due to the wifi card in the device. To test this theory, I will disable the wifi card in bios or manually remove it temporarily and see if that solves the issue and confirms the suspicion. I probably won’t be able to test this until next week. I’ll let you know what I find.

Fog Version: 1.5.10
Kernel: 6.1.22 ; 6.1.22

Having a weird issue I cannot seem to get by. We just got a new fleet of HP Probook G9s in. They all quick register without issue. However, when going to “Deploy Image” it asked for the Username and Password, as usual. When we type it in, we have to enter 3 times. After which, it does not load into the deploy image screen - instead it loads into the Compatibility screen.

Our only work around at the moment is to search the host in the Fog server and manually start the deploy task. That does work. I was able to test that Fog is working normally on our older (G7) models without issue.

We are getting a warning when loading into fog, however.

WARNING Using legacy NIC wrapper on 00:00:00:00:00:00

That is the only thing showing up that is remotely different from other machines.

Quick update. I did not solve the issue with ubuntu but I have it working on CentOS 7. Should have just done that sooner, much smoother experience.

I would still like to know why I was having issues on the Ubuntu install, however.

Tried a few more things and getting no where. Installed PHP7.1 and confirmed it’s running. Installed php-gettext as well. Attempted to install fog with ‘-x’ and managed to skip by the original issue and get here where it failed again:

So checking that directory, it’s showing this:

Even though I confirmed php7.1 is running it’s not in the directory that fog is trying to find?

EDIT:

Checked again to see the status of php7.1 and I got this:

It was running prior to attempting to install fog and now it’s gone, even though I told fog ‘n’ to deleting those packages.

Here is the original output after I installed php7.1:

Try updating the kernel in the server config?

@lobomarinho In Fog config, update to the latest Kernel.

I had a similar issue with some Dell Precision desktops. Updating to the latest Kernel fixed it.