Weird Traffic from FOG Server

[cwgcad]

Since upgrading to 1.5.9 our FOG Server has been sending traffic that is being picked up by our firewall as “CoinMiner.C_4 (trojan)” it says that the destination is 10.2.254.102, 54614. I was wondering if anyone has had this happen to them.

[Sebastian Roth]

@cwgcad As George said this looks like a response packet. With source port 80 this must be the case I would think. So please figure out what the other IP is!

Matching the alerts with httpd log in the FOG server is a great idea.

[george1421]

@cwgcad So who are the actors here?

I assume that 10.2.25.80 is your fog server. What is device 10.2.254.100 (I’m thinking a client computer with the fog client service running)?

Make sure you have NTP setup on your fog server so the time is accurate according to your network. I’m almost thinking that the FOG server is responding to a query at that time and its response appears virus like.

The idea is to coordinate the time of the firewall log capture with the apache web server access log at the time of the incident. /var/log/httpd for RHEL based systems and /var/log/apache2 for debian based. There should be an access and error log in that directory we need to coordinate the access log to see if we have a query in from that external host just before the firewall detected the virus like response.

I’m thinking this is a false positive but lets not go there yet. We also can use tcpdump on the fog server to get a packet capture of this traffic for analysis. tcpdump would be outside of anything and should capture the packets that is triggering your firewall. If it is a false positive you can submit the packet capture to your firewall company for them to review.

[cwgcad]

@george1421 Thanks! I will try this the next time it happens

So I was able to catch one and this is what I got back. It’s on port 80 which would be httpd

sudo netstat -nap | grep :49412
tcp6       0      0 10.2.25.80:80           10.2.254.100:49412      TIME_WAIT   - 

which gives me no more info than the firewall alert does

Also it seems to use a different port all the time

sudo netstat -nap | grep 10.2.254.100
tcp6       0      0 10.2.25.80:80           10.2.254.100:49501      TIME_WAIT   -                   
tcp6       0      0 10.2.25.80:80           10.2.254.100:49500      TIME_WAIT   -                   
tcp6       0      0 10.2.25.80:80           10.2.254.100:49498      TIME_WAIT   -                   
tcp6       0      0 10.2.25.80:80           10.2.254.100:49497      TIME_WAIT   -                   
tcp6       0      0 10.2.25.80:80           10.2.254.100:49504      TIME_WAIT   -                   
tcp6       0      0 10.2.25.80:80           10.2.254.100:49499      TIME_WAIT   - 

[cwgcad]

@sebastian-roth I do get the source IP which is the FOG Server. I had started the server up again a bit ago and it started sending more requests. I stopped the httpd service and they stopped. I just started the service again for sanity check and so far it hasn’t sent any but I have seen the server go hours without anything and then send them continuously for 1 - 2 hours. So I will wait for emails and report back, I should see something within the next 24 hours if I’m going to at all.

[george1421]

@cwgcad said in Weird Traffic from FOG Server:

10.2.254.102

So the source address is your FOG server and the destination address is the IP address above? If so is that IP address valid for your network?

In your firewall log does it list the FOG server and its source port? Can you catch it while the traffic is happening? If you can and you know the source port run this command on linux sudo netstat -nap | grep :199 Replace 199 with the suspect sending port. Lets see if we can find the process that is sending the traffic.

[Sebastian Roth]

@cwgcad Does your firewall also tell you which source the packets come from?

If you shutdown the FOG server and keep seeing the blocks it’s very likely not the FOG server sending the requests.

[cwgcad]

@tom-elliott no I didn’t run anything at all yesterday, From 5PM - 7:30PM when I shut the server down I kept getting emails from the firewall every minute detecting and blocking the same traffic. I didn’t have this issue with 1.5.8 but as soon as I had upgraded to 1.5.9 it started happening. I don’t want to accuse FOG of anything and it is possible that it is a false positive.

I have been searching online as to how I can identify which process is sending this request I haven’t found anything. I have found lots of stuff but nothing that ties a process to the request. With the knowledge you have, is there anyway I could find this out?

[Tom Elliott]

@cwgcad we used to have a mining system in fos but it needed to be enabled and configured. We have since stopped doing this. For quite some time really. I highly doubt this is something fog did but maybe a firewall rule was put in place for a specific port number? Did you run a multicast task by chance?