FOG multicast issue: udp-sender not starting

[AndersenJ]

I am trying to use FOG for imaging, and it is working great as far as unicast goes. Right now I am having trouble with multicast.

My OS is Linux Mint. FOG version is 4573. On the clients, the multicast session will pause at

Parclone v0.2.80 http://partclone.org
Starting to restore image (-) to device (/dev/sda1)
_

and not go beyond that point. I tried stopping the unicast session with

sudo killall udp-sender

and got

udp-sender: no process found

so I did

top | grep udp

and got nothing. It seems like udp-sender isn’t starting at all; any ideas on how to fix this?

[Wayne Workman]

So - generally with multicast - the interface name that is to be used must be set in the storage node settings.

In most Linux distributions (not sure about Mint), you can issue this to get the interface names:

ip link show

On a regular one-NIC system, there will be normally “lo” which is the local loop back address and then there will be another.
Traditionally it’s eth0 but on newer Fedora and possibly other red-hat based systems it could be something totally different.

Just make sure you have the right name set in the storage node settings.

You might also try to restart the multicast manager. The service name is FOGMulticastManager.
In Ubuntu, you’d issue

service FOGMulticastManager restart

and in Fedora or other Red-Hat systems you’d issue

systemctl restart FOGMulticastManager

Let us know what happens, we are here to help.

[AndersenJ]

I have verified the interface name and I’ve tried restarting the multicast manager service. Still not working. I even tried installing Fedora 21 instead of Mint. Exactly the same problem.

[Wayne Workman]

@AndersenJ Can you test multicasting using an unmanaged switch? This will narrow down where the problem is.

Put both FOG server and the two target hosts on a single unmanaged switch. The switch can be plugged into your network - that’s fine, but the multicast traffic for this test should be going through the unmanaged switch - if you understand what I mean.

[nengelhardt]

@Wayne-Workman I have been working with him a little and looking at his logs and I think I see something. Where does fog pull the interface to use for udp-sender from? Because for me it keeps using the wrong interface name when starting udp-sender and I suspect it’s doing the same for him.

[Wayne Workman]

@nengelhardt said:

@Wayne-Workman I have been working with him a little and looking at his logs and I think I see something. Where does fog pull the interface to use for udp-sender from? Because for me it keeps using the wrong interface name when starting udp-sender and I suspect it’s doing the same for him.

Well, I’m not positive on what interface - but it can only be a handful of possible settings.

Obviously the storage node interface name needs to be correct for said storage node.

Also in here there is an interface name specific to multicast… I ASSUME that this setting is for the master storage node??? not sure… Perhaps the @Developers can elaborate… but mess with this setting:

FOG Configuration -> FOG Settings -> Multicast Settings -> FOG_UDPCAST_INTERFACE

And it’s probably irrelivant but here’s some other spots where the interface name occurs:

FOG Configuration -> FOG Settings -> NFS Server -> FOG_NFS_ETH_MONITOR
and in this file
/opt/fog/.fogsettings the interface= field. However this setting is not used during production, only during installation I believe. The only field from this file actively used during production is the MySQL settings… because the fog server has to know what DB to connect to and what password to use, all other production settings are pulled from the DB I believe.

After changing any of that stuff - you always have to restart the FOGMulticastManager, or the changes won’t matter.

[AndersenJ]

@Wayne-Workman I have checked the interface names in those settings. They seem normal. I have also tried using a dumb switch. The FOG Server is also it’s own DHCP server, so for testing I connect it to a dumb switch and connect the client to the same switch.

[AndersenJ]

@nengelhardt When it sends you the wrong interface name, does it at least run the process udp-sender? When I run

top | grep udp-sender

I get nothing.

[Wayne Workman]

@AndersenJ out of nothing but curiosity, can you try Fedora?

This should take under an hour to follow through once you have the ISO downloaded. You can use Fedora 22, just replace “yum” with “dnf”. https://wiki.fogproject.org/wiki/index.php/Fedora_21_Server

[AndersenJ]

@Wayne-Workman I tried Fedora 21 a few days ago and got precisely the same result.

[Wayne Workman]

@AndersenJ said:

@Wayne-Workman I tried Fedora 21 a few days ago and got precisely the same result.

Then something isn’t right - plain and simple. You can’t try two completely different distributions on the same machine and the same unmanaged switch and something not be screwy…

Better question is… has multicast via FOG ever worked in your environment?
Are you setting any custom multicast settings?

[Tom Elliott]

I don’t know how to put this any more simply. Hopefully you all will understand.

Dev/1.3 FOG will attempt to use the interface specified on the Master Storage Nodes of the relative systems. So Please start by checking the interface name on the Storage Nodes that are the “masters” for the groups they are in. This is done through the storage management page on the GUI.

If for SOME reason or other the storage node’s interface field is unable to be received the fallback is the FOG_UDPCAST_INTERFACE. This is again just a fallback, that should basically never even be able to happen. The FOG_NFS_ETH_MONITOR file was only used for bandwidth tracking but since moving the nodes to contain their own relevant interface setting, I’ve basically eradicated any need for this setting.

[nengelhardt]

Okay. So here’s what’s happening for me and what I suspect is happening for @AndersonJ as well. @tom-elliott was right, the setting was wrong under “storage management”>“DefaultMember”>“interface”. So the udp-sender would start, then end itself when it realized it was started with an invalid interface. Thanks for the help, everyone.

[AndersenJ]

@Tom-Elliott It’s a good guess that this would be the problem, but I just checked and my interface is set correctly to ‘eth0’ in the location you described, and in all others I could find. As far as I can tell, udp-sender is never starting in the first place.

It is worth noting that I have had multicasting work in the past; last time it worked was August 12th. I’m pretty sure I didn’t change any settings at that time, nor did an update to FOG or any other software instigate it. The server isn’t even connected to the internet.

Also relevant: I just re-checked my logs; the server hasn’t updated any of the files in /opt/fog/log/ since that day. I thought that failed attempts at least would be in there.

[Tom Elliott]

@AndersenJ What is the output of ip addr or ifconfig where the Ethernet name is listed for the relevant IP Address of your fog server?

You saying your server’s interface is eth0 doesn’t simply make it so. I’m not aware of Fedora 21 setting an interface to eth0, I don’t recall any eth setting being specified from Fedora 14 and up. Again I could be wrong and this is where the commands I requested would be of most use.

[AndersenJ]

@Tom-Elliott I’m actually back to using Mint 17.2 now, since I had the same multicasting problem with Fedora as I had using Mint. ifconfig returns eth0 and lo. Not sure what it was when I was using Fedora, but I assure you I did check to make sure the interface I was using actually existed.

[AndersenJ]

@Tom-Elliott I figured out the issue: it was my ethernet adapter. I plugged in a USB 2.0 ethernet adapter, changed my interface settings to use it, and multicasting worked perfectly (at USB 2.0 speeds). It was the Dell Latitude E6530 ethernet adapter which wasn’t working.

[Wayne Workman]

@AndersenJ said:

@Tom-Elliott I figured out the issue: it was my ethernet adapter. I plugged in a USB 2.0 ethernet adapter, changed my interface settings to use it, and multicasting worked perfectly (at USB 2.0 speeds). It was the Dell Latitude E6530 ethernet adapter which wasn’t working.

ha. I knew something was screwy.

I’m very happy you figured it out.

[inst]

i have the same problem ! is there a solution here ?

[Sebastian Roth]

@inst Only very seldomly issues are exactly the same in such complex environments. Espacially as this post is about a year old I kond of doubt that you have exactly the same version of FOG running in the same kind of environment. So please do us a favour and open a complete new topic and explain your problem in detail to us (step by step on how you get to the error, plus pictures and screenshots will help a lot).