Multicast off of bond

David Osinski

We’ve been using fog for years and am just now trying to get multicast to work correctly. I use 4 ethernet nics bonded together in CentOS 7. I start the multicast session and look at the logs:

"/usr/local/sbin/udp-sender --interface rope --min-receivers 1 --max-wait 600 --mcast-data-address 239.255.20.105 --mcast-rdv-address 10.10.10.74 --portbase 54232 --full-duplex --ttl 32 --nokbd --nopointopoint --file /images/W10EDULenovoFlex3x64/d1p1.img;/usr/local/sbin/udp-sender --interface rope --min-receivers 1 --max-wait 10 --mcast-data-address 239.255.20.105 --mcast-rdv-address 10.10.10.74 --portbase 54232 --full-duplex --ttl 32 --nokbd --nopointopoint --file /images/W10EDULenovoFlex3x64/d1p2.img;/usr/local/sbin/udp-sender --interface rope --min-receivers 1 --max-wait 10 --mcast-data-address 239.255.20.105 --mcast-rdv-address 10.10.10.74 --portbase 54232 --full-duplex --ttl 32 --nokbd --nopointopoint --file /images/W10EDULenovoFlex3x64/d1p3.img;/usr/local/sbin/udp-sender --interface rope --min-receivers 1 --max-wait 10 --mcast-data-address 239.255.20.105 --mcast-rdv-address 10.10.10.74 --portbase 54232 --full-duplex --ttl 32 --nokbd --nopointopoint --file /images/W10EDULenovoFlex3x64/d1p4.img;"

As you can see the command is listed 4 times. If i change the interface in either multicast settings or nfs server, it still uses the interface “rope”

I dont know if this is my problem but my clients stay at the image is starting within part clone. I looked in your troubleshooting guide and ran the MySQL commands, no effect.
I have ran a RTP stream from my vlan to the server vlan, Multicast streaming works. We use HP Comware devices.

My next choise if you dont have anything is to either take an interface out of the bond and set it up as the only interface to test it, or build another fog server, only using 1 nic, for testing purposes.

george1421

@David-Osinski said in Multicast off of bond:

rope

What is the output from ip addr show because rope is not a typical name of a bonded group.

David Osinski

@george1421 said in Multicast off of bond:

ip addr show

[root@tisd-fog01 ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: em1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master rope state UP qlen 1000
    link/ether d4:be:d9:b1:19:cb brd ff:ff:ff:ff:ff:ff
3: em2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master rope state UP qlen 1000
    link/ether d4:be:d9:b1:19:cb brd ff:ff:ff:ff:ff:ff
4: em3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master rope state UP qlen 1000
    link/ether d4:be:d9:b1:19:cb brd ff:ff:ff:ff:ff:ff
5: em4: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master rope state UP qlen 1000
    link/ether d4:be:d9:b1:19:cb brd ff:ff:ff:ff:ff:ff
6: rope: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether d4:be:d9:b1:19:cb brd ff:ff:ff:ff:ff:ff
    inet 10.10.10.74/24 brd 10.10.10.255 scope global rope
       valid_lft forever preferred_lft forever
    inet6 fe80::32e0:22c8:7b56:41c9/64 scope link
       valid_lft forever preferred_lft forever

(i dont abide by your linux bond0 rules…)

also heres this

DEVICE=rope
BONDING_OPTS="miimon=100 updelay=100 downdelay=2000 mode=balance-rr"
TYPE=Bond
BONDING_MASTER=yes
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=no
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=rope
UUID=5b88e67a-093b-491b-9540-c172fb9f726d
ONBOOT=yes
IPADDR=10.10.10.74
PREFIX=24
GATEWAY=10.10.10.254
DNS1=10.10.10.11
DNS2=10.10.10.12
DOMAIN=terrellisd.lan

“/etc/sysconfig/network-scripts/ifcfg-bond-rope”

[MOD Note] Fixed formatting for readability. -Geo

george1421

@David-Osinski said in Multicast off of bond:

As you can see the command is listed 4 times. If i change the interface in either multicast settings or nfs server, it still uses the interface “rope”

OK interface rope is the correct one to use. From what I’m seeing you maybe have set a receive address outside of the multicast range?

Are you trying to multicasts between subnets or on the same subnet (vlan). The out of box settings should just work as long as its on the same vlan and your infrastructure is configured for multicasting.

Bonding works at OSI layer 2, where muticasting works at OSI layer 3, so it should ride the bonding without issue.

Sebastian Roth

@David-Osinski The best testing you can do is usually manually running udp-sender on the FOG server (using the command you posted above) and udp-receiver on the client machine. For the later you need to schedule a debug deploy task and when you get to the shell on bootup run:

udp-receiver --nokbd --portbase 54232  --ttl 32 --mcast-rdv-address 10.10.10.74 >/dev/null

In this command we write to device /dev/null instead of the read disk just to not cause any harm.

David Osinski

@Sebastian-Roth
Here is what i ran on the server

udp-sender --interface rope --min-receivers 1 --max-wait 600 --mcast-rdv-address 10.10.10.74 --portbase 56854 --full-duplex --ttl 32 --nokbd --nopointopoint --file /images/W10EDULenovoFlex3x64/d1p1.img

Here is what i ran on the client

udp-reciever --nokbd --portbase 56854 --ttl 32 --mcast-rdv-address 10.10.10.74 > /dev/null

I am getting the same results, nothing in coming through.
Im going to be a little more specific to help out.
Im running on a physical Dell emc server with 4 Nics bonded together. I can see all multicast traffic from other servers on the same vlan just fine, This server just seems to not be broadcasting its multicast session to the switch
I run

 ip maddr

but am not getting any other multicast on the server except 224.0.0.1
I boot the client and join the multicast session i can see that the client is trying to join a multicast session on the switch.

I have other windows servers nic teamed together on the same vlan without a problem on multicast. It just seems to be Cento7 w/fog.

The receiving client is on another vlan than the server.

I would like to also point to attention that i dont know if its a bond issue or a issue with this particular server/settings

(Display igmp-snooping Switch Client is on)

VLAN 2: Total 11 entries.
  (0.0.0.0, 234.10.10.74)
    Host ports (1 in total):
      GE2/0/7

(Display igmp group Switch Server is on)

Vlan-interface1010(10.10.10.254):
  IGMP groups reported in total: 2
   Group address   Last reporter   Uptime      Expires
   239.255.255.250 10.10.10.90     9w:1d       00:04:19
   239.255.255.254 10.10.10.20     1w:1d       00:04:19

Server ip address is 10.10.10.74, its not joining the multicast group.

Sebastian Roth

@David-Osinski said in Multicast off of bond:

I have other windows servers nic teamed together on the same vlan without a problem on multicast.

What multicast “application” do you run on those servers?

David Osinski

@Sebastian-Roth
i used VLC to run a RTP stream off the servers to test.

Sebastian Roth

@David-Osinski Debugging multicast issues in a multi VLAN setup is not easy for me not being at site. My suggestion is to rule out things. Start by disabling the bond on your FOG server, configuring the IP address to a single NIC, even unplugging the other cables and doing the same manual udpcast test as before.

george1421

@David-Osinski said in Multicast off of bond:

–mcast-rdv-address 10.10.10.74

This is what bugs me a bit. The rendezvous address should be a multicast address. That is where all of the multicast clients go to find each other. I would expect that to be at least 224.0.0.1

udp-sender --interface rope --min-receivers 1 --max-wait 600 --mcast-rdv-address 10.10.10.74 --portbase 56854 --full-duplex --ttl 32 --nokbd --nopointopoint

When you don’t define an multicast address, the address is created by udpsender as a composite of the FOG server’s IP address. I think it should be in the range of 239.10.10.74, possibly the second octet is something different. But the data channel IS calculated so its important for the rendezvous address to be defined, so the targets can locate the data channel.

David Osinski

@george1421
I chose this IP address due to the documentation on https://wiki.fogproject.org/wiki/index.php/Multicasting

The direct line where i took this information was
If your image is partclone

udp-receiver --nokbd --portbase 9000 --ttl 32 --mcast-rdv-address $fogserverip | \
partclone.restore --ignore_crc -O /dev/sda<filenumber> -N -f 1

Being that said, i will definately try 239.10.10.74

Tom Elliott

@David-Osinski Normally, the rendevous address should only be set to the FOG Server IP if you have multiple VLAN’s. Otherwise, this setting should be safe to keep unset.

@george1421 is absolutely correct in that you can set this up to a Multicast Address for rendevous, but this basically makes it so all traffic connects to the FOG server over UDP. When using the FOG Server’s actual IP address, the establishment of connection to the rendevous, from what I can remember, actually connects over TCP/IP allowing the clients to at least see the fog server and make the connection directly. After all clients check in, then Multicast starts sending the stream.

In either case, multicast or actual IP of fog server, the vlans still need to be allowed to stream multicast traffic.

george1421

@David-Osinski To tag along with what Tom said, I would not define either addresses unless you have a specific use case. Let udp-send do that.

Just to prove the multicasting is working with your hardware, please do what both Sebastian and I recommended. Start out small 2 clients connected to the same switch and same vlan as the fog server. Then test your multicast deployment. By doing this you will know if multicasting across the bond layer works and that your core switch is setup correctly. If it doesn’t work here then we’ll need to dig into that. Start small with a very controlled environment.

If your end goal is multicasting across subnets then you need need a multicastrouter (mrouter) or an igmp proxy service (akin to a dhcp-relay service) setup on your core router. But again, start with a small and controlled test.

David Osinski

@george1421

George/Tom…you are genius’s

I am overcomplicating things. To not define address’s was the answer. Multicast works on my network as i said earlier because i can VLC RTP stream just fine. I have routers setup to do PIM SM. Specifying a multicast rdv was my downfall.

All of you are amazing. Please keep up the great work!

Multicast off of bond

190

11.6k

17.1k

154.6k