Multicast very slow

[plegrand]

Hello,
i’m trying multicast with fog 1.5.5 and alcatel switchs (OS6250 and OS6450).
When i launch a multicast session it’s extremely slow.
For the configuration of the switchs i made like this :

On all switchs enable igmp globaly and on vlan 2 (fog server)

ip multicast status enable
ip multicast vlan 2 status enable
ip multicast zapping enable
ip multicast vlan 2 zapping enable
ip multicast version 3
ip multicast vlan 2 version 3
ip multicast querier-forwarding enable
ip multicast vlan 2 querier-forwarding enable

On the querier switch (the closest to the fog server)

ip multicast querying enable
ip multicast vlan 2 querying enable
ip interface Querier address 192.168.39.244 mask 255.255.255.0 vlan 2

Disable igmp on all other vlans on all the switchs

All computers are on 100M port and the fog server on 1G port

You can see on this diagram http://plegrand1.free.fr/IGMP.png (old diagram )
not a ghost server but a fog server
I add global statement
The server is on a 1G port

the flow is very fast at the beginning then goes down to be very slow (25M / min)
Here is the server characteristics

PowerEdge T310
Intel(R) Xeon(R) CPU X3480 @ 3.07GHz
cores	      =	4
enabledcores  =	4
threads       =	8
Ram           = 32GiB

Ethernet interface
NetXtreme II BCM5716 Gigabit Ethernet
eth0
1Gbit/s

Could you help me to solve this problem ?

[Sebastian Roth]

@plegrand Hey, welcome back to the forums! Reading your post I just remembered that we have been discussing this in the past already: https://forums.fogproject.org/topic/6655/multicast-very-slow

the flow is very fast at the beginning then goes down to be very slow

To me this sounds as if some of the clients are the bottleneck here. Maybe I am wrong but I think it’d be worth a try. Arrange your clients into smaller sub-groups of 3 machines in each group. Do multicast tests on each of those groups and see if the behaviour is exactly the same as before or if some sub-groups go faster than the others. Please note down the speeds and executing time of those multicast sessions and post results here.

[plegrand]

@Sebastian-Roth
Hello Sebastian and thanks for your answer.
I made these tests (sorry for the french words) :

Troubleshoot MultiCast
https://wiki.fogproject.org/wiki/index.php/Multicasting

Afficher le mode debug sur les clients
Menu Fog Configuration
iPXE Menu Item Settings
fog.debug
L’option “Menu Show with” est par défaut sur “Debug Options”. La mettre sur “All hosts”

Test avec 4 clients
Sur le serveur, lancer la commande suivante :
udp-sender --file /opt/fog/.fogsettings --log /opt/fog/log/multicast.log --ttl 32 --nopointopoint

Sur les clients
Booter en mode debug
Commande : udp-receiver
Sur le serveur, on doit voir les 4 clients connectés
“Enter” pour lancer le transfert

Sur le serveur
“Enter” pour lancer le transfert

Tests avec les postes:
192.168.39.181
192.168.39.145
192.168.39.121
192.168.39.189
Multicast litening : 232.168.39.243

Logs sur le serveur :

New connection from 192.168.39.181  (#0) 00000009
Ready. Press any key to start sending data.
New connection from 192.168.39.145  (#1) 00000009
Ready. Press any key to start sending data.
New connection from 192.168.39.189  (#2) 00000009
Ready. Press any key to start sending data.
New connection from 192.168.39.121  (#3) 00000009
Ready. Press any key to start sending data.

094  re-xmits=0000002 (100.0%) slice=0112 -   0

09:24:59.309910 Using mcast address 232.168.39.243
09:24:59.309962 UDP sender for /opt/fog/.fogsettings at 192.168.39.243 on eth0 
09:24:59.309973 Broadcasting control to 192.168.39.255
09:25:03.709692 Using mcast address 232.168.39.243
09:25:03.709744 UDP sender for /opt/fog/.fogsettings at 192.168.39.243 on eth0 
09:25:03.709756 Broadcasting control to 192.168.39.255
09:30:03.457992 Using mcast address 232.168.39.243
09:30:03.458037 UDP sender for /opt/fog/.fogsettings at 192.168.39.243 on eth0 
09:30:03.458047 Broadcasting control to 224.0.0.1
10:13:28.830979 Starting transfer: 00000009
10:13:28.842773 RETX         1    1
10:13:28.843867 RETX         2    1
10:13:28.844919 Transfer complete.
10:13:28.845001 Disconnecting #0 (192.168.39.181)
10:13:28.845050 Disconnecting #1 (192.168.39.145)
10:13:28.845063 Disconnecting #2 (192.168.39.189)
10:13:28.845075 Disconnecting #3 (192.168.39.121)
10:13:28.845090

For me it works fine. What do you think about that?

[plegrand]

For information i’ve got these kinds of logs :
http://plegrand1.free.fr/multicast.log.udpcast.6

unicast works fine

[george1421]

Well the first thing I see here is that your clients are on 100MB links. So that will limit your transfer rates to 12MB/s maximum (all traffic).

Once the network settles down you are getting 25MB/s transfer rate is very good. Also the slowest computer in the multicast will be the maximum speed of your multicast.

So I would ask you in your network design. What is the average unicast transfer rate when you image one computer (direct download not multicast)? Then send a multicast image to that same 1 computer. Is there a large difference?

[plegrand]

@george1421
unicast : 30minutes for 14 Computers
multicast : 24hours for the same computers

[george1421]

@plegrand said in Multicast very slow:

@george1421
unicast : 30minutes for 14 Computers
multicast : 24hours for the same computers

Sorry I was not clear in my request. I only want information on 1 computer (the same computer) One test via unicast and the second test via multicast. I want to compare the speeds between unicast and multicast on the same computer.

To me you have too many variables in your environment. We need to understand the basics first.

[plegrand]

@george1421 How can i make multicast on 1 computer? I can’t see this option

[george1421]

@plegrand

Then pxe boot the client and select the session name.

[plegrand]

@george1421 Sorry but where is this “Start Multicast session” ?

[plegrand]

@george1421
Image menu
Multicast Image
Start Multicast Session

[plegrand]

@george1421

With only one computer it seams to works fine.

Unicast
file:///home/pascal/Bureau/unicast.jpg

Multicast
file:///home/pascal/Bureau/multicast.jpg

[george1421]

@plegrand OK very good. This tells us that your network between the fog server and the target computer is good. You are getting ~28MB/s transfer rate. That’s a bit of a lie because your 100MB/s network can only transfer 12MB/s. The 20MB/s is the rate at which partclone can expand the image onto the disk of the target computer. But if you can’t feed partclone fast enough that rate will drop off quickly. If you are getting an expansion rate of the image faster than the network speeds your network rate is good.

Now can you make the same test with a computer located on the switch far away from the FOG server. In your drawing it would be the switch I want to test on the left of the drawing. This test will be the full network path between the fog server and the target PC.

One question I didn’t ask, how many target computers are in your multicast session?

[plegrand]

@george1421
To answer to your last question it was a session with 15 target computer and theses target computer was very near the fog server, on my drawing it’s the swith just below the fog server

[george1421]

@plegrand Well that makes me think a little differently if they are that near to your fog server.

Ok lets do a new test for multicast. Lets test 2 computers and then 3 computers. Now these computers must be the same model as the one done in your first test. We need to remove the variable of different models in this test.

What I expect to see for 2 computers is about the same speed as 1 computer in the multicast. For 3 computers slightly less than 2 computers. In my mind I question at what point do we go from an acceptable level of speed to bad.

I can tell you with unicasting on a pure 1GbE network you can fill up a 1GbE link with 3 simultaneous unicast deployments of FOG with modern target computers. That is a concern for the link to the fog server and switch to switch links mainly. Under a multicast you should get just slightly less than unicast speeds.

[george1421]

@plegrand Also I just looked into these switching thinking they were old. They are not (at least performance wise) they are better capacity than Cisco small business switch SG300.

So I have to ask you why are you running 100MB/s to the desktops?

Do you have the capabilities to run a second network wire from the switch where the fog server is to the switch where the target computers are? (I’ll explain more if you can).

[plegrand]

@george1421
With a target computer far from the fog server it seems to works also

file:///home/pascal/Bureau/multicast-205.jpg

[plegrand]

@george1421
I’m not sure to well understand
Actually my diagram is not “really” true.
The fog server is on a GB port
All desktop clients are on 100 M ports
I cant do anything else, no enough GB ports

[plegrand]

@george1421 For these tests for the moment i’ve no time enough but i’ll do these tests in a near futur

[plegrand]

@george1421 said in Multicast very slow:

Do you have the capabilities to run a second network wire from the switch where the fog server is to the switch where the target computers are? (I’ll explain more if you can).

It’s quiet already the case
The switch (OS6250) where are connected all my target computers is just below the switch where the fog server is (OS6450).
And directly connected by a 1GB link
Then, only target computers are on a 100MB link