MultiCast very slow



  • Hello,
    i’ve got some problem with multicast.
    I’m trying to deploy an image on 12 windows 7 stations. The stations are identicals. I didnt make any configuration in fog. Just configure manageable swith like that :
    http://plegrand1.free.fr/pasted_image001.png

    I can see in the log a lot of lines like that :
    Timeout notAnswered=[0,1,2,3,4,5,6,7,8,9,10,11] notReady=[0,1,2,3,4,5,6,7,8,9,10,11] nrAns=0 nrRead=0 nrPart=12 avg=450

    And in the “task” bar 170MB/min

    May be it could be a client which has problem. Is there a way to detect if there is a problematic client ?

    Does someone could help me to solve this problem ?

    Thank you for help



  • This post is deleted!


  • @Sebastian-Roth Hello
    for the moment i didnt make any test about multicast.
    But it’s planed
    I will tell you
    I have to look on switch configuration


  • Developer

    @plegrand Any news on this? Do you still see slow speeds when using multicast?


  • Testers

    @Wayne-Workman I wonder if I got compression part from that post a year ago. Some of that is what I remember from my college Networking class.

    @plegrand Looks like the problem shouldn’t be the server power, and I was wrong about that part anyway.
    Considering the speed difference you already are seeing, why not just use unicast? I’m just curious if there’s a specific requirement for multicast?



  • @Arrowhead-IT
    lscpu
    Architecture : x86_64
    Mode(s) opératoire(s) des processeurs : 32-bit, 64-bit
    Boutisme : Little Endian
    Processeur(s) : 8
    Liste de processeur(s) en ligne : 0-7
    Thread(s) par cœur : 2
    Cœur(s) par socket : 4
    Socket(s) : 1
    Nœud(s) NUMA : 1
    Identifiant constructeur : GenuineIntel
    Famille de processeur : 6
    Modèle : 30
    Nom de modèle : Intel® Xeon® CPU X3480 @ 3.07GHz
    Révision : 5
    Vitesse du processeur en MHz : 3059.238
    BogoMIPS : 6118.47
    Virtualisation : VT-x
    Cache L1d : 32K
    Cache L1i : 32K
    Cache L2 : 256K
    Cache L3 : 8192K
    Nœud NUMA 0 de processeur(s) : 0-7

    free -m
    total used free shared buffers cached
    Mem: 32233 1192 31040 23 196 624
    -/+ buffers/cache: 372 31860
    Swap: 65514 0 65514

    df -h
    Sys. de fichiers Taille Utilisé Dispo Uti% Monté sur
    /dev/sda1 8,2G 1,2G 6,6G 15% /
    udev 10M 0 10M 0% /dev
    tmpfs 6,3G 8,6M 6,3G 1% /run
    tmpfs 16G 0 16G 0% /dev/shm
    tmpfs 5,0M 0 5,0M 0% /run/lock
    tmpfs 16G 0 16G 0% /sys/fs/cgroup
    /dev/sda5 2,7G 445M 2,1G 18% /var
    /dev/sda7 360M 3,8M 334M 2% /tmp
    /dev/sda8 842G 25G 774G 4% /home

    eth0: NIC Copper Link is Up, 1000 Mbps full duplex
    eth1: NIC Copper Link is Up, 1000 Mbps full duplex

    I made a test : Multicast and unicast 12 stations

    **Multicast **=> 04h 40min
    **Unicast ** => 28min


  • Moderator

    @Arrowhead-IT said:

    Something to understand about multicast (and someone should correct me if I’m wrong) is that is does all the decompression on the server and then sends the packets out following multicast protocol.

    That’s wrong, multicast sends the data compressed just like unicast deployments do. Decompression happens on the client side. And the idea of multicast is to broadcast throughout a broadcast domain, in order to send data only once instead of many times. There is a lot of overhead with multicast, but the more clients that are receiving the stream, the bigger the payoff.

    Don’t feel bad - I thought the same thing, but tom corrected me then as I am now for you. The post is somewhere on here - from maybe a year ago lol.


  • Testers

    @plegrand A couple questions
    What are the specs (cpu(s) model and speed, Ram speed and size, virtual/physical) of your FOG server?
    What are the specs of the workstations (specifically the slowest one)?
    What is the compression rating on the image you’re pushing?

    Something to understand about multicast (and someone should correct me if I’m wrong) is that is does all the decompression on the server and then sends the packets out following multicast protocol. The idea of multicast is to do more on the server side and less on the client and get a constant speed. It also is meant to prevent using a file or part of a file that is in use to make a fail safe against corrupting files and such. The downside is that if you’re using an image with a compression level of 9 and your server isn’t all that powerful, multi-cast is going to slow down a lot compared to unicast where decompression is done on the client.

    My point is if your server has less power or less optimization for decompressing then your clients, multicast will just be slower. Considering your speed difference of multicast and unicast, my guess would be this is the case.

    Also, a side note in favor of just using unicast, I have used fog to deploy to a lab of 40 computers (10 at a time, which can be changed with the Max clients setting in the storage configuration, just fyi) with the same image on unicast and had no problems with corruption and the speed was glorious.

    Another limiting factor is client and server port speeds. If your server is on a 10/100Mb port, you should upgrade that because it will make your life painful. But if your server is on a Gigabit switch then 10 clients on 100Mb switches will all run at around 1.7 GB/min (estimated mathematically and based on experience) If everything is on gigabit switches/ports then you might see a decrease in client speed as you add more simultaneous clients , but will still be around 2 or 3 times faster for 10 clients. Just to give you a little reference off the top of my head.

    It would probably be a good idea to start recording average speeds for common configurations and post them in the forum somewhere so people have a reference of how fast to configuration ought to be performing.


  • Moderator

    @plegrand said:

    On my switch i configure (force) the igmp version to “3”, may be i should use the default version : “2” ?

    Try it and let us know. at this point, anything that causes a change is good.

    An other question : i thought that fog used “udpcast” for multicasting, is it true ?

    I think fog uses udpsender.



  • @Wayne-Workman Hello
    just for information :
    On my switch i configure (force) the igmp version to “3”, may be i should use the default version : “2” ?

    An other question : i thought that fog used “udpcast” for multicasting, is it true ?



  • Now, just to see if multicast is the issue, can you multicast to just two computers? Make a temp group, put two computers in it, and then use that group to multicast. Is there a speed difference?

    I’ll make the test

    Also - the switch is important when multicasting.

    It 's Alcatel switch OS6250 and OS6450 configured like that : http://plegrand1.free.fr/pasted_image001.png
    I’m not sure if the configuration is ok
    Thanks for your help



  • @Wayne-Workman Roger that. I was thinking he might would have re-upload but wasn’t sure. Thank you sir.


  • Moderator

    @EAHarvey The compression setting is only used for image upload/capture. He would need to re-upload to try a different compression setting.



  • I would be willing to bet that it has a lot to do with your compression level. Check under FOG Configuration>FOG Settings>FOG Boot Settings>FOG_PIGZ_COMP. Set this to 2 or 3 and see if that makes a big difference. You may need to redo your image that your pushing but I don’t think you will. This may help[ with the speed issue.


  • Moderator

    @plegrand said:

    Storage Management => Default Member => Max Clients

    That’s the correct spot, but I’d strongly recommend decreasing it to about 3 instead of increasing it further. I see far better performance when less are running simultaneously.

    Now, just to see if multicast is the issue, can you multicast to just two computers? Make a temp group, put two computers in it, and then use that group to multicast. Is there a speed difference?

    If it’s a failing hdd or faulty patch cable, and that’s one of the two you choose, it’d be difficult to analyze the results.

    Also - the switch is important when multicasting.

    There is a certain amount of processing power involved with replicating a packet to all ports - and cheap switches just don’t cut it.

    There’s also maximum total throughput to consider. For example, at home I have a consumer grade Cisco Small business switch. It’s 1Gbps on each port and has 5 ports. But total internal throughput is 3Gbps. That means that I would never be able to multicast at home using that switch at 1Gbps speeds for more than 2 computers at a time. However I have a new 8 port 1Gbps z-link switch from China (for 28 bucks new) that has internal throughput of 5Gbps. Meaning that device would be able to multicast to 4 computers at once with 1Gbps speed to each.

    Again, cheap equipment just doesn’t cut it when it comes to multicast and really needing every port to operate at it’s maximum speed. The higher end Cisco equipment usually doesn’t have a problem though with this.



  • @Wayne-Workman
    Hello,
    as i said yesterday, once multicast finished, i launched an unicast task on the 12 stations.
    it worked fine. I thought 2 stations were failed but it was because of the limit of unicast clients (10)
    Then where is it possible to change the limit of unicast ?
    Fog Configuration => General settings => FOG_QUEUESIZE
    or
    Storage Management => Default Member => Max Clients

    For the multicast problem here is the result :
    for 12 station
    Multicast => 04:40:00
    Unicast => 00:28:00

    Thanks for your help



  • @Wayne-Workman
    I just made the test unicast (deploy and not multicast)
    It’s a room with 12 stations.
    2 stations failed
    10 works fine (1.6 GB/min)
    i’ll tell you more tomorrow



  • @Wayne-Workman
    I’ll made the test tomorrow with one station and unicast
    then with 2 stations and i’ll give you the result


  • Moderator

    @plegrand We can’t really troubleshoot while the multicast is running. I’d suggest waiting until it’s done, and then we can troubleshoot with just one or two computers.



  • @Junkhacker
    I would like to use multicast and solve this problem.
    Could it be a version multicast problem?
    I configure my switch with version 3


Log in to reply
 

430
Online

39.3k
Users

11.0k
Topics

104.4k
Posts

Looks like your connection to FOG Project was lost, please wait while we try to reconnect.