• Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login
  • Recent
  • Unsolved
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Register
  • Login

MultiCast very slow

Scheduled Pinned Locked Moved Solved
FOG Problems
7
23
13.4k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • P
    plegrand
    last edited by plegrand Feb 9, 2016, 7:27 AM Feb 9, 2016, 12:32 PM

    Hello,
    i’ve got some problem with multicast.
    I’m trying to deploy an image on 12 windows 7 stations. The stations are identicals. I didnt make any configuration in fog. Just configure manageable swith like that :
    http://plegrand1.free.fr/pasted_image001.png

    I can see in the log a lot of lines like that :
    Timeout notAnswered=[0,1,2,3,4,5,6,7,8,9,10,11] notReady=[0,1,2,3,4,5,6,7,8,9,10,11] nrAns=0 nrRead=0 nrPart=12 avg=450

    And in the “task” bar 170MB/min

    May be it could be a client which has problem. Is there a way to detect if there is a problematic client ?

    Does someone could help me to solve this problem ?

    Thank you for help

    1 Reply Last reply Reply Quote 0
    • W
      Wayne Workman
      last edited by Wayne Workman Feb 9, 2016, 9:14 AM Feb 9, 2016, 2:47 PM

      What version of FOG?

      Remember multicast runs at the speed of the slowest member in the group. It could be a bad patch cable or a bad HDD slowing the whole group down.

      Just as a baseline, can you see what speeds you get when you image an individual computer in the same group?

      I played around with doing an iPerf throughput task type for FOG in the past - might be time to resurrect that project.

      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
      Daily Clean Installation Results:
      https://fogtesting.fogproject.us/
      FOG Reporting:
      https://fog-external-reporting-results.fogproject.us/

      P 1 Reply Last reply Feb 9, 2016, 3:12 PM Reply Quote 0
      • P
        plegrand @Wayne Workman
        last edited by Feb 9, 2016, 3:12 PM

        @Wayne-Workman
        Hello
        Version 6237 in the cloud
        The deploy task is not finished (4 hours for the moment)
        I will make the unicast test just after
        Thanks for your help

        1 Reply Last reply Reply Quote 0
        • J
          Junkhacker Developer
          last edited by Feb 9, 2016, 3:14 PM

          if multicast is going that slow, it would probably go much faster to just unicast instead

          signature:
          Junkhacker
          We are here to help you. If you are unresponsive to our questions, don't expect us to be responsive to yours.

          P 1 Reply Last reply Feb 9, 2016, 3:23 PM Reply Quote 1
          • P
            plegrand @Junkhacker
            last edited by Feb 9, 2016, 3:23 PM

            @Junkhacker
            I would like to use multicast and solve this problem.
            Could it be a version multicast problem?
            I configure my switch with version 3

            W 1 Reply Last reply Feb 9, 2016, 3:31 PM Reply Quote 0
            • W
              Wayne Workman @plegrand
              last edited by Wayne Workman Feb 9, 2016, 9:32 AM Feb 9, 2016, 3:31 PM

              @plegrand We can’t really troubleshoot while the multicast is running. I’d suggest waiting until it’s done, and then we can troubleshoot with just one or two computers.

              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
              Daily Clean Installation Results:
              https://fogtesting.fogproject.us/
              FOG Reporting:
              https://fog-external-reporting-results.fogproject.us/

              P 3 Replies Last reply Feb 9, 2016, 3:32 PM Reply Quote 0
              • P
                plegrand @Wayne Workman
                last edited by plegrand Feb 9, 2016, 9:33 AM Feb 9, 2016, 3:32 PM

                @Wayne-Workman
                I’ll made the test tomorrow with one station and unicast
                then with 2 stations and i’ll give you the result

                1 Reply Last reply Reply Quote 0
                • P
                  plegrand @Wayne Workman
                  last edited by Feb 9, 2016, 4:05 PM

                  @Wayne-Workman
                  I just made the test unicast (deploy and not multicast)
                  It’s a room with 12 stations.
                  2 stations failed
                  10 works fine (1.6 GB/min)
                  i’ll tell you more tomorrow

                  1 Reply Last reply Reply Quote 1
                  • P
                    plegrand @Wayne Workman
                    last edited by plegrand Feb 10, 2016, 1:39 AM Feb 10, 2016, 7:12 AM

                    @Wayne-Workman
                    Hello,
                    as i said yesterday, once multicast finished, i launched an unicast task on the 12 stations.
                    it worked fine. I thought 2 stations were failed but it was because of the limit of unicast clients (10)
                    Then where is it possible to change the limit of unicast ?
                    Fog Configuration => General settings => FOG_QUEUESIZE
                    or
                    Storage Management => Default Member => Max Clients

                    For the multicast problem here is the result :
                    for 12 station
                    Multicast => 04:40:00
                    Unicast => 00:28:00

                    Thanks for your help

                    W 1 Reply Last reply Feb 10, 2016, 2:52 PM Reply Quote 0
                    • W
                      Wayne Workman @plegrand
                      last edited by Feb 10, 2016, 2:52 PM

                      @plegrand said:

                      Storage Management => Default Member => Max Clients

                      That’s the correct spot, but I’d strongly recommend decreasing it to about 3 instead of increasing it further. I see far better performance when less are running simultaneously.

                      Now, just to see if multicast is the issue, can you multicast to just two computers? Make a temp group, put two computers in it, and then use that group to multicast. Is there a speed difference?

                      If it’s a failing hdd or faulty patch cable, and that’s one of the two you choose, it’d be difficult to analyze the results.

                      Also - the switch is important when multicasting.

                      There is a certain amount of processing power involved with replicating a packet to all ports - and cheap switches just don’t cut it.

                      There’s also maximum total throughput to consider. For example, at home I have a consumer grade Cisco Small business switch. It’s 1Gbps on each port and has 5 ports. But total internal throughput is 3Gbps. That means that I would never be able to multicast at home using that switch at 1Gbps speeds for more than 2 computers at a time. However I have a new 8 port 1Gbps z-link switch from China (for 28 bucks new) that has internal throughput of 5Gbps. Meaning that device would be able to multicast to 4 computers at once with 1Gbps speed to each.

                      Again, cheap equipment just doesn’t cut it when it comes to multicast and really needing every port to operate at it’s maximum speed. The higher end Cisco equipment usually doesn’t have a problem though with this.

                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                      Daily Clean Installation Results:
                      https://fogtesting.fogproject.us/
                      FOG Reporting:
                      https://fog-external-reporting-results.fogproject.us/

                      P 1 Reply Last reply Feb 11, 2016, 3:16 PM Reply Quote 0
                      • E
                        EAHarvey
                        last edited by EAHarvey Feb 10, 2016, 10:01 AM Feb 10, 2016, 3:53 PM

                        I would be willing to bet that it has a lot to do with your compression level. Check under FOG Configuration>FOG Settings>FOG Boot Settings>FOG_PIGZ_COMP. Set this to 2 or 3 and see if that makes a big difference. You may need to redo your image that your pushing but I don’t think you will. This may help[ with the speed issue.

                        W 1 Reply Last reply Feb 10, 2016, 4:03 PM Reply Quote 1
                        • W
                          Wayne Workman @EAHarvey
                          last edited by Feb 10, 2016, 4:03 PM

                          @EAHarvey The compression setting is only used for image upload/capture. He would need to re-upload to try a different compression setting.

                          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                          Daily Clean Installation Results:
                          https://fogtesting.fogproject.us/
                          FOG Reporting:
                          https://fog-external-reporting-results.fogproject.us/

                          E P 2 Replies Last reply Feb 10, 2016, 4:11 PM Reply Quote 1
                          • E
                            EAHarvey @Wayne Workman
                            last edited by Feb 10, 2016, 4:11 PM

                            @Wayne-Workman Roger that. I was thinking he might would have re-upload but wasn’t sure. Thank you sir.

                            1 Reply Last reply Reply Quote 1
                            • P
                              plegrand @Wayne Workman
                              last edited by Feb 11, 2016, 3:16 PM

                              Now, just to see if multicast is the issue, can you multicast to just two computers? Make a temp group, put two computers in it, and then use that group to multicast. Is there a speed difference?

                              I’ll make the test

                              Also - the switch is important when multicasting.

                              It 's Alcatel switch OS6250 and OS6450 configured like that : http://plegrand1.free.fr/pasted_image001.png
                              I’m not sure if the configuration is ok
                              Thanks for your help

                              1 Reply Last reply Reply Quote 0
                              • P
                                plegrand @Wayne Workman
                                last edited by plegrand Feb 22, 2016, 9:25 AM Feb 22, 2016, 3:21 PM

                                @Wayne-Workman Hello
                                just for information :
                                On my switch i configure (force) the igmp version to “3”, may be i should use the default version : “2” ?

                                An other question : i thought that fog used “udpcast” for multicasting, is it true ?

                                W 1 Reply Last reply Feb 22, 2016, 4:57 PM Reply Quote 0
                                • W
                                  Wayne Workman @plegrand
                                  last edited by Feb 22, 2016, 4:57 PM

                                  @plegrand said:

                                  On my switch i configure (force) the igmp version to “3”, may be i should use the default version : “2” ?

                                  Try it and let us know. at this point, anything that causes a change is good.

                                  An other question : i thought that fog used “udpcast” for multicasting, is it true ?

                                  I think fog uses udpsender.

                                  Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                                  Daily Clean Installation Results:
                                  https://fogtesting.fogproject.us/
                                  FOG Reporting:
                                  https://fog-external-reporting-results.fogproject.us/

                                  1 Reply Last reply Reply Quote 0
                                  • JJ FullmerJ
                                    JJ Fullmer Testers
                                    last edited by Feb 22, 2016, 8:54 PM

                                    @plegrand A couple questions
                                    What are the specs (cpu(s) model and speed, Ram speed and size, virtual/physical) of your FOG server?
                                    What are the specs of the workstations (specifically the slowest one)?
                                    What is the compression rating on the image you’re pushing?

                                    Something to understand about multicast (and someone should correct me if I’m wrong) is that is does all the decompression on the server and then sends the packets out following multicast protocol. The idea of multicast is to do more on the server side and less on the client and get a constant speed. It also is meant to prevent using a file or part of a file that is in use to make a fail safe against corrupting files and such. The downside is that if you’re using an image with a compression level of 9 and your server isn’t all that powerful, multi-cast is going to slow down a lot compared to unicast where decompression is done on the client.

                                    My point is if your server has less power or less optimization for decompressing then your clients, multicast will just be slower. Considering your speed difference of multicast and unicast, my guess would be this is the case.

                                    Also, a side note in favor of just using unicast, I have used fog to deploy to a lab of 40 computers (10 at a time, which can be changed with the Max clients setting in the storage configuration, just fyi) with the same image on unicast and had no problems with corruption and the speed was glorious.

                                    Another limiting factor is client and server port speeds. If your server is on a 10/100Mb port, you should upgrade that because it will make your life painful. But if your server is on a Gigabit switch then 10 clients on 100Mb switches will all run at around 1.7 GB/min (estimated mathematically and based on experience) If everything is on gigabit switches/ports then you might see a decrease in client speed as you add more simultaneous clients , but will still be around 2 or 3 times faster for 10 clients. Just to give you a little reference off the top of my head.

                                    It would probably be a good idea to start recording average speeds for common configurations and post them in the forum somewhere so people have a reference of how fast to configuration ought to be performing.

                                    Have you tried the FogApi powershell module? It's pretty cool IMHO
                                    https://github.com/darksidemilk/FogApi
                                    https://fogapi.readthedocs.io/en/latest/
                                    https://www.powershellgallery.com/packages/FogApi
                                    https://forums.fogproject.org/topic/12026/powershell-api-module

                                    W P 2 Replies Last reply Feb 22, 2016, 10:41 PM Reply Quote 1
                                    • W
                                      Wayne Workman @JJ Fullmer
                                      last edited by Feb 22, 2016, 10:41 PM

                                      @Arrowhead-IT said:

                                      Something to understand about multicast (and someone should correct me if I’m wrong) is that is does all the decompression on the server and then sends the packets out following multicast protocol.

                                      That’s wrong, multicast sends the data compressed just like unicast deployments do. Decompression happens on the client side. And the idea of multicast is to broadcast throughout a broadcast domain, in order to send data only once instead of many times. There is a lot of overhead with multicast, but the more clients that are receiving the stream, the bigger the payoff.

                                      Don’t feel bad - I thought the same thing, but tom corrected me then as I am now for you. The post is somewhere on here - from maybe a year ago lol.

                                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                                      Daily Clean Installation Results:
                                      https://fogtesting.fogproject.us/
                                      FOG Reporting:
                                      https://fog-external-reporting-results.fogproject.us/

                                      JJ FullmerJ 1 Reply Last reply Feb 23, 2016, 9:00 PM Reply Quote 1
                                      • P
                                        plegrand @JJ Fullmer
                                        last edited by Feb 23, 2016, 8:36 AM

                                        @Arrowhead-IT
                                        lscpu
                                        Architecture : x86_64
                                        Mode(s) opératoire(s) des processeurs : 32-bit, 64-bit
                                        Boutisme : Little Endian
                                        Processeur(s) : 8
                                        Liste de processeur(s) en ligne : 0-7
                                        Thread(s) par cœur : 2
                                        Cœur(s) par socket : 4
                                        Socket(s) : 1
                                        Nœud(s) NUMA : 1
                                        Identifiant constructeur : GenuineIntel
                                        Famille de processeur : 6
                                        Modèle : 30
                                        Nom de modèle : Intel® Xeon® CPU X3480 @ 3.07GHz
                                        Révision : 5
                                        Vitesse du processeur en MHz : 3059.238
                                        BogoMIPS : 6118.47
                                        Virtualisation : VT-x
                                        Cache L1d : 32K
                                        Cache L1i : 32K
                                        Cache L2 : 256K
                                        Cache L3 : 8192K
                                        Nœud NUMA 0 de processeur(s) : 0-7

                                        free -m
                                        total used free shared buffers cached
                                        Mem: 32233 1192 31040 23 196 624
                                        -/+ buffers/cache: 372 31860
                                        Swap: 65514 0 65514

                                        df -h
                                        Sys. de fichiers Taille Utilisé Dispo Uti% Monté sur
                                        /dev/sda1 8,2G 1,2G 6,6G 15% /
                                        udev 10M 0 10M 0% /dev
                                        tmpfs 6,3G 8,6M 6,3G 1% /run
                                        tmpfs 16G 0 16G 0% /dev/shm
                                        tmpfs 5,0M 0 5,0M 0% /run/lock
                                        tmpfs 16G 0 16G 0% /sys/fs/cgroup
                                        /dev/sda5 2,7G 445M 2,1G 18% /var
                                        /dev/sda7 360M 3,8M 334M 2% /tmp
                                        /dev/sda8 842G 25G 774G 4% /home

                                        eth0: NIC Copper Link is Up, 1000 Mbps full duplex
                                        eth1: NIC Copper Link is Up, 1000 Mbps full duplex

                                        I made a test : Multicast and unicast 12 stations

                                        **Multicast **=> 04h 40min
                                        **Unicast ** => 28min

                                        1 Reply Last reply Reply Quote 0
                                        • JJ FullmerJ
                                          JJ Fullmer Testers @Wayne Workman
                                          last edited by Feb 23, 2016, 9:00 PM

                                          @Wayne-Workman I wonder if I got compression part from that post a year ago. Some of that is what I remember from my college Networking class.

                                          @plegrand Looks like the problem shouldn’t be the server power, and I was wrong about that part anyway.
                                          Considering the speed difference you already are seeing, why not just use unicast? I’m just curious if there’s a specific requirement for multicast?

                                          Have you tried the FogApi powershell module? It's pretty cool IMHO
                                          https://github.com/darksidemilk/FogApi
                                          https://fogapi.readthedocs.io/en/latest/
                                          https://www.powershellgallery.com/packages/FogApi
                                          https://forums.fogproject.org/topic/12026/powershell-api-module

                                          1 Reply Last reply Reply Quote 0
                                          • 1
                                          • 2
                                          • 1 / 2
                                          • First post
                                            Last post

                                          180

                                          Online

                                          12.0k

                                          Users

                                          17.3k

                                          Topics

                                          155.2k

                                          Posts
                                          Copyright © 2012-2024 FOG Project