What should I expect speed wise from multicast?

[forte647]

Heya, guys.

During a unicast deployment deploying 1 image to 1 machine at a time, I can get speeds of about 5-6 gb/s, but when I multicast out to 7 machines, it slows down to 1 gb/s. Is this normal? I was under the impression that with multicast I could achieve speeds closer to 5 or 6 gb/s. All the machines (including the FOG server) are on a pretty new enterprise grade Netgear switch (S3300-52x ProSAFE stackable smart switch) There isn’t any packet shaping or any features like that enabled on the switch. I checked on the switch’s config page and verified that every connection is reading as a full 1000 Mbps.

Are there any other config settings I should look at, or is 1 gb/s transfer the best I’m going to get? Thanks for your time, any suggestions would be very helpful.

[george1421]

For a multicast stream you will be limited in speed to your slowest device. So I would expect that if you are getting 5GB/s I would budget about 50% loss because of multicasting. So a good estimate would be around 2.5GB/s in a perfect world. Maybe more, maybe less. What has a direct impact on the performance is two things.

1. Your networking infrastructure
2. Your target computers (i.e. mixing all quad core i7 computers with a Pentium P4 system will give you the transfer speed of the P4 [more or less]).

Your networking in infrastructure (more importantly) your switche(s) need to have the internal bandwidth to send multicast traffic to all ports who subscribe to the stream.

In a FOG deployment (unicast or multicast) the real work and CPU load is done by the target computers. All the FOG server does is send the image to the client and manage the process. The target computers take the image, uncompress it on the fly and write the image to disk. That is where the time to image comes from.

Even if you are getting 1GB/s that is not bad since you are imaging at 1/5 a single unicast speed you are imaging (in your testing) 20% more systems in the same amount of time. That 1GB/s should not drop significantly if you add 10 more systems. Now whats your deployment percentage against that single unicast stream.

[Sebastian Roth]

@forte647 While George is mostly right with all he said I want to add that in my ideal multicast world (which somehow not many people are able to achieve - and I still don’t understand why) a multicast test with udpcast (used in FOG) was almost as fast as unicasting to a single PC (minus 5% maybe but definitely not 50%).

Never forget that the slowest horse is the speed all horses can go! Try clustering in different sub groups to see which PC is going the slowest and see what’s wrong with it.

From this manual it sounds like this switch is definitely capable of handling multicast properly. Check out the multicast filtering features on page number 7 of this PDF. Not sure if any of these could possibly slow down multicast. To my knowledge most features configured wrong would completely prevent multicast from happening but not slow it down. Anyhow, there are still lots of things that I don’t know about networking…

[forte647]

Thanks for the responses. Every machine is identical. Same model, processor, ram, HDD, NIC, all that. I’ve taken everything off the switch except 30 machines I plan to image + the server running FOG. I’ve verified that every port in use on the switch has a full 1000Mbps link. I just imaged the 30 on my switch at about 1.80gb/s but I’d still like to go a bit faster if you think that’s possible.

I’ve verified that Spanning Tree Protocol is enabled on all ports in RSTP mode.

1.8 gb/s is much better than 1 gb/s, but can anyone think of anything else?

[george1421]

@Sebastian-Roth said in What should I expect speed wise from multicast?:

@forte647 While George is mostly right with all he said…

The wise guy in me what to say: I’m not sure if I should be honored or offended… <jk>

A lot depend on the switch(es) involved and your network design. IF you are talking about a single switch with 48 ports then you should get near unicast speeds as long as its an enterprise switch. With the prosafe listed in the OP it should handle 48 devices being multicast’d at one time. You have to consider what is happening when a multicast is running. The switching fabric on the switch has to send data to all multicast subscribers on that switch at the same time. While in general its only one stream, you have that single stream going 48 different places on that one switch. So that adds a continuous load to the backplane. Now once you start to leave the deployment switch as spread your multicast traffic across your campus network, now your multicast traffic has to compete with other traffic on the switch up links. This could leave to congestion since this is continuous traffic, unlike normal business network type of traffic that happens in burst. As Sebastian said the “slowest horse is the speed all horses can go”. So if you have a computer one a remote slow link that is participating in the multicast then all will be limited to the speed of the slowest horse.

If the OP is testing all 30 systems on this S3300-52x ProSAFE and he is getting 1/5 of a unicast stream, I might dig into the configuration a bit more (or even contact netgear to see if there is something specific that needs to be done). If the OP is multicasting across campus then there is probably not much he can do.

I was just going to run the numbers, but I see a confusion in the unit of measure. The OP posted 5-6gb/s (giga bits per second) (hint that’s faster then the 1GbE network he’s running on). I would expect the number to be 5-6GB/min (giga bytes per minute).

I just ran the numbers and 1.8GB/min translates to a throughput of less than 30MB/s (or about 3 times a 10Mhz network).

[phishphan]

Just an FYI from my experience today with multicast speeds. I was using a dumb Netgear GS516T and getting multicast speeds ranging from 900 mb to 1.5 gig at best. After switching (pun intended) to a better Netgear (GS716T) our speeds jumped to pushing 5 gigs consistently. I was always under the impression that a dumb switch was dumb and gigabit meant gigabit speeds. Turns out I was dumb and gigabit doesn’t guarantee gigabit. There are many more elements in play in switching speeds like its processor and backplane. So if everything else seems to work with fog but you’re still not achieving the speeds you thought or hoped for - always remember the IT troubleshooting mantra. Eliminate the hardware issues first.

[Sebastian Roth]

Thanks everyone for their great posts on this!

[MarkP]

@forte647 Are you using compressed images or uncompressed images for your multicast deployment?

If I remember correctly Unicast decompression is done on the client and Multicast decompression is done on the server.
We run all our images uncompressed for multicast purposes.

[Tom Elliott]

@MarkP Images are always compressed when sent and the client always decompresses them.

This is so network traffic is not such a big hinderance.

Why?

Regardless of how the data is sent, the amount of data is what causes bandwidth usage to spike. So if you have a 42GB Image file (uncompressed) being sent, Unicast or Multicast, uncompressed, it means your network has to transfer 42GB of data to the client. If you send the data compressed, that 42GB Image might be around 40% smaller. This means of the 42GB image your network only has to transfer 30GB, while the client can continue chugging on the data and how to place it. It’s faster to send data and less taxing on the server.

At one point (0.32 and earlier I believe), the way you described decompression is correct. It was sending compressed data to the client in unicast (meaning the client had to decompress) while sending decompressed data to the client in multicast. For 1.0.0 and up to now, the client always handles the decompression. This means the server has more room to work with when dealing with imaging and things can complete faster (from a server standpoint).

[Wayne Workman]

@MarkP said in What should I expect speed wise from multicast?:

If I remember correctly Unicast decompression is done on the client and Multicast decompression is done on the server.
We run all our images uncompressed for multicast purposes.

If you’re setting your compression down to 1, that’s probably your problem. Leave compression at the default, 6. You’ll need to re-upload after you change this value.