Multicast: udp-sender max-bitrate extra parameter

[gencap]

Hi there,

I’m using FOG 1.2.0 to take care of a lab with 30 machines running Win10 enterprise and Linux Mint.

The fog server runs under a CentOS 6.7 VM in a VMware ESXi 5.5 environment with multiple 1Gbps NICs and multiple VLANs.
The whole network backbone runs at 1Gbps (Hypervisors, core switches, trunks) while the NICs on the clients are connected to 100Mbps ports on distribution switches.

In such a situation, the unicast deployment runs quite well since I’m able to clone 5-6 clients at about 1.2 GByte/min, maxing out the fast ethernet channels.

Unfortunately, multicast has severe limitations due to the asymmetric bandwidth on the sender and on the receivers.
The transfer rate at the partclone side was a risible 20MByte/min.

I tried to set 100Mbps speed on the server before multicasting (by using ethtool after changing the virtual NIC from VMXNET3 to E1000)
but this didn’t help at all.

Then, thinking about a multicast issue in ESXi virtual switches, I reverted back to VMXNET3 to enable SplitRX Mode that gave me some MByte/min more (about 30MBps).

At this point, I modified the sources enabling a bandwidth shaping at the sender endpoint.
I added a “$udpsender_extra_opts = ’ --max-bitrate 90m ';” to MulticastTask.class.php, changing the related “UDPSENDERPATH” lines accordingly, and I was able to measure a speedup of a 20x factor (about 650-700 MByte/min at partclone UI for the same partition).
You’ll find a diff at the end of this message.

Anyway, /opt/fog/log/multicast.log.udpcast.1 shows very high re-xmits percentage indicating that there still are several network issues to solve. Multicasting to 4 clients shows about 49% re-xmit; this value jumps to 83% when multicasting to 9 clients.

I also tested “–async” and “–fec” options trying to make things smoother, but that didn’t work at all (udp-sender waited forever for “muted” clients).

It would be useful to add the “–max-bitrate” parameter to FOG server UI.

Another big improvement would be the possibility to state the multicast interface for every single group, so that a single fog server would be able to multicast to different groups on different interfaces/VLANs.

Any idea to improve my cloning infrastructure will be appreciated (using a physical fog-server is not feasible at this moment).

Regards,
GC

---

[root@fogserver ~]# diff MulticastTask.class.php ORIG_MulticastTask.class.php
62,65d61
<
<               // added by gencap
<               $udpsender_extra_opts = ' --max-bitrate 90m ';
<
96,97c92,93
<                               $cmd = 'cat '.$strRec.'|'.UDPSENDERPATH.$udpsender_extra_opts.' --min-receivers '.$this->getClientCount().' --portbase '.$this->getPortBase().' '.$interface.' '.$wait.' --full-duplex --ttl 32 --nokbd;';
<                               $cmd .= 'cat '.$strSys.'|'.UDPSENDERPATH.$udpsender_extra_opts.' --min-receivers '.$this->getClientCount().' --portbase '.$this->getPortBase().' '.$interface.' '.$wait.' --full-duplex --ttl 32 --nokbd;';
---
>                               $cmd = 'cat '.$strRec.'|'.UDPSENDERPATH.' --min-receivers '.$this->getClientCount().' --portbase '.$this->getPortBase().' '.$interface.' '.$wait.' --full-duplex --ttl 32 --nokbd;';
>                               $cmd .= 'cat '.$strSys.'|'.UDPSENDERPATH.' --min-receivers '.$this->getClientCount().' --portbase '.$this->getPortBase().' '.$interface.' '.$wait.' --full-duplex --ttl 32 --nokbd;';
100c96
<                               $cmd = 'cat '.$strSys.'|'.UDPSENDERPATH.$udpsender_extra_opts.' --min-receivers '.$this->getClientCount().' --portbase '.$this->getPortBase().' '.$interface.' '.$wait.' --full-duplex --ttl 32 --nokbd;';
---
>                               $cmd = 'cat '.$strSys.'|'.UDPSENDERPATH.' --min-receivers '.$this->getClientCount().' --portbase '.$this->getPortBase().' '.$interface.' '.$wait.' --full-duplex --ttl 32 --nokbd;';
125c121
<                                       $cmd .= 'cat '.$path.'|'.UDPSENDERPATH.$udpsender_extra_opts.' --min-receivers '.$this->getClientCount().' --portbase '.$this->getPortBase().' '.$interface.' '.$wait.' --full-duplex --ttl 32 --nokbd;';
---
>                                       $cmd .= 'cat '.$path.'|'.UDPSENDERPATH.' --min-receivers '.$this->getClientCount().' --portbase '.$this->getPortBase().' '.$interface.' '.$wait.' --full-duplex --ttl 32 --nokbd;';
129c125
<                               $cmd = 'cat '.rtrim($this->getImagePath(),'/').'|'.UDPSENDERPATH.$udpsender_extra_opts.' --min-receivers '.$this->getClientCount().' --portbase '.$this->getPortBase().' '.$interface.' '.$wait.' --full-duplex --ttl 32 --nokbd;';
---
>                               $cmd = 'cat '.rtrim($this->getImagePath(),'/').'|'.UDPSENDERPATH.' --min-receivers '.$this->getClientCount().' --portbase '.$this->getPortBase().' '.$interface.' '.$wait.' --full-duplex --ttl 32 --nokbd;';
157c153
<                                       $cmd .= 'cat '.$path.'|'.UDPSENDERPATH.$udpsender_extra_opts.' --min-receivers '.$this->getClientCount().' --portbase '.$this->getPortBase().' '.$interface.' '.$wait.' --full-duplex --ttl 32 --nokbd;';
---
>                                       $cmd .= 'cat '.$path.'|'.UDPSENDERPATH.' --min-receivers '.$this->getClientCount().' --portbase '.$this->getPortBase().' '.$interface.' '.$wait.' --full-duplex --ttl 32 --nokbd;';
182c178
<                                       $cmd .= 'cat '.$path.'|'.UDPSENDERPATH.$udpsender_extra_opts.' --min-receivers '.$this->getClientCount().' --portbase '.$this->getPortBase().' '.$interface.' '.$wait.' --full-duplex --ttl 32 --nokbd;';
---
>                                       $cmd .= 'cat '.$path.'|'.UDPSENDERPATH.' --min-receivers '.$this->getClientCount().' --portbase '.$this->getPortBase().' '.$interface.' '.$wait.' --full-duplex --ttl 32 --nokbd;';

[Tom Elliott]

If you upgrade to trunk versions the max bitrate parameter is GUI set table already. You edit your relevant node or nodes and add the information you need on the bitrate field.

[djc]

hi gencap
thanks for this post, new version MulticastTask.class.php in fog 1.5.4 have different code, if i change code like in image can run?