Fog Multicast Sessions: What happens when a host in session is powered off and what happens when it is powered back on?

[Sebastian Roth]

@george1421 Great, thanks for looking at the code! Seeing the figure now, 200, I notice that it’s hard to estimate how long it takes for a client to drop as the timeout for each single package seems to be some kind of calculated waitAverage time in the code (senddata.c).
@zacadams Nevertheless you can just start playing with the option to see if you can pull enhance the slowdown situation when a client is being shut down in the middle of multicasting. For testing this I’d suggest the following strategy: Do three tests each:

• do not turn off any of the clients to see if it still is reliable
• pull the network cable of one of the clients while multicast is on for 5 seconds - see if that is enough for it to be dropped already
• turn off one of the clients while multicasting and see how long it takes for it to drop

I’d suggest first try a value of 100, if that is working great (client not dropping even for the 5 second disconnect test) test 50, then 25, then 12. The closer you get to 1 the more “responsive” your dropout will be noticed and speed coming backup again hopefully. If you see the client being dropped on the 5-second-disconnect-test I’d go the other way, so if it drops at 50, do your next test with 75…

[Tom Elliott]

@sebastian-roth I’m wondering if it’d be worth our time to build our own method of multi casting? It’d be quite an undertaking and all but if udpsender is no longer under active development, it might make it worth our time and we’d have full controls - in theory.

[Sebastian Roth]

@Tom-Elliott Well that would definitely be an interesting project and I’ve thought about this as well when looking into this. But it’s a huge undertaking I am afraid. I’d schedule this on the wish list for FOG 2.0 at best.

[zacadams]

@Sebastian-Roth where will I need to apply these options? I went to 420 on multicasttask.class.php but it was the end of a block?

[Sebastian Roth]

@zacadams Well, which version of FOG are you running?!

[zacadams]

@sebastian-roth sorry for the delayed response. We are currently running FOG 1.5.0

[Sebastian Roth]

@zacadams Sorry, I meant below 421… see here. To add the --retries-until-drop option insert a field into the $buildcmd array. Let us know if you need assistance with coding.

[Szeraax]

@tom-elliott

have full controls

This would be cool. I don’t use multicast currently cause I’ve used it on a prod network midday before… But, being able to do stuff like “rolling multicast” where the sender will start over for clients that joined later and get them all to completion ASAP (maybe that’s a thing that udpsender already does and I’ve missed it). etc.

[Tom Elliott]

@szeraax by full controls I simple meant joining /leaving sessions/clients while still being able to image. Rolling multicast doesn’t make sense to me as it would cause the whole session to wait until the other “behind” are catching up. This would seem to be more appropriate, then, as a unicast. At least in my eyes. Of course as @Sebastian-Roth stated, this would be a massive undertaking just getting what I hope.

[Szeraax]

@tom-elliott I mean rolling as in:

If the multicast is 50% done and a new client connects, it will begin writing at the same block as the others. Once all others complete, the storage node sees that there are clients who aren’t done. Asks them what block ranges they still need, and then gives out those blocks. Thus, I could start a multicast and join any number of clients while it is going. Assuming that the last one joins before the first one finishes, the total data sent would be less than 2x the image. Additionally, the first client was not slowed down in any way by the new clients joining.

a la, a feature that I think Ghost had once upon a time.

[zacadams]

@sebastian-roth I apologize again they have had me bouncing around all last week so I haven’t had a change to test the
–retries-until-drop option yet. I’ll report back this morning about what I find.

[zacadams]

@sebastian-roth that did not appear to do anything.

[Sebastian Roth]

@zacadams What exactly did you try? Which values? Did the udp-sender calls on the FOG server look different according to the changes you made in the code? Check with ps ax | grep udp on the console while a multicast task is scheduled.

[zacadams]

@sebastian-roth to line 421 I have changed to:

$buildcmd = array(
            UDPSENDERPATH,
            (
                $this->getBitrate() ?
                sprintf(' --max-bitrate %s', $this->getBitrate()) :
                null
            ),
            (
                $this->getInterface() ?
                sprintf(' --interface %s', $this->getInterface()) :
                null
            ),
            sprintf(
                ' --min-receivers %d',
                (
                    $this->getClientCount() ?
                    $this->getClientCount():
                    self::getClass('HostManager')->count()
                )
            ),
            sprintf(' --max-wait %s', '%d'),
            (
                $address ?
                sprintf(' --mcast-data-address %s', $address) :
                null
            ),
            (
                $multicastrdv ?
                sprintf(' --mcast-rdv-address %s', $multicastrdv) :
                null
            ),
            sprintf(' --portbase %s', $this->getPortBase()),
            sprintf(' %s', $duplex),
            ' --ttl 32',
            ' --nokbd',
            ' --nopointopoint',
            ' --retries-until-drop retries 100',
        );

This has been done to the main fog server and the fog storage node that will be performing the multicast session. The udp-sendercalls did look different in the logs as well.

[zacadams]

Oh lord I see what I may have done wrong, I need to remove the retries. Let me try this again this morning.

[zacadams]

@sebastian-roth ok after fixing my error to the multicasttask.class.php I ran the 3 tests you suggested

• do not turn off any of the clients to see if it still is reliable

• pull the network cable of one of the clients while multicast is on for 5 seconds - see if that is enough for it to be dropped already

• turn off one of the clients while multicasting and see how long it takes for it to drop

During the first test, multicast worked normally and the new argument added to multicasttask.class.php was reflecting in the logs. During the second test pulling the network cable did not appear to drop the disconnected host, but upon plugging the network cable back in the session returned to normal speeds. During the last lest of powering a client off during a session, the multicast session came to a near halt and is only transferring at around 100 MBpm. I’ve left the session going all night and the session is still active and moving at the throttled pace.