Multicasting Issues



  • I’m having a lot of trouble getting multicast working in my environment. I work for a University where our networks are all segmented into multiple VLANs, and I don’t have the ability to put FOG on the same VLAN as the clients - we have about 10 computer labs and our FOG server is a VMWare virtual machine in our data center. Each computer lab is on its own VLAN, and the data center is as well.

    Our network team has a lot of controls on the network that I cannot see, but they assigned me a multicast range of 239.132.20.0 /24. Since this doesn’t match the default that FOG uses, I upgraded to the Trunk version of FOG to allow me to customize that. The network team setup a test multicast node within that multicast range and ran several small tests from their utilities across all of my VLANs and have had no hiccups at all, leading them to believe that the problem lies on the FOG server somewhere.

    Here’s where I’m at on the server:
    OS: Ubuntu Server 14.04.3 LTS
    FOG Trunk revision: 4623

    On the FOG server, I have a small 8GB image for testing. Under FOG Configuration -> FOG Settings -> Multicast Settings, I have configured the following:

    FOG_UDPCAST_INTERFACE: eth0
    FOG_UDPCAST_STARTINGPORT: 63072
    FOG_MULTICAST_MAX_SESSIONS: 64
    FOG_UDPCAST_MAXWAIT	: 10
    FOG_MULTICAST_ADDRESS: 239.132.20.10
    FOG_MULTICAST_PORT_OVERRIDE: 0
    FOG_MULTICAST_DUPLEX: FULL_DUPLEX
    

    I have registered two test hosts, and am attempting to get one working before trying two. I’ve double-checked under Active Tasks and Active Multicast tasks to ensure nothing is currently running. I’ve rebooted the server and restarted FOGMulticastManager, FOGImageReplicator, and FOGScheduler services (in case they didn’t start properly).

    When trying to multicast to a single client, I get inconsistent results. Most of the time, the client gets stuck at “Starting to restore image ( - ) to device (/dev/sda1).” Other times, it’ll get past that point and show a status bar, but the bar will be stuck at 0% with a transfer rate of 0.00bytes/min. The latter of which is incredibly difficult for me to replicate, I’ve found. Most of the time, I’m stuck at the former.

    Here’s what the multicast.log looks like:

    [01-09-16 6:31:41 pm] Interface Ready with IP Address: 10.133.132.127
    
    [01-09-16 6:31:41 pm]  * Starting MulticastManager Service
    
    [01-09-16 6:31:41 pm]  * Checking for new items every 10 seconds
    
    [01-09-16 6:31:41 pm]  * Starting service loop
    
    [01-09-16 6:31:41 pm]  * No tasks found!
    [01-09-16 6:31:51 pm]  * No tasks found!
    
    [01-09-16 6:32:01 pm]  * No tasks found!
    
    [01-09-16 6:32:11 pm]  | Sleeping for 10 seconds to ensure tasks are properly submitted
    
    [01-09-16 6:32:21 pm]  | 0 tasks to be cleaned
    [01-09-16 6:32:21 pm]  | 1 task found
    
    [01-09-16 6:32:21 pm]  | Task (11) Multi-Cast Task is new!
    
    [01-09-16 6:32:21 pm]  | Task (11) Multi-Cast Task has been cleaned.
    
    [01-09-16 6:32:21 pm]  | Task (11) /images/Test162016 image file found.
    
    [01-09-16 6:32:21 pm]  | Task (11) 1 client(s) found.
    Udp-sender 20120424
    
    [01-09-16 6:32:21 pm]  | Task (11) Multi-Cast Task sending on base port: 64512
    UDP sender for (stdin) at 10.133.132.127 on eth0 
    Broadcasting control to 224.0.0.1
    [01-09-16 6:32:21 pm]  | CMD: cat /images/Test162016/d1p1.img | /usr/local/sbin/udp-sender --interface eth0 --min-receivers 1 --max-wait 600 --mcast-data-address 239.132.20.10 --portbase 64512 --full-duplex --ttl 32 --nokbd --nopointopoint;cat /images/Test162016/d1p2.img | /usr/local/sbin/udp-sender --interface eth0 --min-receivers 1 --max-wait 600 --mcast-data-address 239.132.20.10 --portbase 64512 --full-duplex --ttl 32 --nokbd --nopointopoint;
    [01-09-16 6:32:21 pm]  | Task (11) Multi-Cast Task has started.
    
    [01-09-16 6:32:31 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:32:31 pm]  | 1 task found
    
    [01-09-16 6:32:31 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    [01-09-16 6:32:41 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:32:41 pm]  | 1 task found
    
    [01-09-16 6:32:41 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    [01-09-16 6:32:51 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:32:51 pm]  | 1 task found
    
    [01-09-16 6:32:51 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    [01-09-16 6:33:01 pm]  | 0 tasks to be cleaned
    [01-09-16 6:33:01 pm]  | 1 task found
    
    [01-09-16 6:33:01 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    [01-09-16 6:33:11 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:33:11 pm]  | 1 task found
    
    [01-09-16 6:33:11 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    [01-09-16 6:33:21 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:33:21 pm]  | 1 task found
    
    [01-09-16 6:33:21 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    New connection from 10.20.129.47  (#0) 00000009
    Starting transfer: 00000009
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:33:31 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:33:31 pm]  | 1 task found
    
    [01-09-16 6:33:31 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:33:41 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:33:41 pm]  | 1 task found
    
    [01-09-16 6:33:41 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:33:51 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:33:51 pm]  | 1 task found
    
    [01-09-16 6:33:51 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:34:02 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:34:02 pm]  | 1 task found
    
    [01-09-16 6:34:02 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:34:12 pm]  | 0 tasks to be cleaned
    [01-09-16 6:34:12 pm]  | 1 task found
    
    [01-09-16 6:34:12 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:34:22 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:34:22 pm]  | 1 task found
    
    [01-09-16 6:34:22 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:34:32 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:34:32 pm]  | 1 task found
    
    [01-09-16 6:34:32 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:34:42 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:34:42 pm]  | 1 task found
    
    [01-09-16 6:34:42 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:34:52 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:34:52 pm]  | 1 task found
    
    [01-09-16 6:34:52 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:35:02 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:35:02 pm]  | 1 task found
    
    [01-09-16 6:35:02 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:35:12 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:35:12 pm]  | 1 task found
    
    [01-09-16 6:35:12 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:35:22 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:35:22 pm]  | 1 task found
    
    [01-09-16 6:35:22 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:35:32 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:35:32 pm]  | 1 task found
    
    [01-09-16 6:35:32 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:35:42 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:35:42 pm]  | 1 task found
    
    [01-09-16 6:35:42 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:35:52 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:35:52 pm]  | 1 task found
    
    [01-09-16 6:35:52 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:36:02 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:36:02 pm]  | 1 task found
    
    [01-09-16 6:36:02 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:36:12 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:36:12 pm]  | 1 task found
    
    [01-09-16 6:36:12 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:36:22 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:36:22 pm]  | 1 task found
    
    [01-09-16 6:36:22 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:36:32 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:36:32 pm]  | 1 task found
    
    [01-09-16 6:36:32 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    
    [01-09-16 6:36:42 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:36:42 pm]  | 1 task found
    
    [01-09-16 6:36:42 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    Dropping client #0 because of timeout
    Disconnecting #0 (10.20.129.47)
    bytes=                re-xmits=0000000 (  0.0%) slice=0112 -   0
    Transfer complete.
    
    cat: write error: Broken pipe
    Udp-sender 20120424
    UDP sender for (stdin) at 10.133.132.127 on eth0 
    Broadcasting control to 224.0.0.1
    
    [01-09-16 6:36:52 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:36:52 pm]  | 1 task found
    
    [01-09-16 6:36:52 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    [01-09-16 6:37:02 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:37:02 pm]  | 1 task found
    
    [01-09-16 6:37:02 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    [01-09-16 6:37:12 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:37:12 pm]  | 1 task found
    
    [01-09-16 6:37:12 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    [01-09-16 6:37:22 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:37:22 pm]  | 1 task found
    
    [01-09-16 6:37:22 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    [01-09-16 6:37:32 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:37:32 pm]  | 1 task found
    
    [01-09-16 6:37:32 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    [01-09-16 6:37:42 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:37:42 pm]  | 1 task found
    
    [01-09-16 6:37:42 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    [01-09-16 6:37:52 pm]  | 0 tasks to be cleaned
    
    [01-09-16 6:37:52 pm]  | 1 task found
    
    [01-09-16 6:37:52 pm]  | Task (11) Multi-Cast Task is already running PID 3793
    
    
    

    I’m not sure where to go next in my troubleshooting. Any help?



  • Cool - issue resolved!

    This was absolutely an issue on our network, but we’re not sure if it was an old static Rendezvous Point that was configured on our core or if it was our VPC redundancy between our two nexus switches, but we were able to solve it by disabling pim on one of the nexuses and removing that static RP.

    Thanks for the help on investigating, everyone. It’s much appreciated.



  • @Sebastian-Roth Yes, we’ve looked at IGMP, but our topology is fairly complex. Currently the network team is working on moving rendezvous points closer to the server/clients and investigating potential issues in NXOS, as we recently migrated from Cisco IOS cores to Nexus.

    Unfortunately, my visibility on this is very limited, so I’m doing my best to keep this rolling on the server side as we troubleshoot.

    I’ll update with more info later as we progress.


  • Developer

    @kenneth.sisco said:

    We’re still having trouble narrowing this down, but is it safe to assume that if Multicast is working across the same VLAN that it’s no longer a fog issue?

    I would think so! Looked into igmp settings on your switches yet?



  • We’re still having trouble narrowing this down, but is it safe to assume that if Multicast is working across the same VLAN that it’s no longer a fog issue?


  • Moderator

    @kenneth.sisco said:

    Is there a way to see that? Or a way to specify it?

    It’s set to 32.

    The setting is part of the init, so to change it, you’ll need to edit it.

    Here are instructions on editing the init:
    https://wiki.fogproject.org/wiki/index.php?title=Modifying_the_Init_Image

    Here’s the command I used to find the command and it’s output.

    grep -rnw '/root/svn/trunk' -e "udp-receiver"

    /root/svn/trunk/.svn/pristine/34/347286bf65c7493cf5cd96049b458c34ad245cff.svn-base:374:        udp-receiver --nokbd --portbase $port --ttl 32 --mcast-rdv-address $storageip 2>/dev/null > /tmp/pigz1 &
    /root/svn/trunk/.svn/pristine/9f/9fbd1e42a40bc6fc567380e512a84c93d309d6b7.svn-base:442:            udp-receiver --nokbd --portbase $port --ttl 32 --mcast-rdv-address $storageip 2>/dev/null >/tmp/pigz1 &
    /root/svn/trunk/src/buildroot/package/fog/scripts/usr/share/fog/lib/funcs.sh:442:            udp-receiver --nokbd --portbase $port --ttl 32 --mcast-rdv-address $storageip 2>/dev/null >/tmp/pigz1 &
    

    For the record, I have went through 0.32’s code and I don’t see that ttl is specified at all in 0.32.
    Here is the output of the above search command when I pointed it at the 0.32 directory I checked out via svn:

    (the code lines are very wide, click and drag right to see it all)

    /root/svn/0.32/trunk/.svn/pristine/0c/0ce1eec0cc580ce17d377e5ef7c3ca6b4d39abab.svn-base:271:                                                        udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | gunzip -f -c 2>/dev/null | partimage -f3 -b restore $win7sys stdin
    /root/svn/0.32/trunk/.svn/pristine/0c/0ce1eec0cc580ce17d377e5ef7c3ca6b4d39abab.svn-base:273:                                                        udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | gunzip -f -c 2>/dev/null |  partimage -f3 -b restore $win7rec stdin
    /root/svn/0.32/trunk/.svn/pristine/0c/0ce1eec0cc580ce17d377e5ef7c3ca6b4d39abab.svn-base:274:                                                        udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | gunzip -f -c 2>/dev/null | partimage -f3 -b restore $win7sys stdin
    /root/svn/0.32/trunk/.svn/pristine/0c/0ce1eec0cc580ce17d377e5ef7c3ca6b4d39abab.svn-base:278:                                                        udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | gunzip -f -c 2>/dev/null | partimage -f3 -b restore ${part} stdin
    /root/svn/0.32/trunk/.svn/pristine/0c/0ce1eec0cc580ce17d377e5ef7c3ca6b4d39abab.svn-base:381:                                                        udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | gunzip -f -c 2>/dev/null | partimage -f3 -b restore $part stdin;
    /root/svn/0.32/trunk/.svn/pristine/0c/0ce1eec0cc580ce17d377e5ef7c3ca6b4d39abab.svn-base:469:                                                        udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | gunzip -f -c 2>/dev/null |  partimage -f3 -b restore $part stdin ;
    /root/svn/0.32/trunk/.svn/pristine/0c/0ce1eec0cc580ce17d377e5ef7c3ca6b4d39abab.svn-base:519:                                                    udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | gunzip -f -c 2>/dev/null | dd of=${hd}
    /root/svn/0.32/trunk/.svn/pristine/24/2461bc1e810a56b21208edfc4164f513270cdba2.svn-base:340:                                                        udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | partimage -f3 -b restore $win7sys stdin
    /root/svn/0.32/trunk/.svn/pristine/24/2461bc1e810a56b21208edfc4164f513270cdba2.svn-base:342:                                                        udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null |  partimage -f3 -b restore $win7rec stdin
    /root/svn/0.32/trunk/.svn/pristine/24/2461bc1e810a56b21208edfc4164f513270cdba2.svn-base:343:                                                        udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | partimage -f3 -b restore $win7sys stdin
    /root/svn/0.32/trunk/.svn/pristine/24/2461bc1e810a56b21208edfc4164f513270cdba2.svn-base:346:                                                        #udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null |  partimage -f3 -b restore ${hd}1 stdin
    /root/svn/0.32/trunk/.svn/pristine/24/2461bc1e810a56b21208edfc4164f513270cdba2.svn-base:347:                                                        #udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null |  partimage -f3 -b restore ${hd}2 stdin
    /root/svn/0.32/trunk/.svn/pristine/24/2461bc1e810a56b21208edfc4164f513270cdba2.svn-base:348:                                                        #udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | partimage -f3 -b restore $win7sys stdin
    /root/svn/0.32/trunk/.svn/pristine/24/2461bc1e810a56b21208edfc4164f513270cdba2.svn-base:352:                                                        udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | partimage -f3 -b restore ${part} stdin
    /root/svn/0.32/trunk/.svn/pristine/24/2461bc1e810a56b21208edfc4164f513270cdba2.svn-base:467:                                                        udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | partimage -f3 -b restore $part stdin;
    /root/svn/0.32/trunk/.svn/pristine/24/2461bc1e810a56b21208edfc4164f513270cdba2.svn-base:555:                                                        udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null |  partimage -f3 -b restore $part stdin ;
    /root/svn/0.32/trunk/.svn/pristine/24/2461bc1e810a56b21208edfc4164f513270cdba2.svn-base:605:                                                    udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | gunzip -f -c 2>/dev/null | dd of=${hd}
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog:340:                                                                   udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | partimage -f3 -b restore $win7sys stdin
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog:342:                                                                   udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null |  partimage -f3 -b restore $win7rec stdin
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog:343:                                                                   udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | partimage -f3 -b restore $win7sys stdin
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog:346:                                                           #       udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null |  partimage -f3 -b restore ${hd}1 stdin
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog:347:                                                           #       udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null |  partimage -f3 -b restore ${hd}2 stdin
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog:348:                                                           #       udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | partimage -f3 -b restore $win7sys stdin
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog:352:                                                           udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | partimage -f3 -b restore ${part} stdin
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog:467:                                                                       udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | partimage -f3 -b restore $part stdin;
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog:555:                                                                       udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null |  partimage -f3 -b restore $part stdin ;
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog:605:                                                   udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | gunzip -f -c 2>/dev/null | dd of=${hd}
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog.bkup:271:                                                                  udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | gunzip -f -c 2>/dev/null | partimage -f3 -b restore $win7sys stdin
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog.bkup:273:                                                                  udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | gunzip -f -c 2>/dev/null |  partimage -f3 -b restore $win7rec stdin
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog.bkup:274:                                                                  udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | gunzip -f -c 2>/dev/null | partimage -f3 -b restore $win7sys stdin
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog.bkup:278:                                                              udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | gunzip -f -c 2>/dev/null | partimage -f3 -b restore ${part} stdin
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog.bkup:381:                                                                  udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | gunzip -f -c 2>/dev/null | partimage -f3 -b restore $part stdin;
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog.bkup:469:                                                                  udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | gunzip -f -c 2>/dev/null |  partimage -f3 -b restore $part stdin ;
    /root/svn/0.32/trunk/src/buildroot/package/customize/fog/scripts/bin/fog.bkup:519:                                                      udp-receiver --nokbd --portbase ${port} --mcast-rdv-address ${storageip} 2>/dev/null | gunzip -f -c 2>/dev/null | dd of=${hd}
    


  • So I’m working with our network team to troubleshoot this, and we have “broadcast forwarding” working for the most part, but we’re going to keep digging. One thing we were concerned about was TTL on the client side. We can see the TTL from the server in the multicast.log, but we’re concerned about what the TTL is on the udp-receiver command. Is there a way to see that? Or a way to specify it? I was looking for the task files that 0.32 used to have stored in the tftpboot folder, but this version of Fog handles that differently, so I’m not sure where to find them.


  • Moderator

    @Tom-Elliott said:

    I’d take a look if you can on whether broadcast forwarding is on.

    I agree with Tom. Broadcast forwarding is exactly what you need. This happens on your router - as a routing device is the only way that data from one LAN (or V-LAN) can traverse to other LANs (or V-LANS).

    Broadcast forwarding is called different things based on what vendor / piece of equipment you use. But it’s quite simply that which allows broadcast traffic to pass through a router. You’re network team don’t need to open everything up, they just need to allow passage from the server vlan to other vlans, and back. If your network team is really skilled, they can even limit it to a specific port on a specific switch, or a specific sending IP (the fog server’s IP).


  • Senior Developer

    @kenneth.sisco making a connection is simple but the udp stream is not being allowed to pass after the initial connection is made. The separated network method you used sounds to have a slight difference from the real network side. I don’t know what or where. This does not sound like something fog is failing to do though, like I said, I’d take a look if you can on whether broadcast forwarding is on.



  • So I worked with our network team and we were able to setup a small DHCP pool on the server VLAN temporarily for a test, and it worked fine. Once we tried again cross-VLAN, we had the same issues. I’ve copied the multicast.log section of the successful multicast below. The unsuccessful one is the same as my prior post in this thread. What I find interesting is that the multicast.log file shows that there is a new connection from the client cross-VLAN and even echoes its IP Address, but never transfers any data.

    Successful multicast (same VLAN; shortened to allow for forum posting):

    [01-11-16 4:54:24 pm]  * No tasks found!
    [01-11-16 4:54:34 pm]  | Sleeping for 10 seconds to ensure tasks are properly submitted
    [01-11-16 4:54:44 pm]  | 0 tasks to be cleaned
    [01-11-16 4:54:44 pm]  | 1 task found
    [01-11-16 4:54:44 pm]  | Task (14) Multi-Cast Task is new!
    [01-11-16 4:54:44 pm]  | Task (14) Multi-Cast Task has been cleaned.
    [01-11-16 4:54:44 pm]  | Task (14) /images/Test162016 image file found.
    [01-11-16 4:54:44 pm]  | Task (14) 1 client(s) found.
    [01-11-16 4:54:44 pm]  | Task (14) Multi-Cast Task sending on base port: 50702
    Udp-sender 20120424
    UDP sender for (stdin) at 10.133.132.127 on eth0 
    Broadcasting control to 224.0.0.1
    [01-11-16 4:54:44 pm]  | CMD: cat /images/Test162016/d1p1.img | /usr/local/sbin/udp-sender --interface eth0 --min-receivers 1 --max-wait 600 --mcast-data-address 239.132.20.10 --portbase 50702 --full-duplex --ttl 32 --nokbd --nopointopoint;cat /images/Test162016/d1p2.img | /usr/local/sbin/udp-sender --interface eth0 --min-receivers 1 --max-wait 600 --mcast-data-address 239.132.20.10 --portbase 50702 --full-duplex --ttl 32 --nokbd --nopointopoint;
    [01-11-16 4:54:44 pm]  | Task (14) Multi-Cast Task has started.
    [01-11-16 4:54:54 pm]  | 0 tasks to be cleaned
    [01-11-16 4:54:54 pm]  | 1 task found
    [01-11-16 4:54:54 pm]  | Task (14) Multi-Cast Task is already running PID 27247
    New connection from 10.133.132.242  (#0) 00000009
    Starting transfer: 00000009
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    [01-11-16 4:55:04 pm]  | 0 tasks to be cleaned
    [01-11-16 4:55:04 pm]  | 1 task found
    [01-11-16 4:55:04 pm]  | Task (14) Multi-Cast Task is already running PID 27247
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    [01-11-16 4:55:14 pm]  | 0 tasks to be cleaned
    [01-11-16 4:55:14 pm]  | 1 task found
    [01-11-16 4:55:14 pm]  | Task (14) Multi-Cast Task is already running PID 27247
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    [01-11-16 4:55:24 pm]  | 0 tasks to be cleaned
    [01-11-16 4:55:24 pm]  | 1 task found
    [01-11-16 4:55:24 pm]  | Task (14) Multi-Cast Task is already running PID 27247
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    [01-11-16 4:55:34 pm]  | 0 tasks to be cleaned
    [01-11-16 4:55:34 pm]  | 1 task found
    [01-11-16 4:55:34 pm]  | Task (14) Multi-Cast Task is already running PID 27247
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    [01-11-16 4:55:44 pm]  | 0 tasks to be cleaned
    [01-11-16 4:55:44 pm]  | 1 task found
    [01-11-16 4:55:44 pm]  | Task (14) Multi-Cast Task is already running PID 27247
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
    bytes=     131 040  re-xmits=0000090 (100.0%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=2143
    bytes=  4 925 648  re-xmits=0000285 (  8.4%) slice=0112 -   0
    bytes=  8 441 244  re-xmits=0000408 (  7.0%) slice=0112 -   0
    
    Transfer complete.
    Disconnecting #0 (10.133.132.242)
    
    Udp-sender 20120424
    UDP sender for (stdin) at 10.133.132.127 on eth0 
    Broadcasting control to 224.0.0.1
    [01-11-16 4:55:54 pm]  | 0 tasks to be cleaned
    [01-11-16 4:55:54 pm]  | 1 task found
    [01-11-16 4:55:54 pm]  | Task (14) Multi-Cast Task is already running PID 27247
    New connection from 10.133.132.242  (#0) 00000009
    [01-11-16 4:56:04 pm]  | 0 tasks to be cleaned
    [01-11-16 4:56:04 pm]  | 1 task found
    [01-11-16 4:56:04 pm]  | Task (14) Multi-Cast Task is already running PID 27247
    Starting transfer: 00000009
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1606
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1554
    bytes=  4 976 608  re-xmits=0000206 (  6.0%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=877
    bytes= 33 904 416  re-xmits=0003153 ( 13.5%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=414
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1051
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1001
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=917
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=981
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1439
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1504
    bytes= 51 679 264  re-xmits=0005546 ( 15.6%) slice=0112 -   0
    bytes= 67 660 320  re-xmits=0009744 ( 20.9%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1059
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=732
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=553
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=576
    bytes=102 883 872  re-xmits=0013016 ( 18.4%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=725
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=666
    bytes=133 985 488  re-xmits=0015786 ( 17.1%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=837
    bytes=154 532 560  re-xmits=0020475 ( 19.2%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=678
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=898
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=778
    bytes=161 870 800  re-xmits=0021278 ( 19.1%) slice=0112 -   0
    bytes=170 513 616  re-xmits=0021443 ( 18.3%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=546
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=577
    bytes=181 951 952  re-xmits=0022699 ( 18.1%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=6126
    bytes=203 314 384  re-xmits=0026621 ( 19.0%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=312
    bytes=250 215 056  re-xmits=0027340 ( 15.9%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=415
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=492
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=597
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=543
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=619
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=717
    bytes=279 522 880  re-xmits=0030935 ( 16.1%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=835
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=713
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=600
    bytes=293 384 000  re-xmits=0034010 ( 16.8%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1189
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1052
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=943
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=949
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1505
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1134
    [01-11-16 4:56:14 pm]  | 0 tasks to be cleaned
    [01-11-16 4:56:14 pm]  | 1 task found
    [01-11-16 4:56:14 pm]  | Task (14) Multi-Cast Task is already running PID 27247
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1066
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=959
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1214
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1660
    bytes=302 842 176  re-xmits=0036928 ( 17.7%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1634
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1452
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1422
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1340
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1596
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1744
    bytes=313 278 784  re-xmits=0039658 ( 18.4%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=569
    bytes=333 988 928  re-xmits=0044769 ( 19.5%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=690
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=784
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=640
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=886
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=819
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=807
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=805
    bytes=356 819 008  re-xmits=0049007 ( 19.9%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=816
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=709
    bytes=378 670 656  re-xmits=0053067 ( 20.4%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=614
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=596
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=778
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=878
    bytes=400 848 448  re-xmits=0057995 ( 21.0%) slice=0112 -   0
    bytes=423 189 312  re-xmits=0062372 ( 21.4%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=696
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=924
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=644
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=652
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=695
    bytes=444 225 600  re-xmits=0066687 ( 21.8%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=612
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=647
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=749
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=581
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=544
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=538
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=659
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=546
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=832
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=546
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=540
    bytes=465 588 032  re-xmits=0071328 ( 22.3%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=821
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=810
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=857
    bytes=484 341 312  re-xmits=0075419 ( 22.6%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1287
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1190
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1295
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1136
    bytes=496 897 856  re-xmits=0078113 ( 22.8%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=945
    bytes=518 097 216  re-xmits=0082459 ( 23.1%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=902
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1023
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1383
    bytes=537 665 856  re-xmits=0086766 ( 23.4%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=718
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=752
    bytes=561 637 440  re-xmits=0090529 ( 23.4%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=756
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=796
    bytes=586 587 456  re-xmits=0094555 ( 23.4%) slice=0112 -   0
    bytes=611 048 256  re-xmits=0098563 ( 23.4%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=912
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=668
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=596
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=610
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=716
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=540
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=540
    bytes=636 324 416  re-xmits=0102313 ( 23.4%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=539
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=727
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=927
    bytes=664 535 872  re-xmits=0105315 ( 23.0%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=746
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=499
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=751
    bytes=688 670 528  re-xmits=0109292 ( 23.1%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=406
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=705
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=868
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=753
    bytes=713 131 328  re-xmits=0113039 ( 23.0%) slice=0112 -   0
    [01-11-16 4:56:24 pm]  | 0 tasks to be cleaned
    [01-11-16 4:56:24 pm]  | 1 task found
    [01-11-16 4:56:24 pm]  | Task (14) Multi-Cast Task is already running PID 27247
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=630
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=464
    bytes=741 832 000  re-xmits=0115772 ( 22.7%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=724
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=721
    bytes=770 206 528  re-xmits=0119096 ( 22.5%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=644
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=639
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1026
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=580
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=581
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=504
    bytes=799 722 560  re-xmits=0121997 ( 22.2%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=623
    bytes=826 140 224  re-xmits=0124891 ( 22.0%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=620
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=826
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=948
    bytes=850 601 024  re-xmits=0129301 ( 22.1%) slice=0112 -   0
    bytes=874 409 536  re-xmits=0134118 ( 22.3%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=661
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=560
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=638
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=548
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=667
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=549
    bytes=900 174 912  re-xmits=0138525 ( 22.4%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=450
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=546
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=518
    bytes=939 801 408  re-xmits=0140648 ( 21.7%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=924
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=755
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=767
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1026
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1114
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=723
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=712
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=652
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=595
    bytes=954 640 960  re-xmits=0144344 ( 22.0%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1225
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=978
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1123
    bytes=973 720 384  re-xmits=0147661 ( 22.0%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=523
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=599
    bytes=     984 659K re-xmits=0150993 ( 21.8%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=674
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=531
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=593
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=638
    bytes=  1 012 528K re-xmits=0153893 ( 21.6%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1122
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1069
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=996
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1102
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=742
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=646
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=713
    bytes=  1 031 001K re-xmits=0156927 ( 21.6%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=968
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=951
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1503
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1364
    bytes=  1 043 487K re-xmits=0159664 ( 21.7%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=449
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=886
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=703
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=684
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=660
    bytes=  1 065 598K re-xmits=0162034 ( 21.6%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=567
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=543
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=594
    bytes=  1 110 987K re-xmits=0163125 ( 20.8%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=311
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=357
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=809
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=764
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=759
    bytes=  1 126 912K re-xmits=0164574 ( 20.7%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=472
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1227
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1171
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1080
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1242
    bytes=  1 141 722K re-xmits=0167156 ( 20.8%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1113
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=856
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=718
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=469
    bytes=  1 167 043K re-xmits=0171682 ( 20.9%) slice=0112 -   0
    [01-11-16 4:56:34 pm]  | 0 tasks to be cleaned
    [01-11-16 4:56:34 pm]  | 1 task found
    [01-11-16 4:56:34 pm]  | Task (14) Multi-Cast Task is already running PID 27247
    bytes=  1 201 919K re-xmits=0174903 ( 20.6%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=704
    bytes=  1 225 966K re-xmits=0176769 ( 20.5%) slice=0112 -   0
    bytes=  1 240 457K re-xmits=0178749 ( 20.4%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=866
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=736
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=878
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=935
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=892
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=779
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=748
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1026
    bytes=  1 244 757K re-xmits=0179769 ( 20.5%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=516
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=736
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=913
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=829
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=873
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=939
    bytes=  1 262 593K re-xmits=0183852 ( 20.7%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=651
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=708
    bytes=  1 286 481K re-xmits=0188220 ( 20.8%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=553
    bytes=  1 315 464K re-xmits=0192460 ( 20.8%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=836
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=765
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=524
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=530
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=537
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1177
    bytes=  1 340 148K re-xmits=0196350 ( 20.8%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=666
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=747
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=774
    bytes=  1 351 614K re-xmits=0198288 ( 20.8%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=448
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=851
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=881
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1157
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1428
    bytes=  1 368 335K re-xmits=0201724 ( 20.9%) slice=0112 -   0
    bytes=  1 392 063K re-xmits=0207147 ( 21.1%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=715
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=732
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=607
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=687
    bytes=  1 414 677K re-xmits=0212032 ( 21.3%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1105
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1198
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=892
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=781
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=705
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=748
    bytes=  1 432 513K re-xmits=0215533 ( 21.3%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=409
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=391
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=416
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=434
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=751
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=756
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=601
    bytes=  1 462 770K re-xmits=0219103 ( 21.2%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=832
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=706
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=641
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=649
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=669
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=511
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=880
    bytes=  1 485 702K re-xmits=0223774 ( 21.4%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=727
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=761
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=595
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=599
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=814
    bytes=  1 507 997K re-xmits=0228263 ( 21.5%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=783
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=897
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=893
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=976
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=929
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=971
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=783
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1336
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=924
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1032
    [01-11-16 4:56:44 pm]  | 0 tasks to be cleaned
    [01-11-16 4:56:44 pm]  | 1 task found
    [01-11-16 4:56:44 pm]  | Task (14) Multi-Cast Task is already running PID 27247
    bytes=  1 519 782K re-xmits=0231518 ( 21.6%) slice=0112 -   0
    
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=636
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=596
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=633
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=636
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=607
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=640
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=638
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=690
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=822
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=516
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=601
    Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=998
    bytes=  1 540 803K re-xmits=0236158 ( 21.7%) slice=0112 -   0
    
    
    Transfer complete.
    Disconnecting #0 (10.133.132.242)
    
    [01-11-16 4:57:44 pm]  | 0 tasks to be cleaned
    [01-11-16 4:57:44 pm]  | 1 task found
    [01-11-16 4:57:44 pm]  | Task (14) Multi-Cast Task is no longer running.
    [01-11-16 4:57:44 pm]  | Task (14) Multi-Cast Task has been cleaned as complete.
    [01-11-16 4:57:54 pm]  * No tasks found!
    

  • Moderator

    @kenneth.sisco Make sure that your network switches have igmp forwarding or snooping enabled. If you have vlans in your network the igmp settings are typically set on a per vlan basis. The igmp settings allows the switch ports to subscribe to the multicast channel. The multicast stream will only be sent to ports and switches that subscribe to the multicast request. This keeps multicast traffic off switches where there are no subscribers.



  • I’ll reach out to our network team on Monday, then, and see if they’ll help me setup a test environment. Thanks for the help - I’ll report back soon!


  • Senior Developer

    @kenneth.sisco said:

    sounds like, to me, the switch is not allowing udp broadcast forwarding. This is just a guess at this point.

    I’ve tested Multicast, and all seems to be working fine.

    I’ve also messed a lot, today, with the fog scripts. Hopefully things are better and I’ve been republishing them as I find things and fix them too.

    To me, like Wayne suggested, you should start by imaging within the same switch first (not across vlan’s).



  • @Tom-Elliott said:

    Most times when I see the “This task has been cleaned” directly after it starting, I find it is because of the interface of the network not being set properly for the interface on the system trying to run the multicast job. I will test just to see if things are still good, but I’m nearly certain this is the case.

    You could try running the command you see in the log by itself to see if it spits out any other pertinent information as well.

    I ran the command by hand and didn’t see anything happen. Booting the client to test the multicast didn’t change this. Logs looked exactly the same as what I posted above. Here’s a screenshot of what I saw when I type the command by hand:
    0_1452382198781_mcastss1.PNG

    @Wayne-Workman said:

    @kenneth.sisco Can FOG Multicast on the VLAN that it’s actually within ? We should focus on that first.

    I wish I could test that, but the network team doesn’t allow DHCP to be enabled on server VLANs (or within the data center for that matter). If you know of a way I could do this without DHCP, I’d be happy to try.


  • Senior Developer

    Most times when I see the “This task has been cleaned” directly after it starting, I find it is because of the interface of the network not being set properly for the interface on the system trying to run the multicast job. I will test just to see if things are still good, but I’m nearly certain this is the case.

    You could try running the command you see in the log by itself to see if it spits out any other pertinent information as well.


  • Moderator

    @kenneth.sisco Can FOG Multicast on the VLAN that it’s actually within ? We should focus on that first.


Log in to reply
 

Looks like your connection to FOG Project was lost, please wait while we try to reconnect.