SVN 2979 multicast issues
-
Just my 2 cents on Multicasting issues. I use fog to image two labs at SCSU. Multicast never would work. Last week was spring break I had the time to run a couple of tests. I followed all the instructions to test multicast on the forums but multicast deploy would hang on the blue screen and that was that. I do not have administrative access to the production switches.
Images are complex with ubunutu with many VMs in the images. Image type; RAW, compression is set to 3. Client hardware; HP Thin Clients with 8 GB of memory and 126GB SSD.
Fog Server Dell 6GB. I do have a HP on the way in a month that will have 64GB to test at the end of spring semester so I may be able to speed thing up a bit.
I decided to swap out, (our office of information technology Cisco managed switches) with Cisco SBM switches out of the “box”.
Multicast in Fog works just fine.I tested the following:
Multicast 8 HP Thin Clients on Cisco SMB Switches - average image 22 minutes.
Multicast 15 HP Thin Clients on Cisco SMB Switches - average image 26 minutes.
Unicast 8 HP Thin Clients on Cisco SMB Switches - average image 1hr 26 minutes.
Unicast 15 HP Thin Clients on Cisco SMB Switches - average image 2hr 54 minutes.Unicast to 15 Thin Clients on production switches to 15 systems would vary from 2 to 4 hours.
I do multi partition non resizable NTFS 40GB images and this averages 24 minutes unicast. I did not have time to test this in multicast.
So multicast does work and for larger deployments is much faster. Performance is 23 minutes vs 1hr and 26 minutes. I have sent the fog logs to our OIT group to have them look into their Cisco’s. I am planning to replace the current production switches . I would suggest to add a line to the testing of multicast to just swap out any old switches.
-
I think one of the biggest core-problems with I.T. support people multicasting is being unable to access their switches / routers.
These dumb barriers between network teams and I.T. support are created. A technician will say “Multicast isn’t working”, network admin says “It’s set up, you must not be doing it right”… vicious circle. And that’s if they even BOTHER to help you… Either person could be wrong, honestly. But, without one being able to see the other’s setup (or communicate really well), it’s pretty pointless to even try to troubleshoot.
I despise things like this.
-
[quote=“Wayne Workman, post: 44225, member: 28155”]I think one of the biggest core-problems with I.T. support people multicasting is being unable to access their switches / routers.
These dumb barriers between network teams and I.T. support are created. A technician will say “Multicast isn’t working”, network admin says “It’s set up, you must not be doing it right”… vicious circle. And that’s if they even BOTHER to help you… Either person could be wrong, honestly. But, without one being able to see the other’s setup (or communicate really well), it’s pretty pointless to even try to troubleshoot.
I despise things like this.[/quote]
After being told by our network group that multicast was enabled on the production switches on Monday, we found out yesterday that only one of the two switches supports layer three. The ports in the labs are randomly patched into the switches, no documentation. Wire rack is locked, etc…
Ran another test this morning. 15 thin clients using production switches. Start the multicast, all systems pxe and sitting on gray screen. Multicast not starting. Go into the task list the ones in the group that are ready are patched to level three multicast. The ones that are still connecting were patched to the layer two switch. Made bets with network guys. Killed task. Created a new group with thin client members that were waiting in the first job (six computers). Started multicast, executes perfectly. I’m very lucky to have had the fog server randomly patched into the switch with layer thee capabilities. I had given up on multicast last summer.
Four years I’ve been using FOG now, always hear it from the networking group that there is nothing wrong. But they put in all into emails so now it is documented. Getting my own switches this summer.
126GB in 23 minutes is just shy of the 20 minutes (the fastest it could go), the only limitation for the multicast was the GB network cards.
Now if I can figure out the TFTP Boot errors…
-
[quote=“Steven B, post: 44546, member: 24174”]After being told by our network group that multicast was enabled on the production switches on Monday, we found out yesterday that only one of the two switches supports layer three. The ports in the labs are randomly patched into the switches, no documentation. Wire rack is locked, etc…
Ran another test this morning. 15 thin clients using production switches. Start the multicast, all systems pxe and sitting on gray screen. Multicast not starting. Go into the task list the ones in the group that are ready are patched to level three multicast. The ones that are still connecting were patched to the layer two switch. Made bets with network guys. Killed task. Created a new group with thin client members that were waiting in the first job (six computers). Started multicast, executes perfectly. I’m very lucky to have had the fog server randomly patched into the switch with layer thee capabilities. I had given up on multicast last summer.
Four years I’ve been using FOG now, always hear it from the networking group that there is nothing wrong. But they put in all into emails so now it is documented. Getting my own switches this summer.
126GB in 23 minutes is just shy of the 20 minutes (the fastest it could go), the only limitation for the multicast was the GB network cards.
Now if I can figure out the TFTP Boot errors…[/quote]
Well, that’s a big jump from where you were before.
We should be able to help with the TFTP errors. I’ll read through this thread again and update this post if I think of anything.
Just read through the thread again. Didn’t see anything about TFTP.
You’ll need to post the errors you’re seeing. A picture of them would be fine, too. -
It’s been a while since I last posted about this but me and the network team has made some headway but we hit another wall. In FOG we had 2 machines in debug mode for the udp-receive and we were able to multicast a log file successfully. However whenever we tried an image file we would always the the result: Timeout notAnswered=[0,1] notReady=[0,1]
Does anybody know why we were able to do a log file but not an image file? -
Where was the log file located? Where on the FOG server?
Is it in a different place than /images ? -
It was the multicast.log file located in /opt/fog/log.
-
Make sure NFS service doesn’t have any errors…
[CODE]sudo service nfs-kernel-server status[/CODE]Make sure the FTP service doesn’t have any errors…
[CODE]sudo service vsftpd status[/CODE]You can restart the FTP service like this:
[CODE]sudo service vsftpd restart[/CODE]Check what’s allowed in/out of your firewall:
[CODE]iptables -L[/CODE]And, here’s some stuff Tom posted earlier here. I think you should try these commands again to see if it helps.
[CODE]sudo service FOGMulticastManager stop
sudo service FOGImageReplicator stop
sudo service FOGScheduler stop
sudo service FOGMulticastManager start
sudo service FOGImageReplicator start
sudo service FOGScheduler start[/CODE]#LetsMakeScripts
-
We have tried the stop and starting of the services many times with no real changes. Here is what my iptables list says:
Chain INPUT (policy ACCEPT)
target prot opt source destinationChain FORWARD (policy ACCEPT)
target prot opt source destinationChain OUTPUT (policy ACCEPT)
target prot opt source Destination -
Maybe upgrade to the latest SVN??
-
I just upgraded to 3180 and it still doesn’t work. Unicast works fine but is there some specific folder permissions that need to be in place for mulicast to work on the server. I doubt there is but I’m running out of ideas.
-
Here is exactly what we used to test the connections:
root@fogVM:/images/DellOptiplex3020syspreptest# udp-sender --file /opt/fog/log/multicast.log --ttl 32 --mcast-data-address 239.168.1.1 --min-receivers 2
Udp-sender 20120424
Using full duplex mode
UDP sender for /opt/fog/log/multicast.log at 172.28.2.21 on eth0
Broadcasting control to 224.0.0.1
New connection from 172.28.16.36 (#0) 00000009
Ready. Press any key to start sending data.
New connection from 172.28.16.49 (#1) 00000009
Ready. Press any key to start sending data.
Starting transfer: 00000009
bytes= 2 200 re-xmits=0000001 ( 50.0%) slice=0112 - 0
Transfer complete.
Disconnecting #0 (172.28.16.36)
Disconnecting #1 (172.28.16.49)Here is the one for the image:
root@fogVM:/images/DellOptiplex3020syspreptest# gunzip -c “/images/DellOptiplex3020syspreptest/d1p1.img” | /usr/local/sbin/udp-sender --min-receivers 2 --mcast-data-address 239.168.1.1 --portbase 9000 --interface 172.28.2.21 --full-duplex --ttl 32
Udp-sender 20120424
UDP sender for (stdin) at 172.28.2.21 on eth0
Broadcasting control to 224.0.0.1
New connection from 172.28.16.36 (#0) 00000009
Ready. Press any key to start sending data.
New connection from 172.28.16.49 (#1) 00000009
Ready. Press any key to start sending data.
Starting transfer: 00000009
Timeout notAnswered=[0,1] notReady=[0,1] nrAns=0 nrRead=0 nrPart=2 avg=6696
Timeout notAnswered=[0,1] notReady=[0,1] nrAns=0 nrRead=0 nrPart=2 avg=6696
Timeout notAnswered=[0,1] notReady=[0,1] nrAns=0 nrRead=0 nrPart=2 avg=6696
Timeout notAnswered=[0,1] notReady=[0,1] nrAns=0 nrRead=0 nrPart=2 avg=6696
