• Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Register
    • Login
    1. Home
    2. kenneth.sisco
    3. Posts
    K
    • Profile
    • Following 0
    • Followers 0
    • Topics 7
    • Posts 37
    • Groups 0

    Posts

    Recent Best Controversial
    • RE: FOGFTP Faile to rename file.

      Updated to the latest trunk release on the SVN and had to run the SQL commands listed here:
      https://forums.fogproject.org/topic/7734/snapin-id-was-not-set-or-unable-to-be-created-svn-revision-5661/27

      After that, it started working great.

      Thanks for the help, guys!

      posted in FOG Problems
      K
      kenneth.sisco
    • RE: FOGFTP Faile to rename file.

      @Sebastian-Roth Here you are!

      '/dev/sda1 on / type ext4 (rw,errors=remount-ro)
      proc on /proc type proc (rw,noexec,nosuid,nodev)
      sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
      none on /sys/fs/cgroup type tmpfs (rw)
      none on /sys/fs/fuse/connections type fusectl (rw)
      none on /sys/kernel/debug type debugfs (rw)
      none on /sys/kernel/security type securityfs (rw)
      udev on /dev type devtmpfs (rw,mode=0755)
      devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
      tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
      none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
      none on /run/shm type tmpfs (rw,nosuid,nodev)
      none on /run/user type tmpfs (rw,noexec,nosuid,nodev,size=104857600,mode=0755)
      none on /sys/fs/pstore type pstore (rw)
      /dev/sdb on /images type ext3 (rw)
      systemd on /sys/fs/cgroup/systemd type cgroup (rw,noexec,nosuid,nodev,none,name=systemd)
      rpc_pipefs on /run/rpc_pipefs type rpc_pipefs (rw)
      nfsd on /proc/fs/nfsd type nfsd (rw)
      
      posted in FOG Problems
      K
      kenneth.sisco
    • RE: FOGFTP Faile to rename file.

      @Sebastian-Roth Yes, I have. I went ahead and did it again just to be safe. Using WinSCP, I can connect via FTP to the server using the standard fog credentials and traverse all directories no problem.

      posted in FOG Problems
      K
      kenneth.sisco
    • RE: FOGFTP Faile to rename file.

      I updated the credentials in the following locations:
      Web Interface -> Storage Management -> Default Member -> Management Username & Management Password
      Web Interface -> FOG Configuration -> FOG Settings -> TFTP Server -> FOG_TFTP_FTP_USERNAME & FOG_TFTP_FTP_PASSWORD
      /opt/fog/.fogsettings - “password” line and “storageftppass” line.
      Local fog user’s password (passwd command).

      I rebooted the server after doing the above, then double-checked all locations to be sure nothing changed. Tried uploading again, and I get the same as the screenshot in my previous post here. It gets hung after “Stopping FOG Status Reporter.”

      The image files are still located on the server under /images/dev/[mac address], so they’re being uploaded, but I definitely made this worse haha.

      Is there a log file that I can check to get more detail?

      posted in FOG Problems
      K
      kenneth.sisco
    • RE: FOGFTP Faile to rename file.

      @Wayne-Workman Good catch - I need to read the manual! 🙂

      I’ll go through and update everywhere as it states, then try again. Thank you!

      posted in FOG Problems
      K
      kenneth.sisco
    • RE: FOGFTP Faile to rename file.

      I feel like I may have made this worse. Now, my upload stops at this point. If I look under /images/dev/[mac address], I do still see all the files there, but the task on the client just gets stuck here.

      The change that I made was to /opt/fog/.fogsettings, and I changed two lines. I still have the default password of “password” for the “fog” account. These two lines were changed:

      password=‘password’;
      storageftppass=‘password’;

      Before, these two lines showed what appeared to be randomly-generated passwords within the quotes. Did I do something wrong?

      alt text

      posted in FOG Problems
      K
      kenneth.sisco
    • RE: FOGFTP Faile to rename file.

      Thank you both for the advice! I checked under .fogsettings and saw that the password was randomly generated, so it was not what I was expecting. I updated that and am doing a test upload now. Will report back with results! 🙂

      posted in FOG Problems
      K
      kenneth.sisco
    • FOGFTP Faile to rename file.

      I’m running FOG Trunk version 5864 (which I’m sure is a bit outdated, but it’s been working really well for the most part).

      Recently, I had to add a hard drive to my fog server for storing images. I don’t use storage nodes - this is all on one single server. I added a hard drive, formatted it, and mounted it as /images. All my images are stored there no problem, but whenever I upload an image, I get this error message:

      FOGFTP: Failed to rename file. Remote Path: /images/[image name], Local Path: /images/dev/[mac address], Error: ftp_rename(): Permission Denied.

      I’ve done a recursive chmod to change permissions on /images to 777 and also used chgrp and chown to change ownership to “fog.” Not sure what else I can try to get these permissions to work.

      My current workaround is to log into the server manually and do this command:
      sudo mv /images/dev/[mac address] /images/[image name]

      This solves my problem, but I’d like the upload to work on its own. :S

      Any ideas?

      posted in FOG Problems
      K
      kenneth.sisco
    • RE: Multicasting Issues

      Cool - issue resolved!

      This was absolutely an issue on our network, but we’re not sure if it was an old static Rendezvous Point that was configured on our core or if it was our VPC redundancy between our two nexus switches, but we were able to solve it by disabling pim on one of the nexuses and removing that static RP.

      Thanks for the help on investigating, everyone. It’s much appreciated.

      posted in FOG Problems
      K
      kenneth.sisco
    • RE: Multicasting Issues

      @Sebastian-Roth Yes, we’ve looked at IGMP, but our topology is fairly complex. Currently the network team is working on moving rendezvous points closer to the server/clients and investigating potential issues in NXOS, as we recently migrated from Cisco IOS cores to Nexus.

      Unfortunately, my visibility on this is very limited, so I’m doing my best to keep this rolling on the server side as we troubleshoot.

      I’ll update with more info later as we progress.

      posted in FOG Problems
      K
      kenneth.sisco
    • RE: Multicasting Issues

      We’re still having trouble narrowing this down, but is it safe to assume that if Multicast is working across the same VLAN that it’s no longer a fog issue?

      posted in FOG Problems
      K
      kenneth.sisco
    • RE: Multicasting Issues

      So I’m working with our network team to troubleshoot this, and we have “broadcast forwarding” working for the most part, but we’re going to keep digging. One thing we were concerned about was TTL on the client side. We can see the TTL from the server in the multicast.log, but we’re concerned about what the TTL is on the udp-receiver command. Is there a way to see that? Or a way to specify it? I was looking for the task files that 0.32 used to have stored in the tftpboot folder, but this version of Fog handles that differently, so I’m not sure where to find them.

      posted in FOG Problems
      K
      kenneth.sisco
    • RE: Multicasting Issues

      So I worked with our network team and we were able to setup a small DHCP pool on the server VLAN temporarily for a test, and it worked fine. Once we tried again cross-VLAN, we had the same issues. I’ve copied the multicast.log section of the successful multicast below. The unsuccessful one is the same as my prior post in this thread. What I find interesting is that the multicast.log file shows that there is a new connection from the client cross-VLAN and even echoes its IP Address, but never transfers any data.

      Successful multicast (same VLAN; shortened to allow for forum posting):

      [01-11-16 4:54:24 pm]  * No tasks found!
      [01-11-16 4:54:34 pm]  | Sleeping for 10 seconds to ensure tasks are properly submitted
      [01-11-16 4:54:44 pm]  | 0 tasks to be cleaned
      [01-11-16 4:54:44 pm]  | 1 task found
      [01-11-16 4:54:44 pm]  | Task (14) Multi-Cast Task is new!
      [01-11-16 4:54:44 pm]  | Task (14) Multi-Cast Task has been cleaned.
      [01-11-16 4:54:44 pm]  | Task (14) /images/Test162016 image file found.
      [01-11-16 4:54:44 pm]  | Task (14) 1 client(s) found.
      [01-11-16 4:54:44 pm]  | Task (14) Multi-Cast Task sending on base port: 50702
      Udp-sender 20120424
      UDP sender for (stdin) at 10.133.132.127 on eth0 
      Broadcasting control to 224.0.0.1
      [01-11-16 4:54:44 pm]  | CMD: cat /images/Test162016/d1p1.img | /usr/local/sbin/udp-sender --interface eth0 --min-receivers 1 --max-wait 600 --mcast-data-address 239.132.20.10 --portbase 50702 --full-duplex --ttl 32 --nokbd --nopointopoint;cat /images/Test162016/d1p2.img | /usr/local/sbin/udp-sender --interface eth0 --min-receivers 1 --max-wait 600 --mcast-data-address 239.132.20.10 --portbase 50702 --full-duplex --ttl 32 --nokbd --nopointopoint;
      [01-11-16 4:54:44 pm]  | Task (14) Multi-Cast Task has started.
      [01-11-16 4:54:54 pm]  | 0 tasks to be cleaned
      [01-11-16 4:54:54 pm]  | 1 task found
      [01-11-16 4:54:54 pm]  | Task (14) Multi-Cast Task is already running PID 27247
      New connection from 10.133.132.242  (#0) 00000009
      Starting transfer: 00000009
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      [01-11-16 4:55:04 pm]  | 0 tasks to be cleaned
      [01-11-16 4:55:04 pm]  | 1 task found
      [01-11-16 4:55:04 pm]  | Task (14) Multi-Cast Task is already running PID 27247
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      [01-11-16 4:55:14 pm]  | 0 tasks to be cleaned
      [01-11-16 4:55:14 pm]  | 1 task found
      [01-11-16 4:55:14 pm]  | Task (14) Multi-Cast Task is already running PID 27247
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      [01-11-16 4:55:24 pm]  | 0 tasks to be cleaned
      [01-11-16 4:55:24 pm]  | 1 task found
      [01-11-16 4:55:24 pm]  | Task (14) Multi-Cast Task is already running PID 27247
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      [01-11-16 4:55:34 pm]  | 0 tasks to be cleaned
      [01-11-16 4:55:34 pm]  | 1 task found
      [01-11-16 4:55:34 pm]  | Task (14) Multi-Cast Task is already running PID 27247
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      [01-11-16 4:55:44 pm]  | 0 tasks to be cleaned
      [01-11-16 4:55:44 pm]  | 1 task found
      [01-11-16 4:55:44 pm]  | Task (14) Multi-Cast Task is already running PID 27247
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      bytes=     131 040  re-xmits=0000090 (100.0%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=2143
      bytes=  4 925 648  re-xmits=0000285 (  8.4%) slice=0112 -   0
      bytes=  8 441 244  re-xmits=0000408 (  7.0%) slice=0112 -   0
      
      Transfer complete.
      Disconnecting #0 (10.133.132.242)
      
      Udp-sender 20120424
      UDP sender for (stdin) at 10.133.132.127 on eth0 
      Broadcasting control to 224.0.0.1
      [01-11-16 4:55:54 pm]  | 0 tasks to be cleaned
      [01-11-16 4:55:54 pm]  | 1 task found
      [01-11-16 4:55:54 pm]  | Task (14) Multi-Cast Task is already running PID 27247
      New connection from 10.133.132.242  (#0) 00000009
      [01-11-16 4:56:04 pm]  | 0 tasks to be cleaned
      [01-11-16 4:56:04 pm]  | 1 task found
      [01-11-16 4:56:04 pm]  | Task (14) Multi-Cast Task is already running PID 27247
      Starting transfer: 00000009
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1606
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1554
      bytes=  4 976 608  re-xmits=0000206 (  6.0%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=877
      bytes= 33 904 416  re-xmits=0003153 ( 13.5%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=414
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1051
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1001
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=917
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=981
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1439
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1504
      bytes= 51 679 264  re-xmits=0005546 ( 15.6%) slice=0112 -   0
      bytes= 67 660 320  re-xmits=0009744 ( 20.9%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1059
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=732
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=553
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=576
      bytes=102 883 872  re-xmits=0013016 ( 18.4%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=725
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=666
      bytes=133 985 488  re-xmits=0015786 ( 17.1%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=837
      bytes=154 532 560  re-xmits=0020475 ( 19.2%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=678
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=898
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=778
      bytes=161 870 800  re-xmits=0021278 ( 19.1%) slice=0112 -   0
      bytes=170 513 616  re-xmits=0021443 ( 18.3%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=546
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=577
      bytes=181 951 952  re-xmits=0022699 ( 18.1%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=6126
      bytes=203 314 384  re-xmits=0026621 ( 19.0%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=312
      bytes=250 215 056  re-xmits=0027340 ( 15.9%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=415
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=492
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=597
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=543
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=619
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=717
      bytes=279 522 880  re-xmits=0030935 ( 16.1%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=835
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=713
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=600
      bytes=293 384 000  re-xmits=0034010 ( 16.8%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1189
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1052
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=943
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=949
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1505
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1134
      [01-11-16 4:56:14 pm]  | 0 tasks to be cleaned
      [01-11-16 4:56:14 pm]  | 1 task found
      [01-11-16 4:56:14 pm]  | Task (14) Multi-Cast Task is already running PID 27247
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1066
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=959
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1214
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1660
      bytes=302 842 176  re-xmits=0036928 ( 17.7%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1634
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1452
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1422
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1340
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1596
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1744
      bytes=313 278 784  re-xmits=0039658 ( 18.4%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=569
      bytes=333 988 928  re-xmits=0044769 ( 19.5%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=690
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=784
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=640
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=886
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=819
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=807
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=805
      bytes=356 819 008  re-xmits=0049007 ( 19.9%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=816
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=709
      bytes=378 670 656  re-xmits=0053067 ( 20.4%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=614
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=596
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=778
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=878
      bytes=400 848 448  re-xmits=0057995 ( 21.0%) slice=0112 -   0
      bytes=423 189 312  re-xmits=0062372 ( 21.4%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=696
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=924
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=644
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=652
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=695
      bytes=444 225 600  re-xmits=0066687 ( 21.8%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=612
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=647
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=749
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=581
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=544
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=538
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=659
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=546
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=832
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=546
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=540
      bytes=465 588 032  re-xmits=0071328 ( 22.3%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=821
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=810
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=857
      bytes=484 341 312  re-xmits=0075419 ( 22.6%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1287
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1190
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1295
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1136
      bytes=496 897 856  re-xmits=0078113 ( 22.8%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=945
      bytes=518 097 216  re-xmits=0082459 ( 23.1%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=902
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1023
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1383
      bytes=537 665 856  re-xmits=0086766 ( 23.4%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=718
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=752
      bytes=561 637 440  re-xmits=0090529 ( 23.4%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=756
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=796
      bytes=586 587 456  re-xmits=0094555 ( 23.4%) slice=0112 -   0
      bytes=611 048 256  re-xmits=0098563 ( 23.4%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=912
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=668
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=596
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=610
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=716
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=540
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=540
      bytes=636 324 416  re-xmits=0102313 ( 23.4%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=539
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=727
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=927
      bytes=664 535 872  re-xmits=0105315 ( 23.0%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=746
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=499
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=751
      bytes=688 670 528  re-xmits=0109292 ( 23.1%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=406
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=705
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=868
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=753
      bytes=713 131 328  re-xmits=0113039 ( 23.0%) slice=0112 -   0
      [01-11-16 4:56:24 pm]  | 0 tasks to be cleaned
      [01-11-16 4:56:24 pm]  | 1 task found
      [01-11-16 4:56:24 pm]  | Task (14) Multi-Cast Task is already running PID 27247
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=630
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=464
      bytes=741 832 000  re-xmits=0115772 ( 22.7%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=724
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=721
      bytes=770 206 528  re-xmits=0119096 ( 22.5%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=644
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=639
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1026
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=580
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=581
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=504
      bytes=799 722 560  re-xmits=0121997 ( 22.2%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=623
      bytes=826 140 224  re-xmits=0124891 ( 22.0%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=620
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=826
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=948
      bytes=850 601 024  re-xmits=0129301 ( 22.1%) slice=0112 -   0
      bytes=874 409 536  re-xmits=0134118 ( 22.3%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=661
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=560
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=638
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=548
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=667
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=549
      bytes=900 174 912  re-xmits=0138525 ( 22.4%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=450
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=546
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=518
      bytes=939 801 408  re-xmits=0140648 ( 21.7%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=924
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=755
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=767
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1026
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1114
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=723
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=712
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=652
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=595
      bytes=954 640 960  re-xmits=0144344 ( 22.0%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1225
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=978
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1123
      bytes=973 720 384  re-xmits=0147661 ( 22.0%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=523
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=599
      bytes=     984 659K re-xmits=0150993 ( 21.8%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=674
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=531
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=593
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=638
      bytes=  1 012 528K re-xmits=0153893 ( 21.6%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1122
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1069
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=996
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1102
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=742
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=646
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=713
      bytes=  1 031 001K re-xmits=0156927 ( 21.6%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=968
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=951
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1503
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1364
      bytes=  1 043 487K re-xmits=0159664 ( 21.7%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=449
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=886
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=703
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=684
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=660
      bytes=  1 065 598K re-xmits=0162034 ( 21.6%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=567
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=543
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=594
      bytes=  1 110 987K re-xmits=0163125 ( 20.8%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=311
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=357
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=809
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=764
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=759
      bytes=  1 126 912K re-xmits=0164574 ( 20.7%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=472
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1227
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1171
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1080
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1242
      bytes=  1 141 722K re-xmits=0167156 ( 20.8%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1113
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=856
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=718
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=469
      bytes=  1 167 043K re-xmits=0171682 ( 20.9%) slice=0112 -   0
      [01-11-16 4:56:34 pm]  | 0 tasks to be cleaned
      [01-11-16 4:56:34 pm]  | 1 task found
      [01-11-16 4:56:34 pm]  | Task (14) Multi-Cast Task is already running PID 27247
      bytes=  1 201 919K re-xmits=0174903 ( 20.6%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=704
      bytes=  1 225 966K re-xmits=0176769 ( 20.5%) slice=0112 -   0
      bytes=  1 240 457K re-xmits=0178749 ( 20.4%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=866
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=736
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=878
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=935
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=892
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=779
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=748
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1026
      bytes=  1 244 757K re-xmits=0179769 ( 20.5%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=516
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=736
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=913
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=829
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=873
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=939
      bytes=  1 262 593K re-xmits=0183852 ( 20.7%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=651
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=708
      bytes=  1 286 481K re-xmits=0188220 ( 20.8%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=553
      bytes=  1 315 464K re-xmits=0192460 ( 20.8%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=836
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=765
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=524
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=530
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=537
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1177
      bytes=  1 340 148K re-xmits=0196350 ( 20.8%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=666
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=747
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=774
      bytes=  1 351 614K re-xmits=0198288 ( 20.8%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=448
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=851
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=881
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1157
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1428
      bytes=  1 368 335K re-xmits=0201724 ( 20.9%) slice=0112 -   0
      bytes=  1 392 063K re-xmits=0207147 ( 21.1%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=715
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=732
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=607
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=687
      bytes=  1 414 677K re-xmits=0212032 ( 21.3%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1105
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1198
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=892
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=781
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=705
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=748
      bytes=  1 432 513K re-xmits=0215533 ( 21.3%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=409
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=391
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=416
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=434
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=751
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=756
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=601
      bytes=  1 462 770K re-xmits=0219103 ( 21.2%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=832
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=706
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=641
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=649
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=669
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=511
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=880
      bytes=  1 485 702K re-xmits=0223774 ( 21.4%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=727
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=761
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=595
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=599
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=814
      bytes=  1 507 997K re-xmits=0228263 ( 21.5%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=783
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=897
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=893
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=976
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=929
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=971
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=783
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1336
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=924
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=1032
      [01-11-16 4:56:44 pm]  | 0 tasks to be cleaned
      [01-11-16 4:56:44 pm]  | 1 task found
      [01-11-16 4:56:44 pm]  | Task (14) Multi-Cast Task is already running PID 27247
      bytes=  1 519 782K re-xmits=0231518 ( 21.6%) slice=0112 -   0
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=636
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=596
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=633
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=636
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=607
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=640
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=638
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=690
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=822
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=516
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=601
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=998
      bytes=  1 540 803K re-xmits=0236158 ( 21.7%) slice=0112 -   0
      
      
      Transfer complete.
      Disconnecting #0 (10.133.132.242)
      
      [01-11-16 4:57:44 pm]  | 0 tasks to be cleaned
      [01-11-16 4:57:44 pm]  | 1 task found
      [01-11-16 4:57:44 pm]  | Task (14) Multi-Cast Task is no longer running.
      [01-11-16 4:57:44 pm]  | Task (14) Multi-Cast Task has been cleaned as complete.
      [01-11-16 4:57:54 pm]  * No tasks found!
      
      posted in FOG Problems
      K
      kenneth.sisco
    • RE: Multicasting Issues

      I’ll reach out to our network team on Monday, then, and see if they’ll help me setup a test environment. Thanks for the help - I’ll report back soon!

      posted in FOG Problems
      K
      kenneth.sisco
    • RE: Multicasting Issues

      @Tom-Elliott said:

      Most times when I see the “This task has been cleaned” directly after it starting, I find it is because of the interface of the network not being set properly for the interface on the system trying to run the multicast job. I will test just to see if things are still good, but I’m nearly certain this is the case.

      You could try running the command you see in the log by itself to see if it spits out any other pertinent information as well.

      I ran the command by hand and didn’t see anything happen. Booting the client to test the multicast didn’t change this. Logs looked exactly the same as what I posted above. Here’s a screenshot of what I saw when I type the command by hand:
      0_1452382198781_mcastss1.PNG

      @Wayne-Workman said:

      @kenneth.sisco Can FOG Multicast on the VLAN that it’s actually within ? We should focus on that first.

      I wish I could test that, but the network team doesn’t allow DHCP to be enabled on server VLANs (or within the data center for that matter). If you know of a way I could do this without DHCP, I’d be happy to try.

      posted in FOG Problems
      K
      kenneth.sisco
    • Multicasting Issues

      I’m having a lot of trouble getting multicast working in my environment. I work for a University where our networks are all segmented into multiple VLANs, and I don’t have the ability to put FOG on the same VLAN as the clients - we have about 10 computer labs and our FOG server is a VMWare virtual machine in our data center. Each computer lab is on its own VLAN, and the data center is as well.

      Our network team has a lot of controls on the network that I cannot see, but they assigned me a multicast range of 239.132.20.0 /24. Since this doesn’t match the default that FOG uses, I upgraded to the Trunk version of FOG to allow me to customize that. The network team setup a test multicast node within that multicast range and ran several small tests from their utilities across all of my VLANs and have had no hiccups at all, leading them to believe that the problem lies on the FOG server somewhere.

      Here’s where I’m at on the server:
      OS: Ubuntu Server 14.04.3 LTS
      FOG Trunk revision: 4623

      On the FOG server, I have a small 8GB image for testing. Under FOG Configuration -> FOG Settings -> Multicast Settings, I have configured the following:

      FOG_UDPCAST_INTERFACE: eth0
      FOG_UDPCAST_STARTINGPORT: 63072
      FOG_MULTICAST_MAX_SESSIONS: 64
      FOG_UDPCAST_MAXWAIT	: 10
      FOG_MULTICAST_ADDRESS: 239.132.20.10
      FOG_MULTICAST_PORT_OVERRIDE: 0
      FOG_MULTICAST_DUPLEX: FULL_DUPLEX
      

      I have registered two test hosts, and am attempting to get one working before trying two. I’ve double-checked under Active Tasks and Active Multicast tasks to ensure nothing is currently running. I’ve rebooted the server and restarted FOGMulticastManager, FOGImageReplicator, and FOGScheduler services (in case they didn’t start properly).

      When trying to multicast to a single client, I get inconsistent results. Most of the time, the client gets stuck at “Starting to restore image ( - ) to device (/dev/sda1).” Other times, it’ll get past that point and show a status bar, but the bar will be stuck at 0% with a transfer rate of 0.00bytes/min. The latter of which is incredibly difficult for me to replicate, I’ve found. Most of the time, I’m stuck at the former.

      Here’s what the multicast.log looks like:

      [01-09-16 6:31:41 pm] Interface Ready with IP Address: 10.133.132.127
      
      [01-09-16 6:31:41 pm]  * Starting MulticastManager Service
      
      [01-09-16 6:31:41 pm]  * Checking for new items every 10 seconds
      
      [01-09-16 6:31:41 pm]  * Starting service loop
      
      [01-09-16 6:31:41 pm]  * No tasks found!
      [01-09-16 6:31:51 pm]  * No tasks found!
      
      [01-09-16 6:32:01 pm]  * No tasks found!
      
      [01-09-16 6:32:11 pm]  | Sleeping for 10 seconds to ensure tasks are properly submitted
      
      [01-09-16 6:32:21 pm]  | 0 tasks to be cleaned
      [01-09-16 6:32:21 pm]  | 1 task found
      
      [01-09-16 6:32:21 pm]  | Task (11) Multi-Cast Task is new!
      
      [01-09-16 6:32:21 pm]  | Task (11) Multi-Cast Task has been cleaned.
      
      [01-09-16 6:32:21 pm]  | Task (11) /images/Test162016 image file found.
      
      [01-09-16 6:32:21 pm]  | Task (11) 1 client(s) found.
      Udp-sender 20120424
      
      [01-09-16 6:32:21 pm]  | Task (11) Multi-Cast Task sending on base port: 64512
      UDP sender for (stdin) at 10.133.132.127 on eth0 
      Broadcasting control to 224.0.0.1
      [01-09-16 6:32:21 pm]  | CMD: cat /images/Test162016/d1p1.img | /usr/local/sbin/udp-sender --interface eth0 --min-receivers 1 --max-wait 600 --mcast-data-address 239.132.20.10 --portbase 64512 --full-duplex --ttl 32 --nokbd --nopointopoint;cat /images/Test162016/d1p2.img | /usr/local/sbin/udp-sender --interface eth0 --min-receivers 1 --max-wait 600 --mcast-data-address 239.132.20.10 --portbase 64512 --full-duplex --ttl 32 --nokbd --nopointopoint;
      [01-09-16 6:32:21 pm]  | Task (11) Multi-Cast Task has started.
      
      [01-09-16 6:32:31 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:32:31 pm]  | 1 task found
      
      [01-09-16 6:32:31 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      [01-09-16 6:32:41 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:32:41 pm]  | 1 task found
      
      [01-09-16 6:32:41 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      [01-09-16 6:32:51 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:32:51 pm]  | 1 task found
      
      [01-09-16 6:32:51 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      [01-09-16 6:33:01 pm]  | 0 tasks to be cleaned
      [01-09-16 6:33:01 pm]  | 1 task found
      
      [01-09-16 6:33:01 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      [01-09-16 6:33:11 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:33:11 pm]  | 1 task found
      
      [01-09-16 6:33:11 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      [01-09-16 6:33:21 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:33:21 pm]  | 1 task found
      
      [01-09-16 6:33:21 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      New connection from 10.20.129.47  (#0) 00000009
      Starting transfer: 00000009
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:33:31 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:33:31 pm]  | 1 task found
      
      [01-09-16 6:33:31 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:33:41 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:33:41 pm]  | 1 task found
      
      [01-09-16 6:33:41 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:33:51 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:33:51 pm]  | 1 task found
      
      [01-09-16 6:33:51 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:34:02 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:34:02 pm]  | 1 task found
      
      [01-09-16 6:34:02 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:34:12 pm]  | 0 tasks to be cleaned
      [01-09-16 6:34:12 pm]  | 1 task found
      
      [01-09-16 6:34:12 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:34:22 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:34:22 pm]  | 1 task found
      
      [01-09-16 6:34:22 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:34:32 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:34:32 pm]  | 1 task found
      
      [01-09-16 6:34:32 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:34:42 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:34:42 pm]  | 1 task found
      
      [01-09-16 6:34:42 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:34:52 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:34:52 pm]  | 1 task found
      
      [01-09-16 6:34:52 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:35:02 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:35:02 pm]  | 1 task found
      
      [01-09-16 6:35:02 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:35:12 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:35:12 pm]  | 1 task found
      
      [01-09-16 6:35:12 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:35:22 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:35:22 pm]  | 1 task found
      
      [01-09-16 6:35:22 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:35:32 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:35:32 pm]  | 1 task found
      
      [01-09-16 6:35:32 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:35:42 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:35:42 pm]  | 1 task found
      
      [01-09-16 6:35:42 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:35:52 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:35:52 pm]  | 1 task found
      
      [01-09-16 6:35:52 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:36:02 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:36:02 pm]  | 1 task found
      
      [01-09-16 6:36:02 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:36:12 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:36:12 pm]  | 1 task found
      
      [01-09-16 6:36:12 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:36:22 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:36:22 pm]  | 1 task found
      
      [01-09-16 6:36:22 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:36:32 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:36:32 pm]  | 1 task found
      
      [01-09-16 6:36:32 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      Timeout notAnswered=[0] notReady=[0] nrAns=0 nrRead=0 nrPart=1 avg=10000
      
      [01-09-16 6:36:42 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:36:42 pm]  | 1 task found
      
      [01-09-16 6:36:42 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      Dropping client #0 because of timeout
      Disconnecting #0 (10.20.129.47)
      bytes=                re-xmits=0000000 (  0.0%) slice=0112 -   0
      Transfer complete.
      
      cat: write error: Broken pipe
      Udp-sender 20120424
      UDP sender for (stdin) at 10.133.132.127 on eth0 
      Broadcasting control to 224.0.0.1
      
      [01-09-16 6:36:52 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:36:52 pm]  | 1 task found
      
      [01-09-16 6:36:52 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      [01-09-16 6:37:02 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:37:02 pm]  | 1 task found
      
      [01-09-16 6:37:02 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      [01-09-16 6:37:12 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:37:12 pm]  | 1 task found
      
      [01-09-16 6:37:12 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      [01-09-16 6:37:22 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:37:22 pm]  | 1 task found
      
      [01-09-16 6:37:22 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      [01-09-16 6:37:32 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:37:32 pm]  | 1 task found
      
      [01-09-16 6:37:32 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      [01-09-16 6:37:42 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:37:42 pm]  | 1 task found
      
      [01-09-16 6:37:42 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      [01-09-16 6:37:52 pm]  | 0 tasks to be cleaned
      
      [01-09-16 6:37:52 pm]  | 1 task found
      
      [01-09-16 6:37:52 pm]  | Task (11) Multi-Cast Task is already running PID 3793
      
      
      

      I’m not sure where to go next in my troubleshooting. Any help?

      posted in FOG Problems
      K
      kenneth.sisco
    • Configure Multicast Address Fog 1.2.0

      My network team has asked me to customize our multicast settings to use a specific network address. Looking at the wiki, it states that under Fog Settings -> Multicast Settings, I can configure this. But since I installed 1.2.0, I only see options for the Interface, Starting Port, and Max Sessions.

      How can I customize other Multicast settings in 1.2.0?

      Thanks in advance.

      posted in FOG Problems
      K
      kenneth.sisco
    • 1
    • 2
    • 2 / 2