Very slow unicast deploy when imaging 2 or more machines

[george1421]

@Haz In reference to Tom’s post. Was the image recorded with FOG 1.2.0 or newer or is this an old 0.3x image? The newer version of FOG and new captured image with 1.2.0+ will net better results than and older captured image with newer FOG.

If you have the option (if you only have a single disk) use an SSD instead of a HDD. The cost is trivial and the speed is great. Your specs on the 380 is sufficient, just the HDD is in question (in my mind only). FWIW I was able to deploy ~4.5GB/m for a dual unicast deployment using a Dual Core celeron in an intel nuc running off an SSD. So I know its possible to get pretty good speeds out of a low end system. The fog server itself is not really taxed much at all during deployment. The only thing the fog server does during a deployment is move data from disk to the network. The target computer does all of the heavy lifting during imaging.

[Wayne Workman]

@george1421 said in Very slow unicast deploy when imaging 2 or more machines:

I’m personally a bit suspicious of the 9GB/min simply because the theoretical maximum for a single GbE link is 7.5GB/min.

That speed is not transfer rate, it’s write rate, the speed at which partclone is writing to disk. If you want to see actual network speeds, install iftop on the FOG server, and then run iftop -n during imaging on the server.

[Wayne Workman]

@Haz said in Very slow unicast deploy when imaging 2 or more machines:

I repeated the process above with all 3 machines, however I “staggered” the deploy task to give around 1 minute between each one before the deploy starts.
This is where the speed issues began.
The first started normally around 6GB/m, but when the second machine started roughly 1 minute later, it was getting no more than 500MB/m for the first 15 minutes of deploying. The third machine started a minute after again, with no more than 350MB/m for the WHOLE deploy.

That is what I’ve found too honestly. This I believe is normal. A while back, I debated for changing the default Max Clients setting to be 2 or 3 instead of 10. Why? Because this prevents one computer from taking 2 hours to image, by limiting how many are going at once to just 2. I’d recommend you limit your max clients to 2 or 3 and you will most likely see the same overall duration decreases for imaging a lab of 30 computers as I did. Computers that try to image when 3 are already going will simply wait in line for their turn via FOG’s queuing system (you don’t have to do anything for this)- and that’s perfectly fine.

[Haz]

@george1421 said in Very slow unicast deploy when imaging 2 or more machines:

Was the image recorded with FOG 1.2.0 or newer or is this an old 0.3x image?

The image I have been using for these tests was captured using the new 1.3.0. I was thinking of transferring our old images to the new server, however I didn’t have much faith in it actually working or at best there would be issues somewhere down the line so I just captured from fresh.
Using an SSD was discussed, and I noticed an old thread here where somebody was thinking about using one, but I don’t think the results were actually shared. Now that you have clarified this for me I believe I will be going down the SSD route. Thanks.

@Wayne-Workman said in Very slow unicast deploy when imaging 2 or more machines:

That speed is not transfer rate, it’s write rate, the speed at which partclone is writing to disk.

Thanks for confirming this. I was getting rather confused after the server was continuing to display this speed after you stated it was not possible!

@Wayne-Workman said in Very slow unicast deploy when imaging 2 or more machines:

I’d recommend you limit your max clients to 2 or 3

That is a good idea. It will be a much better option for hosts to simply wait in line so that they can image at full speed rather than pull the rest of the hosts down. Considering the vast speed increase of 1.3.0 on just 1 or 2 hosts, this will be just as fast, if not faster than our old method on 0.32 anyway.

Many thanks to all of you for your input and suggestions, great help as usual.
Keep up the good work!

Harrison

[Quazz]

This thread is strange to me, because we can unicast like 7 hosts with barely any speed lost (7-8GB/min). Our Fog server has a refurbished HDD, so I don’t know if putting an SSD in your FOG server would help much.

Sounds more like your clients are being very slow for some reason (or network derping)

[george1421]

@Quazz said in Very slow unicast deploy when imaging 2 or more machines:

This thread is strange to me, because we can unicast like 7 hosts with barely any speed lost (7-8GB/min)

Please describe your fog server then (hardware, and number of nic connected to your lan from fog server)

[Quazz]

@george1421 It’s an HP Elite 6200 I think? Core i5 3470 or something, 6GB RAM, refurbished 5200RPM HDD, one 1gbps NIC, nothing fancy.

[george1421]

@Quazz Hmm, its less than I expected. But impressive speeds none the less.

[Quazz]

@george1421 I should note, that we don’t use FOG client and storage nodes or anything, so it literally only does network boot menu and imaging, which might explain it a bit.

[Wayne Workman]

@Quazz I’ve not had the same luck you’ve had. Our bottleneck was always the network connection to the FOG server, I could clearly see the NIC being maxed out by watching iftop -n

[george1421]

@Wayne-Workman Create a LAG trunk of 2 or more links. Its not fool proof but will help if you are using a static lag or lacp (802.3ad).

Depending on the algorithm used it will select a lag channel based on host target IP or MAC hash. This will spread the traffic out. I have to say it carefully because LAGs don’t multiplex the traffic over all links only allocate the traffic between two devices to different links.

[Wayne Workman]

@Haz The thread has gone off topic and has been forked to here:
https://forums.fogproject.org/topic/9085/host-already-registered-and-memtest-issue