Change Max Clients default from 10 to 3.

[Wayne Workman]

The main reasoning about this is that - once the server’s HDD has reached it’s maximum seek rate while still saturating the slowest link in the network with imaging tasks, anything past that will in fact begin to de-saturate the slowest link in the network because the server’s HDD isn’t able to keep up with the seeking without taking a performance hit.

At my work, we have SAS 6Gbps HDDs in our server, and they can image 2 computers at a time without a performance hit. 3 and it suffers. 10 and it really suffers. 10 is not ideal in most situations.

I’m asking the default max clients be set to 3. If people want more, well it’s just a text box and they can change it. But for those that don’t know, the default should be optimal, or at least a really close guess at optimal. 10 is by no means optimal nor a good guess either.

[Mentaloid]

Interesting - Is that a dedicated server or under a VM host?

My location we have FOG running in a VM under ESXi and assigned 32 gigs of VMem. The ESXi host is backed by a Linux NFS host, running a RAID 6 array with 8 SATA 6gbps 10K drives and 32 gigs of RAM. Both are linked by 4*1Gbit bonded Ethernet to the network. FOG does not have an external storage node on the NFS server, FOG is using it’s own virt hard drive.

Between the RAM being used to cache almost entire images in memory at both NFS and FOG, and due to the bonded ethernet, we can run ~14 clients at max speed in unicast. With multicast, we’ve done 70 at a time with no speed degradation.

My point is maybe we could have a tool to check the Ethernet link speed,hard drive read speed, and cache memory size, and intelligently figure the max clients limit? Maybe have separate limits for multicast vrs unicast.

[Wayne Workman]

@Mentaloid It’s in a Hyper-V VM. The VM is assigned 4GB of RAM.

How do you know that images are being cached in RAM? According to these two articles, NFS caching is done client-side, not server side, and from what I gather only caches file attributes and properties.

https://en.wikipedia.org/wiki/CacheFS#Linux_version

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Storage_Administration_Guide/fscachenfs.html

Additionally, this command will give you in a human-readable format the amount of free RAM a Linux system has:
free -h

Running this on my home’s VM FOG server which has 512MB of RAM assigned to it gives this:

[root@fog-server ~]# free -h
              total        used        free      shared  buff/cache   available
Mem:           488M        267M         43M        648K        177M        174M
Swap:          1.0G         54M        969M
[root@fog-server ~]#

I believe you get the sort of performance that you do simply because of the RAID 6 spread across 8 Sata3 drives. That’s great, but I don’t have that, and I’d bet most don’t dedicate such hardware to FOG. My server at work is RAID 1 for fault tolerance and to keep costs low, and again has a few sets of SAS 6Gbps drives in it. With your HDD setup, your seek time is pretty much a non-issue, where as with my setup it’s the single biggest reason why I cap max clients at 2.

[Wayne Workman]

Here’s a post I made supporting this change:

@Wayne-Workman said in Another slow Deployment qustion:

Take this video for example, look at the video @ 5:34

We see the space used on the image is 31.7, write speed is 2.29GB/min.

We see elapsed time is 13 minutes and 31 seconds.

31.7 divided by 2.29 = 13.84, or 13 minutes and 50 seconds.

Therefore the rate that Partclone displays is write speed (or read speed), not network transfer rate. The image is sent across the network in compressed form, so there is much more room available on the network. Once network saturation is reached, it’s reached. However, if you continue to pile on imaging tasks, eventually the HDD seek time is maxed, and you actually start loosing network saturation. This is my reasoning for setting Max Clients to 2 or 3, not 10.

The idea is to saturate the available bandwidth for imaging tasks, but not exceed the HDD’s seek ability to keep up. If you exceed the HDD’s ability to keep up due to seek times, then you’re going backwards.

The optimal setting for max clients should saturate the network fully, but not exceed the HDD’s ability to seek and keep up with the full network speeds.

https://www.youtube.com/watch?v=OgKwhuNB1cs

[Wayne Workman]

nfsiostat will be the tool I use to demonstrate how latencies can degrade network saturation. I’ll be highlighting the RTT (round trip time) with different numbers of unicast tasks running simultaneously. I’ll also highlight total overall NFS throughput. I predict that once HDD seek times are maxed and the HDD can no longer keep up with the number of simultaneous reads, that network performance will drop below 1Gbps, and latencies will skyrocket.

The man page:
http://man7.org/linux/man-pages/man8/nfsiostat.8.html