Advice on specs for new setup
-
Hi All
I’m working on a project to redo my employer’s imaging setup since we’re moving to a new location and redoing everything ground up. To give you more insight into what we’re currently using. We have FOG on Ubuntu servers using normal Lenovo workstation computers. I have a “bench” of computers which consists of 15 spaces for imaging and a FOG node at each of the four benches giving a total of 60 spaces. We have a huge turnover for computers since we’re a technology and A/V rental company.
We’re looking to potentially double the amount of spaces up to 120 but that doesn’t mean we’ll have all of those spaces imaging simultaneously. Since I’ve found that at a certain point you’ll have reach the turnover where computers that were first setup were already done by the time the last computers are started.
Is there any recommendations on server grade hardware to make sure that half of those spaces are imaging as fast as the endpoints hardware will allow? Would a few beefy servers with a lot of discs be better than a lot of smaller servers? At this point assume the network itself won’t be the bottleneck because that will be completely redone with 10G backbone for the new office.
Thanks in advance for the help.
-
@candidom said in Advice on specs for new setup:
Is there any recommendations on server grade hardware to make sure that half of those spaces are imaging as fast as the endpoints hardware will allow?
Solid state disks in the server would be the best performance boost on a copper 1Gbps network for strictly imaging only purposes.
-
I guess we might need a bit more information on how you plan on imaging.
- How many systems do you plan on imaging at one time. Understand this may be a by using a multicast or staggered unicast imaging. If you could guess, how many systems that might be imaging at one time?
- Will the fog server and all clients on the imaging bench be on the same 1GbE network switch? Or is the FOG server in a computer room someplace on its own switch, connect to a network switch on the deployment bench?
From a FOG imaging standpoint the actual system load is not on the FOG server. The FOG server simply moves the image from the hard drives to the network adapter. The heavy lifting in imaging is done at the target computer. The fog server doesn’t need to be a powerhouse server, it does need a good disk subsystem and network connection. FWIW, I have a raspberry pi setup as a fog server. It does work well, but only if you are sending out a single unicast or multicast stream.
-
To answer your questions
-
I would say that if we could get half of the expected capacity of the new workspace that would be great. So about 60 laptops. I figure that by the time 2 or 3 people get those 60 setup the first ones they started would already be done imaging.
-
We plan on having the FOG servers attached via 10G to a stacked set of 10G distro switches. This would then feed the access switches over 10G as well for the imaging benches which would be regular 1G copper.
We generally don’t do multicast since we have a lot of different images that are getting pushed at any given time. The staggered approach is what we’re used to already. I’m hoping that we can set up the FOG nodes so that at about 60 or so computers we’re just going to be bottlenecked at the target computer and not the FOG hardware.
Right now with 60 spaces open we can load the whole bench in an all hands on deck scenario and when normally on no load we’d hit 3.8GB/min on the laptops we’re now at 756MB/min instead. The goal is the keep as close to maximum imaging speed as possible so we have no doubts that the target computers are whats holding us back.
-
-
@candidom Lets see, I’m not sure how to respond, but with some existing bench marks.
With a virtualized FOG server on a 1GbE infrastructure, I can get about 6GB/min transfer rates (single unicast image). In an ideal world a 1GbE network will sustain (theoretically) about 7.5GB/min transfer rate. This is not on a dedicated imaging network, but on the general business network. My 25GB fat image takes about 4 minutes to push to the target system. The target system is a Dell 7280 with an SSD drive. I think I was getting in the 4.5-5GB/min deploying to a E6430 with a traditional hard drive.
During some bench mark testing with a Dell Optiplex 790 with an SSD drive as the fog server, I was saturating the 1GbE link between the fog server and the core switch between 2 and 3 simulations unicast streams. If you stick with a 1GbE network, its best to setup a LAG team between your FOG server and your business switch. With 10GbE then a 2 port LAG is nice to have but not required. I would expect the 2-3 on 1GbE to scale to at least 15 simultaneous unicast streams on 10GbE (but I would expect your disk subsystem to be the bottleneck with this setup 10GbE == 1,250MB/s). On our production server with 10GbE to the access layer switches we are seeing 13GB/min on modern hardware.
[sidebar] The speed rating in partclone is a bit misleading since that is a composite score of network throughput and extraction speed to the target computer’s hard drive and not actual networking throughput. You can have a perfect 10GbE network to the desktop and still get 1.5GB/min transfer rates if you have a single core celeron processor in the target computer. [/sidbar]
The other issue you will run into is the disk drive in your FOG server. If you have a traditional hard drive in your FOG server running on a single disk. The physical disk it self will be the bottle neck when sending out more than one unicast stream. Consider that unless the unicast streams were started exactly at the same time that hard disk head will be bouncing all over that drive trying to supply data to multiple streams in different spots in the stream. Moving to a disk array with multiple spindles (disk) will alleviate this issue, or by installing a ssd (disk) or array to get the size will address this issue.
All of this again is based on the number of simultaneous streams you need to support. Just using my numbers of a 5 minute image push, if you serialized the image deployment so you didn’t have more than one unicast stream running at any one time, you could image about 12 systems per hour.
-
Thanks for the info george1421. I think because we haven’t tested on 10G backbone yet I am not sure what to expect as far as how many unicast streams we’ll be using before the link is saturated at that level.
The plan that I have right now is to get 3 servers with hard drives in RAID 10 but with a lower end CPU and RAM since the FOG nodes don’t necessarily need that power. As we do some testing with that initial setup and we find that we’re saturating the 10G backbone link or need to add another node then we can.
-
@candidom One other comment on performance is to use the proper image compression when you capture your reference images. Use the zstd format over gzip. zstd is a new(ish) compression algorythum. Its slower on image capture but much faster on image deploy than gzip.
I did some benchmark testing a while ago but I can’t seem to find the post at the moment. In regards to your disk subsystem I had some linux commands that would give you an idea of the performance you can expect from the disks. : https://forums.fogproject.org/topic/11433/rate-is-at-a-slow-crawl-when-trying-to-deploy-capture-image/17
I’ll keep searching for my original post that had comparisons and resultant data.
-
if you’d like my advice, here’s what it would be. just buy 1 server with SSDs in a raid 10 or raid 5. use zstd compression on your images set to a compression level of 11.
i honestly think you’ll get better performance out of a single fog server with a solid storage subsystem then 3 lower speced servers (not to mention you won’t be wasting the storage on redundant copies, allowing better cost benefit per GB)
-
Hey @Junkhacker thanks for the input. The thought of replication and time to replicate did cross my mind. My only concern with one FOG server is being able to image 45+ computers at a time even with drives in RAID 10. I was going to do 3 of the server setups you suggested though. So 3 servers with SSD in RAID 10 which, in my mind, should be more than enough to do 45+ computers at once.
-
@candidom i think one FOG server configured that way should be able to handle it. you might want to consider trying it with 1 before buying the other 2. i was doing 30+ at a time with traditional drives in a raid 5 at speeds that i found quite adequate.
-
@junkhacker That’s probably the best path to follow actually. There’s no rush for us to purchase everything at once. Worst case we have enough nodes now to get us by if we do need more. Thanks again for the help!
