Help with Mass Hard drive cloning station

[zionda]

Hi Guys,
I’m trying to build a cloning station and i was wondering if FOG is suitable for what i am after

I have a i7 PC that has multiple USB 3.0 ports
Each usb port has a Orico 4xBay SATA HDD connected

The PC see’s 40 hard drives connected

Question is , is FOG capable loading a source image and deploy it on the HDDs connected simultaneously?

If so, is it also able to format the 40 HDDs?

Thank you !

Zion

[george1421]

FOG isn’t a good solution for this situation. If you had 40 PCs connected to those 40 hard drives I would say yes its a strong solution.

I would even say for your task Clonezilla wouldn’t be a good solution. If you had a single local drive attached to a usb dock then Clonezilla would be a good fit. You will have a hard time finding a computer as well as a program that had enough bandwidth (cpu and backplane) to push data to all 40 drives all at once.

[zionda]

Thank you for the replay

Is there a image to multiple local hdds option?

I mean if the qty were lower , for example 1 to 10 would that be suitable ?

[george1421]

@zionda No there isn’t. FOG is setup for 1 computer 1 image. You really want a disk duplicator not an imaging solution.

Something like this: https://www.startech.com/HDD/Duplicators/SATA-Hard-Drive-HDD-Duplicator-Dock-eSATA-USB~SATDOCK22RE but on a larger scale.

[george1421]

@zionda Maybe something more on this magnitude
http://www.produplicator.com/products/easy-loading-hdd-ssd-duplicator?variant=39289378703&gclid=CKGL8ZyB-NMCFY66wAodH8oAXw

I’m trying to think if you could do something manually with linux and something like dd to move the image from the host linux server to each drive. But again I’m not sure if you could get a computer big enough to move that much data.

How large of an image are you trying to clone?

If you had 1 hard drive per computer you could use fog and a multicast image to deploy a single image to 50 computers at the same time.

[george1421]

@zionda Hmmm… a raspberry pi cluster/supercomputer and FOG might do it.

Thinking about it you might find that 2 groups of 20 may image faster than 1 group of 40 just because of the data bandwidth you are talking about.

[zionda]

@george1421

Well actually maybe you guys have a diffrent idea

We are installing each time different images to diffrerent schools in our region

We sometimes needs to install 500-800 pcs all exact same pc model and specs and all of them with the exact same image

We use pxe boot to image each pc but i cant find an effective way to so it in mass installation since we have limited space and few technicians and im trying to make the imaging process more effective

Would love to know if you have any suggestions on how to handle mass installations (i tought making a duplicator and just mass install the hdds and assemble onto the pcs will be faster, specially when sometimes there is an Hdd upgrade than we already have the hdds outside the pc)

Any best practice will be helpful

[george1421]

@zionda What is your time span to install the 500 to 800 systems? (minutes, days, weeks?)

Fog can do a multicast deployment where it send out a single image and the devices configured to receive it will do so. If your network is built properly you can do this in place without touching the computers at all (understand there are a few assumptions in that statement). I would probably limit the deployment size to 50 or so just because your overall deployment speed is dependent on your slowest computer being installed. Fog will do this today.

Now if you have a routed network (i.e. subnets, vlans) you will need to setup a multicast router so that the multicast traffic gets routed properly without swamping your network. Having managed switches that support igmp snooping also helps to keep the multicast traffic to switch ports where there are multicast subscribers.

[george1421]

@zionda Hopefullt @Wayne-Workman will be along soon. He worked at a pretty big school district and has first hand experience with imaging on a large scale.

[Wayne Workman]

@zionda said in Help with Mass Hard drive cloning station:

@george1421

Well actually maybe you guys have a diffrent idea

We are installing each time different images to diffrerent schools in our region

We sometimes needs to install 500-800 pcs all exact same pc model and specs and all of them with the exact same image

We use pxe boot to image each pc but i cant find an effective way to so it in mass installation since we have limited space and few technicians and im trying to make the imaging process more effective

Would love to know if you have any suggestions on how to handle mass installations (i tought making a duplicator and just mass install the hdds and assemble onto the pcs will be faster, specially when sometimes there is an Hdd upgrade than we already have the hdds outside the pc)

Any best practice will be helpful

FOG can do this job natively, as @george1421 already said. FOG uses PXE, that’s just one aspect of how it works. Now, before you think that FOG is anything like GHOST, stop. Because GHOST vs FOG is not even a competition. FOG vs WDS is not a competition. Comparing these would be like comparing a lame horse to a new motorcycle. These other solutions don’t come close to FOG’s speed nor it’s functionality, or support for that matter.

The way you are wanting to do this (in your OP) is a bad idea and we aren’t going to help with it, it’s also not best practice and would be super labor intensive and lack all the great features that you don’t know about yet which FOG offers as standard, supported features.

If you want to setup a P.O.C. for FOG, we can assist - you can see how it works, we can go from there. This is what I would suggest. Instead of getting caught up with a billion questions here in a thread without actually trying it out. With our help, and this being your first experience, you can probably have a P.O.C. setup and functional in a couple hours. We have plenty of guides for you to work from, also. I recommend this one: https://wiki.fogproject.org/wiki/index.php?title=CentOS_7 You will need to modify your DHCP server during setup, notes on that are here: https://wiki.fogproject.org/wiki/index.php?title=Modifying_existing_DHCP_server_to_work_with_FOG

[Junkhacker]

i have to recommend that you give FOG a shot as it is supposed to be configured. with as fast as FOG images systems, you’ll have them all imaged in the time it would take you to remove all their hard drives for your original intended method. does anyone have a demo video from after we added zstd compression?

[Tom Elliott]

@Junkhacker I don’t :(. I’ll admit, I don’t record any videos. Not that I couldn’t but I’m usually tweaking so many things at once, that my videos wouldn’t make sense comparitively.

[Wayne Workman]

I can do a screen capture of it, but my equipment is old as dirt and my setup is far from optimal. I’m working at home with a 100Mbps network, and a test windows 10 image. Worse, the test server and test box are virtualized so that further worsens my performance. So I’m only getting about 6 minute imaging times.

[Junkhacker]

@Wayne-Workman i’ve created a demo video in the past for a conference i spoke at
https://youtu.be/gHNPTmlrccM
i just don’t have the time to edit the video for a new one

[Wayne Workman]

@Junkhacker Must be private? I can’t see it.

[Junkhacker]

@Wayne-Workman fixed
of course, this is super slow compared to our current imaging speed. i image in almost half that time now.

[Avaryan]

@zionda said in Help with Mass Hard drive cloning station:

Hi Guys,
I’m trying to build a cloning station and i was wondering if FOG is suitable for what i am after

I have a i7 PC that has multiple USB 3.0 ports
Each usb port has a Orico 4xBay SATA HDD connected

The PC see’s 40 hard drives connected

Question is , is FOG capable loading a source image and deploy it on the HDDs connected simultaneously?

If so, is it also able to format the 40 HDDs?

Thank you !

Zion

In regards to the original question, it may be possible to assign each HDD to a virtual machine and deploy to each one. Can’t say I’d recommend doing that though.

[x23piracy]

@Avaryan @zionda thats really uncomfortable and kind of tinkering. Just forget FOG for this case, just use a cloning solution on that local computer were you have all the drives connected via usb3.

I’ve seen something different in the past at atos origin, they used IBM Tivoli but they connected the whole computers not just the harddrives, but just using a harddrive was also possible with that.

Regards X23

[Junkhacker]

@x23piracy yeah, if he wants to clone to the drives in the way he originally describes clonezilla is probably the best choice.

[zionda]

Thank you for the replies and discussion guys !

So i understand that i might be doing some workaround and it’s not the way to go.

what we are experiencing currently is that we have a limited of PC’s we can install at once from PXE (we can do approx 10-15 at a time)

sometimes the PC’s for the schools supposed to have their HDD’s replaced with SSD’s or bigger SATA ones and than i need to remove the old one and install the new ones, so i figured for this situations i rather put them back after i cloned the new drives with the images.

the FOG as i read is an Amazing solution but my question is do you think the right way is to connected each PC individually ? will it be faster (and less logistic) than doing it bulk directly to the drives ?