@fry_p ok now I think is time for some hokus-pokus magic without breaking much.
From your production server export the image definitions and import them into your dev environment.
From your production server export and then import the host definitions into the test setup. This will create a clone of your production environment without the image files.
Temporarily move the production FOG server’s IP address (i.e. just swing it out of the way) and move the dev FOG client server’s IP address in place of the production fog server (understand this may impact pxe booting if you have them boot through FOG). The goal here is to apply real world loads to the FOG client server. We should watch the CPU usage on the FOG client server (we may need to increase the number of php-fpm workers to satisfy all of the requests) as well as the mariadb service on the dev imaging server. We are looking for high cpu usages on both systems. This might tell us if we need to spin off the sql server onto its own server. We should also gauge the web ui performance on the imaging server, because that was a pain point when you had the check in times set to the default.
I’m not sure I’d leave the check in time at 30 seconds in a production environment, but it would be interesting to see the load.
When you are done testing then swing the dev FOG client server out of the way and put the production fog server back in place.