Slow upload from VMware ESXi
-
Dear all
I am experiencing a bizarre phenomenon, similar to what some others have experienced, but after reading all the other threads I can see on slow uploads/deployments, nothing seems to quite fit.
[U]Background:[/U] I have built a SOE image Windows 7x64 for deployment at our school (single disk, re-sizable). The image was built as a virtual machine on an ESXi v5 host (for easy snapshots etc). The FOG server is also a virtual machine on ESXi v5 (Ubuntu 12.04LTS, FOG 0.32, plenty or RAM, working well)
[U]Phenomenon:[/U] when uploading the transfer speed is quite slow (averaging 300MiB/min), and does a random stop-start at the image upload dialogue screen. I can’t see anywhere in the network that should slow up the transfer (all physical and virtual switches are gigabit, 10gig backbone from storage SAN to ESX hosts, bulk RAM, etc).
The image build machine is utilizing an E1000 network adapter and the FOG server is using VMXnet3 (enabling 10gig networking) and have tried different combinations of adapters. This problem seems to be isolated to the ESXi virtual machines, as other test uploads from physical machines run at “full speed” (ie: gigabit speeds) without the observed pausing.It does seem to isolated to something to do with ESX (maybe?). Has anyone else seen this? See the attached video that describes what is happening.
[media=youtube]Vd-zNtoYczA[/media]
Thanks for your time
-W -
Build a custom kernel with the drivers included for the VMWare drives. This is an issue with hardware and drivers. My recommendation is the scrap that ESXi, it’s damn near useless with FOG. Instead use Vitual Box, which uploads a HELL of a lot faster than VMWare.
I troubleshot this when I started my imaging this summer, and I never really fully figured it out. I got SOME speeds back with a custom kernel and the kitchen sink kernel, but all in all time was saved by using Virtual Box instead.
Don’t bother trying to port your VMWare to Virtual Box either, you’ll save time but just starting over because of the hardware that VMWare uses.
This is just my personal experience, take it with a grain of salt, but the issue is Kernel drivers.
[url]http://fogproject.org/forum/threads/increase-vmware-host-upload-speeds.4388/#post-12649[/url] My post all I had to search for was VMWare, it was the second listing.
-
I had read that thread, but as I said above, I haven’t found that particular stop-start problem described before. The above is with a custom kernel I built, with all the VMware drivers I could find included. The only reason I was doing the build using ESX was it was there and has bulk resources to utilize.
You are right though; it is something with the virtual hardware ESXi uses. I have since managed to upload (slow!) the image to the FOG server, deploy it to a local Virtual Box machine, and work can now continue at full speed!
-
Yeah I wish I could have spent the time figuring out what the issue was with VMWare hardware, but I had to get my images done and I wasn’t running on anything fancy. It was painfully slow trying to operate on the machine I was using as an ESXi server. Virtual Box just WORKS and extremely well
Glad you’re on your way!
If you decide to go back to figuring out the ESXi problem I am willing to help as well!
-
I’ll be going back to figure it out one day, because it’s frustrating that the super powerful (and expensive) commercial system upstairs with 10gig network everywhere is painfully slower than the freeware running on my desktop!
Then again, problems like this are why I work in IT. I’ll get back to you when I’ve deployed these 500 machines…
-
What are your drive speeds that you’re running from? The disk speed will really effect the performance of FOG. We run ESXi and have no problems with imaging speeds either with upload or download. We run our servers with NIC teaming, and run 15k rpm drives in some sort of RAID (I think 5, but not certain).
-
The ESXi hosts that the virtual FOG server resides on are connected to a storage unit (15k disks with SSD cache) via multiple 10gbit iSCSI. The hosts connect back to the user network via multiple gigabit paths each. It is only FOG that has a problem with speeds, as other servers (Windows) are crazy quick. I’ve also used VMXnet3 as the adapter on the FOG server, enabling theoretical 10gig networking. There is no problem with downloading an image from the FOG server to a physical machine, only uploading from an ESXi virtual machine to the FOG server. Random.
Like James-Driver says, it is something to do with the drivers in the FOG kernel.
