Several Fog Issues
-
Compression might be hurting your upload… Depending on how many vCPUs are on your guest. And if your SSD is sandforce based it will get it’s speed boost by using compression… I get similar speeds to you with 1 normal HDD in my test server.
What hypervisor are you using? I use ESXi and Virtualbox. ESXi needs IDE devices (possibly not needed anymore on 0.33b) and virtualbox works out of the box for me.
-
[quote=“VincentJ, post: 21121, member: 8935”]Compression might be hurting your upload… Depending on how many vCPUs are on your guest. And if your SSD is sandforce based it will get it’s speed boost by using compression… I get similar speeds to you with 1 normal HDD in my test server.
What hypervisor are you using? I use ESXi and Virtualbox. ESXi needs IDE devices (possibly not needed anymore on 0.33b) and virtualbox works out of the box for me.[/quote]
On my mac i am using VMWare Fusion 6.0 and i can’t get it to pull it refuses to mount the images folder. My Mac is a Dual Core i7 with 8gb of ram and a 256gb SSD.
My PC is an 8 Core AMD with 32GB of ram and a 512gb SSD Running Workstation 10.0 which I did get it to pull but after it pulled it failed to connect to the TFTP it had lost connection and wouldn’t re-establish connection so i guess it failed, after waiting over an hour for it to pull a simple windows 7 image, and it is compressing if i build this image on real hardware and pull to real hardware its like 40gb but the image on the server is around 10gb so it has to be compressing already
-
how many vCPUs does the VM that is uploading have?
What is running your DHCP service for the VMs? I always have pfsense to run DHCP on my test networks, never tried the internal one. -
[quote=“VincentJ, post: 21123, member: 8935”]how many vCPUs does the VM that is uploading have?
What is running your DHCP service for the VMs? I always have pfsense to run DHCP on my test networks, never tried the internal one.[/quote]2 CPU’s @ 2 core’s when on my AMD. but when on my mac its a single processor dual core. but on my mac i can’t even get it to pull. and in a real hardware situation the uploading machine is a core i5 with 4gb of ram and a 5400rpm HD. and with i upload the image with fog .32 it is much much much faster at least 1/3 the time
-
But as VincentJ stated, it could be a problem with the IDE nature of the hard drives in the virtual hardware itself. I know when I use VMWare I had to build a custom kernel that included scsi drives just to be able to upload, and it was ALWAYS painful to upload from VMWare. I [U][B]NEVER[/B][/U] solved the issue with slow uploads. I switched to using VirtualBox instead. It uploads WAYYYYY faster than VMWare could have hoped to.
I know that’s probably not what you wanted to hear, but I would test some other physical hardware (Nothing spectacular just upload and image and push it to a test machine) so you can see if the speeds are normal. If the image uploads/downloads at a decent rate, you know the issue has to be with the virtual hardware. In which case I would recommend building a custom kernel, which MAY NOT solve the issue.
Hell I will link you another topic I was a part of about this discussion maybe you can pull some information from it.
[URL=‘http://fogproject.org/forum/threads/slow-upload-from-vmware-esxi.4541/#post-13208’]http://fogproject.org/forum/threads/slow-upload-from-vmware-esxi.4541/[/URL]
[url]http://fogproject.org/forum/threads/increase-vmware-host-upload-speeds.4388/[/url]In these particular topics you can see the discussion I had with the posters, some decided to barate me instead of condusively figuring out the issue. Some people have no issues with VMWare, but I never could get anyone to stick around long enough to troubleshoot the issue with me.
-
[quote=“Jaymes Driver, post: 21135, member: 3582”]But as VincentJ stated, it could be a problem with the IDE nature of the hard drives in the virtual hardware itself. I know when I use VMWare I had to build a custom kernel that included scsi drives just to be able to upload, and it was ALWAYS painful to upload from VMWare. I [U][B]NEVER[/B][/U] solved the issue with slow uploads. I switched to using VirtualBox instead. It uploads WAYYYYY faster than VMWare could have hoped to.
I know that’s probably not what you wanted to hear, but I would test some other physical hardware (Nothing spectacular just upload and image and push it to a test machine) so you can see if the speeds are normal. If the image uploads/downloads at a decent rate, you know the issue has to be with the virtual hardware. In which case I would recommend building a custom kernel, which MAY NOT solve the issue.
Hell I will link you another topic I was a part of about this discussion maybe you can pull some information from it.
[URL=‘http://fogproject.org/forum/threads/slow-upload-from-vmware-esxi.4541/#post-13208’]http://fogproject.org/forum/threads/slow-upload-from-vmware-esxi.4541/[/URL]
[url]http://fogproject.org/forum/threads/increase-vmware-host-upload-speeds.4388/[/url]In these particular topics you can see the discussion I had with the posters, some decided to barate me instead of condusively figuring out the issue. Some people have no issues with VMWare, but I never could get anyone to stick around long enough to troubleshoot the issue with me.[/quote]
I set my virtual machine HD’s to SATA, IDE & SCSi are options but i don’t use either… should i bother trying the other ways? also it uploads slow on brand new hardware… but if i run the old fog .32 withtout partclone i don’t deploy as fast but i pull at 500-700mb/min vs topping out under 300mb/minute so i don’t think its my hardware. PartClone just seems a lot slower at pulling…
-
That may be a possibility. You can try the Partimage solution.
-
I’m going to try a little something here, basically We need to use PIGZ/GZIP for compression, but I think I’ll add PIGZ_COMP as an added variable to to the PXE Menu Generation and remove it from the fog scripts. It will be in it’s infancy, but it, should, work. Right now I’ve got PIGZ set to best compression, which means it takes a while to compress the data, which would ultimately be the bottleneck in this situation. This, theoretically, would speed things up tremendously on the upload, as well as during the download tasks. You’re making the compromise of speed/disk space.
-
r1059 released to test this new functionality out. Please test and play. You can change the setting under:
FOG Configuration->FOG Settings->FOG PXE Settings->FOG_PIGZ_COMP
Hopefully you’ll enjoy it. The values are between 0 and 9. 0 being the fastest/worst compression rating and 9 being the slowest/best compression rating.
-
[quote=“Tom Elliott, post: 21139, member: 7271”]r1059 released to test this new functionality out. Please test and play. You can change the setting under:
FOG Configuration->FOG Settings->FOG PXE Settings->FOG_PIGZ_COMP
Hopefully you’ll enjoy it. The values are between 0 and 9. 0 being the fastest/worst compression rating and 9 being the slowest/best compression rating.[/quote]
damn your quick. Ask you shall receive
I will have to wait to get home to my Desktop of DOOM… i can’t even get my mac to mount the images folder when i try to use the private internal network in Fusion
