Uploading an image from Hyper-V is extremely slow

Sebastian Roth

@loosus456 Which version of FOG (see in the blue cloud on the web interface) and kernel version (FOG configuration -> Kernel update) do you use? Maybe related to this: https://forums.fogproject.org/topic/6695/performance-decrease-using-hyper-v-win10-clients

A Former User

I’ll check all this when I get back to work tomorrow.

I will say just a few things now:

The performance of the VM is gorgeous until I use FOG. I usually get really good performance in the VM when it’s loaded with Windows 7 or Windows 10.
One weird thing that I noticed is that the reason the upload seems slow is that it sits on a group of blocks for long periods of time. It’ll upload a bunch of blocks, and then it’ll just sit for like 5 seconds and upload absolutely nothing. It does this throughout the upload. So it seems to have bursts of activity and then nothing, over and over.
From what I can remember the upload speed was about 600 MB/min most of the time. The compression on the image was set to 6.
If I use the exact same FOG settings on the exact same same hardware (but on the physical hardware instead of a VM in Hyper-V), I can usually get about 8 GB/min upload speed.

I’ll try to get more specifics tomorrow if I get a chance.

ch3i

@loosus456 Hi, the 600MB/min looks like a FastEthernet speed, not a Giga.

george1421

@ch3i said:

@loosus456 Hi, the 600MB/min looks like a FastEthernet speed, not a Giga.

I agree with ch3i’s conclusion. 600MB/min translates to 10MB/s which is just below 100Mhz (fast) ethernet’s theoretical max of 12.5MB/s.

If this was a GbE network, I might expect slightly less than 125MB/s or (7GB/min) transfer rates. As you noted with physical hardware I would expect between 4-6GB/m download rate with less depending on the hardware and compression ratio on the upload side.

I would be sure to update FOG first then check your network path for a 100Mhz links.

Andy Abplanalp

@loosus456 does this server have Broadcom NICs in it by chance? If so, you can look into disabling Virtual Machine Queues.

A Former User

Haven’t had a chance to look at everything just yet, but the version of FOG is 6321.

In regards to it being FastEthernet: it sometimes bursts as high as gig does, which is what makes me believe there is an issue.

Sebastian Roth

@loosus456 Any news on this? Did you have time to look into all the suggestions and try things out?

rdw

@loosus456 Any resolution? Exact same symptoms here. Thanks.

Wayne Workman

@rdw I remember that there is a specific linux kernel argument for Hyper-V. I would guess that this would really help out - definitely wouldn’t hurt anything. I’ll see if I can find it…

Wayne Workman

https://technet.microsoft.com/en-us/library/dn720239.aspx

Since FOS (fog operating system) is linux, all that stuff could apply potentiall.

There is a kernel argument though. I have to get on a actual computer to find it…

sudburr

If it is Hyper-V 2012 R2 and you are running a Generation 1 VM which uses the Legacy Network Adapter for PXE booting then 100 Mb is all you will get as the Legacy Network Adapter is only a 100 Mb NIC.

Having said that, some kernels in the past have on occasion switched to the 1Gb NIC for uploading … including right now. I just rolled back to 4.1.4 from 4.6.2 and I’m getting Gb.

Wayne Workman

I know it’s been a while, but I just came across this: http://www.funtoo.org/HyperV_Kernel_Configuration

These are the Hyper-V kernel arguments listed on that page, just so they aren’t lost in the depths of the internet:

HyperV Kernel Configuration

For a Hyper-V generation 2 system, you’ll need certain options enabled in the Kernel in this order because latter options aren’t available until earlier options are enabled:

    CONFIG_HYPERVISOR_GUEST: Processor type and featueres > Linux Guest Support
    CONFIG_PARAVIRT: Processor type and features > Linux Guest Support > Enable paravirtualization code
    CONFIG_PARAVIRT_SPINLOCKS: Processor type and features > Linux Guest Support > Paravirtualization layer for spinlocks
    CONFIG_HYPERV: Device Drivers > Microsoft Hyper-V guest support > Microsoft Hyper-V client drivers
    CONFIG_HYPERV_UTILS: Device Drivers > Microsoft Hyper-V guest support > Microsoft Hyper-V Utilities driver
    CONFIG_HYPERV_BALLOON: Device Drivers > Microsoft Hyper-V guest support > Microsoft Hyper-V Balloon driver
    CONFIG_HYPERV_NET: Device Drivers > Network device support > Microsoft Hyper-V virtual network driver
    CONFIG_HYPERV_STORAGE: Device Drivers > SCSI device support > SCSI low-level drivers > Microsoft Hyper-V virtual storage driver
    CONFIG_HYPERV_KEYBOARD: Device Drivers > Input device support > Hardware I/O ports > Microsoft Synthetic Keyboard driver
    CONFIG_FB_HYPERV: Device Drivers > Graphics support > Frame buffer Devices > Microsoft Hyper-V Synthetic Video support
    CONFIG_HID_HYPERV_MOUSE: Device Drivers > HID support > Special HID drivers > Microsoft Hyper-V mouse driver

Sebastian Roth

@Wayne-Workman This is what we have in the current config:

grep -e HYPERV -e PARAVIRT TomElliott.config.asc 
CONFIG_HYPERVISOR_GUEST=y
CONFIG_PARAVIRT=y
# CONFIG_PARAVIRT_DEBUG is not set
# CONFIG_PARAVIRT_SPINLOCKS is not set
# CONFIG_PARAVIRT_TIME_ACCOUNTING is not set
# CONFIG_PCI_HYPERV is not set
# CONFIG_SYS_HYPERVISOR is not set
CONFIG_HYPERV_STORAGE=y
CONFIG_HYPERV_NET=y
CONFIG_HYPERV_KEYBOARD=y
CONFIG_FB_HYPERV=y
# CONFIG_HID_HYPERV_MOUSE is not set
CONFIG_HYPERV=y
CONFIG_HYPERV_BALLOON=y

Are you keen to compare those and let us know if we should any option.

Scott Adams

Any update on this? I have just started universal imaging using Hyper-V, which is working great. However, the upload speed from VM to FOG-server is slow. Currently uploading now, and am getting >300MB/min

Uploading an image from Hyper-V is extremely slow

192

12.1k

17.3k

155.4k