HP ProBook 640 G8 imaging extremely slowly

Jacob Gallant

@sebastian-roth Thanks again Sebastian, here is the results of that command. Unfortunately we see the same network performance issues with this kernel.

https://photos.app.goo.gl/6Ur7aNkjF92MEKAi6

Sebastian Roth

@Jacob-Gallant Ok, so the 3.8.7 driver version does initialize and probe the NIC correctly.

Unfortunately we see the same network performance issues with this kernel.

Oh well… we have tried! Can’t believe we are the only ones seeing this issue. Maybe this NIC is not in use widely?!

If you are really keen you could try contacting people on the mailinglist or through email directly - Linux kernel or Intel drivers.

Jacob Gallant

@sebastian-roth Thanks, I appreciate your efforts!

Dungody

Hello .

I have the same problem. But when i plug a usb Stick ( empty or not ) at start , the speed is ok . But i dont know why.

Sebastian Roth

@Dungody That really is strange, though interesting. Have you been able to verify this by testing with and without the USB key for several times??

Sure you have exactly the same model, HP ProBook 640 G8?

From what we have seen in the network dump I was sure this is due to network congestion. But maybe I was on the wrong track with this and network was only what we saw slowing down because of IO slowness being the root cause. Does FOG deploy to the USB key when it’s plugged in?? Which device name do you see in the blue partclone screen? /dev/sda1?

george1421

@dungody said in HP ProBook 640 G8 imaging extremely slowly:

Hello .

I have the same problem. But when i plug a usb Stick ( empty or not ) at start , the speed is ok . But i dont know why.

Depending on the firmware I have seen inserting a usb drive change the device naming order, where depending on how the disks are detected on one firmware the usb drive would be detected first making /dev/sda be the usb drive, and on some other firmware (hardware) inserting the usb drive it will take on the /dev/sdb name. It would be interesting to prove out by setting up a debug deploy with the usb drive installed and then run the lsblk command to see what was in the /dev/sda slot. Then issue the fog command to start imaging from the debug console.

It would be strange to see it deploy to the USB drive faster than the onboard nvme disk.

Dungody

@george1421 I deploy everday 5 or 6 650 G8 / 640 G8, and the only way is to plug an usb stick. Maybe for cache .

Dungody

@sebastian-roth Hello , sorry for my bad english - i m french .

In fact i imaging on the disk NVME .
I have the same problem on the 650G8 and 640G8. when I have no usb stick plugged in the speed is very slow,
alt text

but when I have a usb stick plugged the speed is normal on the nvme,
alt text
that’s why I thought has a file cache problem.

Sebastian Roth

@Dungody Thanks for the update and pictures! Definitely looks like you deploy to the NVMe driver. I still can’t see why a USB key plugged into the ProBook would make a difference. Though it sounds like it does.

In the picture I notice that you have partclone version 0.2.89. In FOG 1.5.9 we use the newer partclone 0.3.13. So I am wondering which version of FOG you use and if that is also playing a role here.

@Jacob-Gallant Would you want to give that a try as well?

Dungody

@sebastian-roth i m using FOG 1.5.7 , Kernel 5.6.18.

george1421

@dungody said in HP ProBook 640 G8 imaging extremely slowly:

i m using FOG 1.5.7

If you don’t want to upgrade to the latest version of FOG we might have you try at least the FOG 1.5.9 init.xz file to see if the updated version of partclone works better with that hardware. I’m still at a loss why using a USB drive would settle down the system, unless they have some kind of strange power saving mode where it would enter a low power state when it thinks there is no user input.

Does other USB devices (external mouse) have this type if impact or is it only usb storage devices that “cure” this issue?

diegotiranya

Same problem here but solved (I think). Same model, same extremely slow cloning performance with 5.10.12-e1000e-3.8.7 kernel and same workaround with usb device connected (pendrive, but not other like mouse). Solution for me was enabling “Storage Controller for VMD” under “System Settings” menu.

I’m pending to test another laptops, only got one yet, but tried twice and looks to be working as spected (5-6 GB/s while enabled, 45MB/s while disabled).

Congrats and thank you very very very much for FOG.

george1421

@diegotiranya Would you help us test something on this hardware?

Ensure you know the state of this checkbox in the firmware. We need to capture the hardware IDs with this setting on and off.
Set up a debug capture/deploy it doesn’t matter. Tick the debug checkbox before scheduling the task.
PXE boot the target computer.
After a few screens of text you need to clear with the enter key you will be dropped to the FOS Linux command prompt. At that command prompt key in lspci -k -nn > vmdon.txt This will output the hardware configuration into the vmdon.txt file. To get the file off the pxe booted computer follow the process below.
Power off the computer and repower the computer, change the VMD setting to OFF.
Continue to PXE boot the computer, it should boot you right back into FOS Linux. Run the same command as above but send to vmdoff.txt.
Post both files in this thread.

Now to get the files out of FOS Linux do the following.

Run ip a s to get the IP address of the target computer in FOS Linux.
Change root’s password by keying in passwd set it to something simple like hello.
Now you can use scp or WinSCP to move the files from the FOS Linux computer to another system to post here. Login using root and the password you assigned in 2 to the IP address you collected in 1.

diegotiranya

Glad to help. Output files attached. Let me know if I can do anything else

vmdon.txt vmdoff.txt

Jacob Gallant

@sebastian-roth said in HP ProBook 640 G8 imaging extremely slowly:

@Dungody Thanks for the update and pictures! Definitely looks like you deploy to the NVMe driver. I still can’t see why a USB key plugged into the ProBook would make a difference. Though it sounds like it does.

In the picture I notice that you have partclone version 0.2.89. In FOG 1.5.9 we use the newer partclone 0.3.13. So I am wondering which version of FOG you use and if that is also playing a role here.

@Jacob-Gallant Would you want to give that a try as well?

I’d be happy to test this (and @diegotiranya storage controller settings) when I can, but we’ve been locked down due to COVID where I am so I can’t get back into the office in the near term. Looks like you have a few other testers now though! Glad I’m not the only one haha.

george1421

@diegotiranya Just to confirm that VMDON works at normal speed without needing to have a usb flash device installed?

VMDON

0000:00:1d.0 System peripheral [0880]: Intel Corporation Device [8086:09ab]

10000:e0:1d.0 PCI bridge [0604]: Intel Corporation Device [8086:a0b0] (rev 20)
	Kernel driver in use: pcieport

VMDOFF

00:1d.0 PCI bridge [0604]: Intel Corporation Device [8086:a0b0] (rev 20)
	Kernel driver in use: pcieport

Looks like the base addressing also changes between vmd on and off.

diegotiranya

@george1421 That’s right. In my case, with VMD enabled, there is no need to plug in any usb storage device in order to get normal throughput.

Sebastian Roth

@diegotiranya said in HP ProBook 640 G8 imaging extremely slowly:

In my case, with VMD enabled, there is no need to plug in any usb storage device in order to get normal throughput.

But if VMD is disabled you can still get faster speeds if you plugin the USB key?

@Dungody Can you confirm enabling VMD in the UEFI settings will get you faster speeds without plugging in a USB key?

Seems like VMD support has been in the Linux kernel since quite some time now. Though I still wonder why we see it being so terribly slow when VMD is disabled.

diegotiranya

@sebastian-roth Yes, the USB key workaround works for me too with VMD disabled (but I had to use a usb3, with an old usb2 didnt’ work).

Anyway, I’ve been doing some write testing with dd and rsync from a debug task, getting same speeds in both VMD modes.

Sorry, I didn’t notice before that you suggested to upgrade because of partclone version. I’m using 0.2.89 too (FOG 1.4.4), so I’m going to get the latest and try again.

Thank you

diegotiranya

Upgraded to 1.5.9 and same behavior: fast with vmd on and slow with vmd off.

I don’t know if this helps but, after some testing, writing speed looks good and the slowlyness seems to occur only when copying from server, but testing network the bandwith looks good too.

One thing that pictures don’t show is that, with vmd off, rsync freezes for a while (even a few minutes) before starting or at the beginning of the copy but not with vmd on. I tried to copy with scp and it hangs too. Locally rsync, cp or dd from /dev/zero work well.

HP ProBook 640 G8 imaging extremely slowly

182

11.7k

17.2k

154.7k