Very slow registering host and pushing images to Dell Optiplex 7070 ultras.

[darkxeno]

@george1421 do you think it has anythign to do with the ultra having a nvme drive?

[george1421]

@darkxeno Well what’s not clear is if its a network issue or a disk (nvme) issue at the moment. We are seeing newer (select) nvme drives having a slowness issue writing the image to disk. That issue is being discussed in this thread: https://forums.fogproject.org/topic/13777/extremely-slow-deploy-to-nvme-drives/28

One thing you can test is to go into the global kernel parameters under the fog configuration and add in this kernel parameter
nvme_core.default_ps_max_latency_us=0 You will see an error with that name during booting, but its a warning, it will set the kernel parameter correctly. The issue is that there is a dot in the parameter name. See if that kernel parameter changes anything with write speed. What it does is keep the nvme drive from using any low power mode during imaging. We are seeing random successes with that.

We might also want to get you to move to the dev branch of FOG, that will take your FOG version to 1.5.7.55 or later. The developers added a nvme command line tool that we can use to push commands directly to the nvme disk. But lets first see if the kernel parameter masks the issue.

[darkxeno]

@george1421 I will try that Update you onmonday when I get back to work thanks for your help today

[darkxeno]

@george1421 that seemed to do the trick I’ll do some more Monday and let you now I will go ahead and move to the dev branch as well

[george1421]

@darkxeno Thank you for the feedback.

@Sebastian-Roth tagging you only so you are aware of the issue with the Dell system disks too.

[Sebastian Roth]

@george1421 Yeah seems like we have another slow disk issue model here. Good to know the kernel parameter did help in this case.

@darkxeno About all the other pre partclone (blue screen) issues on bootup. Those have nothing to do with the slowness issue but have different causes. Nie as we see you made you made it past that point several times I am wondering if it’s some network/DHCP issue that happens not always but only sometimes?? Maybe you have a rough DHCP in your network?

[george1421]

@Sebastian-Roth said in Very slow registering host and pushing images to Dell Optiplex 7070 ultras.:

Maybe you have a rough DHCP in your network?

Just as a comment, I’ve seen organizations that have a primary and secondary dhcp servers setup intentionally, but only one of them has the pxe boot information. But looking at the OPs picture it looks like FOS Linux is unable to get an IP address. At that point the PXE boot information isn’t needed. But it does make me think its a networking (or possibly) a dhcp server issue. We probably can/should trap that error with wireshark so we can see who is saying what to the target computer.

[darkxeno]

@Sebastian-Roth I’m curious on this, the reason being is one we have been using fog for years since v 0.27 and haven’t have this issue before. And it’s only been on these 7070 ultras we just got in. I just did an image on the regular optiplex 7070s and had no issue with them at all. However this past Tuesday we had a battery back up fail and would keep failing every time we though we had it back up so we had a lot of unsafe shutdowns. So that could be a possibility. However when I pushed those other images that was after all of that was resolved and we didn’t have those issues but we’ll do a wireshark and check I’m curious now.

[george1421]

@darkxeno First thing you have to keep in mind that the XX70 generation is still very new. The FOG Project relies heavily on the Linux Kernel developers for hardware support. When hardware support is added by the Linux kernel developers the FOG Project devs integrate that kernel into FOS Linux. The Linux kernel developers will always be behind in releasing supporting hardware drivers because hardware vendors and chip manufacturers sometimes release hardware in a closed manner where they release the chips and provide the windows driver.

But from a wireshark perspective, we can see what is flying down the wire using a witness computer connected to the same subnet, or the FOG server if its on the same subnet as the pxe booting computer. The FOG server is preferable since it would also collect unicast communications between the target computer and the FOG server. If the fog server can be used (i.e. same subnet as target computer) I have a tutorial for that: https://forums.fogproject.org/topic/9673/when-dhcp-pxe-booting-process-goes-bad-and-you-have-no-clue

If you are using wireshark from a witness computer (i.e. computer plugged into the same subnet as the pxe booting computer) you can (should) use the capture filter of udp port 67 or udp port 68 That will only capture the dhcp/bootp part of the booting process. This is possible since dhcp uses broadcast messages to communicate so all hosts on the subnet will receive these broadcast dhcp messages.

[darkxeno]

@george1421 thanks for the info I’ll get that done Monday and I’ll left you know what we find