M.2 not recognised during deploying stage

[Will]

Hi, there

I recently encountered an error message when deploying Windows 10 OS with a specific motherboard and a specific M.2, GigaByte B550M Gaming and Lexar M.2 NM700 256 GB.

This M.2 is shown in BIOS but FOG reported “can not find disk” after entering deploying stage.

FOG version: 1.5.9
Kernel bzImg593RTZ
init-201114.xz

[george1421]

@will Its possible that the disk is hiding behind a raid controller. I don’t know the Gigabyte motherboard, but look in the firmware under disk controller mode. If the setting is anything other than ahci mode, try it in ahci mode. There is a known issue between linux, uefi, and the intel raid controller in raid-on mode. You are running a pretty new kernel so it should support that mobo. FOG does have a later kernel 5.10.12 but then you will loose the realtek patch in the kernel you have. I would check to see if there is a disk controller mode first.

[Will]

@george1421 Hi, George

I can confirm this M.2 is running at AHCI mode, not behind any RAID controller.

This M.2 model is working with other motherboards (i.e GigaByte B550 Aorus Elite & Gaming X, ASUS & MSI B550, B550M, X570…).
GigaByte B550M Gaming motherboard is also working with other M.2 models (i.e WD SN550 1TB, Crucial P1 1TB…).

But when put these 2 modules together, FOG will report “cannot find disk on system”.

[george1421]

@will OK this will take a little work on your side.

1. Setup a new debug capture/deploy (doesn’t matter). Schedule a new deploy task but before you hit the schedule task button tick the debug checkbox.
2. Now pxe boot the target computer after a few screens of text you will be dropped to the FOS Linux command prompt.
3. Key in ip a s and collect the ip address of the target computer
4. Key in passwd and give root a one time password. Make it simple like hello
5. Use putty or ssh to connect to the target compute using the above information. hint this method will allow you to copy and paste commands into the FOS Linux system.
6. Now key in the following. I need to know the output of each.
   lsblk
lspci -k -nn
7. Using Winscp or scp I need the contents of the syslog file from /var/log directory.

That should give us an idea what we are dealing with.

[Fog_Newb]

@will said in M.2 not recognised during deploying stage:

But when put these 2 modules together

What do you mean by put them together? Like in onboard RAID?

Is it possible like with NVMe drives, the M.2s are initializing at different times during the boot process and the drives switch places? Like changes from say /dev/nvme0n1 to /dev/nvme1n1?

[Will]

@george1421 The output for the two commands:

[Fri Mar 12 root@fogclient ~]# lsblk
[Fri Mar 12 root@fogclient ~]# lspci -k -nn
00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
	Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
lspci: Unable to load libkmod resources: error -12
00:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
	Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
	Kernel driver in use: pcieport
00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
	Kernel driver in use: pcieport
00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
	Kernel driver in use: pcieport
00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
	Kernel driver in use: pcieport
00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
	Kernel driver in use: pcieport
00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
	Subsystem: Gigabyte Technology Co., Ltd FCH SMBus Controller [1458:5001]
00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
	Subsystem: Gigabyte Technology Co., Ltd FCH LPC Bridge [1458:5001]
00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 0 [1022:1440]
00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 1 [1022:1441]
00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 2 [1022:1442]
00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 3 [1022:1443]
00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 4 [1022:1444]
00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 5 [1022:1445]
00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 6 [1022:1446]
00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 7 [1022:1447]
01:00.0 Non-Volatile memory controller [0108]: Marvell Technology Group Ltd. Device [1b4b:1092]
	Subsystem: Device [1d97:1092]
	Kernel driver in use: nvme
02:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ee]
	Subsystem: ASMedia Technology Inc. Device [1b21:1142]
	Kernel driver in use: xhci_hcd
02:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] Device [1022:43eb]
	Subsystem: ASMedia Technology Inc. Device [1b21:1062]
	Kernel driver in use: ahci
02:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43e9]
	Kernel driver in use: pcieport
03:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
	Kernel driver in use: pcieport
03:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
	Kernel driver in use: pcieport
03:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
	Kernel driver in use: pcieport
03:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
	Kernel driver in use: pcieport
03:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ea]
	Kernel driver in use: pcieport
08:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 16)
	Subsystem: Gigabyte Technology Co., Ltd Onboard Ethernet [1458:e000]
	Kernel driver in use: r8169
09:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Baffin [Radeon RX 560] [1002:67ff] (rev ff)
	Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Baffin [Radeon RX 550 640SP / RX 560/560X] (Radeon RX 560) [1002:0b04]
09:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Baffin HDMI/DP Audio [Radeon RX 550 640SP / RX 560/560X] [1002:aae0]
	Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Baffin HDMI/DP Audio [Radeon RX 550 640SP / RX 560/560X] [1002:aae0]
0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
	Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
	Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
0b:00.1 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP [1022:1486]
	Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP [1022:1486]
0b:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
	Subsystem: Gigabyte Technology Co., Ltd Matisse USB 3.0 Host Controller [1458:5007]
	Kernel driver in use: xhci_hcd
0b:00.4 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller [1022:1487]
	Subsystem: Gigabyte Technology Co., Ltd Starship/Matisse HD Audio Controller [1458:a184]

messages.txt

[Will]

@fog_newb Sorry for the confusion. I mean the Gigabyte B550M Gaming motherboard and Lexar NM700 256GB M.2 SSD for the “2 modules”.

The Lexar M.2 is having the same “can not find disk on system” issue with Asus Prime B550M-A motherboard.

[Sebastian Roth]

@will said in M.2 not recognised during deploying stage:

The Lexar M.2 is having the same “can not find disk on system” issue with Asus Prime B550M-A motherboard.

Have you tried installing plain Windows on the Lexar M.2 in combination with those mainboards yet? Just want to rule out general hardware incompatibility…

[Will]

@sebastian-roth Yes. There is no issue when using USB Windows 10 Installation media.

[Sebastian Roth]

@Will Searching the web I might have found out why it doesn’t work - thanks to the messages.txt and lspci output you posted earlier.

There are some special flags needed in the Linux kernel to work around an issue with that particular M.2 drive. There is an interesting discussion on this: https://bugs.mageia.org/show_bug.cgi?id=28417

This was reported to the NVMe Linux kernel mailing list as well - so there is hope this is being added at some point: http://lists.infradead.org/pipermail/linux-nvme/2021-February/023121.html

To make sure this patch would actually work in your case as well I build a one off kernel binary for you to test: https://fogproject.org/kernels/bzImage-5.10.19-Lexar_M.2

Please let us know if it works using that kernel!

[Will]

@sebastian-roth Thanks. I will do the test later.

Can you confirm that does this special kernel “bzImage-5.10.19-Lexar_M.2” include NIC driver for Realtek 8125B and Intel I225-V?

[Sebastian Roth]

@will said in M.2 not recognised during deploying stage:

Can you confirm that does this special kernel “bzImage-5.10.19-Lexar_M.2” include NIC driver for Realtek 8125B and Intel I225-V?

Looks like Realtek 8125B is supported since Linux kernel 5.9.x and we also added the firmware blobs needed. As well the IGC kernel driver (discussed here) is also included in bzImage-5.10.19-Lexar_M.2. So I would say both should work. Let is know if it doesn’t.

[Sebastian Roth]

@Will Please let us know if this patched kernel fixed the issue.

[Will]

@sebastian-roth Hi, Sebastian I can confirm this special kernel fixed the Lexar M.2 non detecting issue. Thank you.

[Sebastian Roth]

@Will Thanks for the update. Just found that the patch was actually accepted upstream in kernel 5.10.23. As we will update to newer kernel version as we go this will be in the official FOG kernels as well. No extra patching needed.

[Dinesh]

@sebastian-roth i have the same issue when registering the system having the M.2 SSD. How can i upgrade the kernel 5.10.23. can you please help me to register and deploy the windows 10

FYI - I using the latest version of the fog itself 1.5.9.63 and kernel version is ( 5.10.19)

[george1421]

@dinesh said in M.2 not recognised during deploying stage:

How can i upgrade the kernel 5.10.23

The devs have not posted that public yet.

FWIW I see the official linux kernels are now at 5.10.28.

[Sebastian Roth]

@Dinesh Right, we have not released a newer kernel version yet but you can still download and use the kernel including the patch until we release a new kernel.

[Dinesh]

@sebastian-roth Can you send me the steps to install the patch(kernels/bzImage-5.10.19-Lexar_M.2)

[Sebastian Roth]

@dinesh said in M.2 not recognised during deploying stage:

Can you send me the steps to install the patch(kernels/bzImage-5.10.19-Lexar_M.2)

sudo -i
cd /var/www/fog/service/ipxe/
wget https://fogproject.org/kernels/bzImage-5.10.19-Lexar_M.2
chown www-data:www-data bzImage-5.10.19-Lexar_M.2

If you use a RedHat/CentOS Linux then you need to use apache instead of www-data with the last command above.

Now you have the kernel available and just need to set this as default kernel (FOG Configuration -> FOG Settings -> TFTP Server -> TFTP PXE KERNEL change from bzImage to bzImage-5.10.19-Lexar_M.2)

Note that down so you can switch back to bzImage when the next kernel is officially released and you want to use the default again.