Surface Pro 4 won't get to registration menu

sarge_212

Awesome! I was able to make a USB ubuntu live image and I’ve booted into that and am more than happy to provide some command output. I’m not sure what you guys are looking for. Just let me know, thanks team!

george1421

It would be interesting to capture the output of
lspci -m
and
lsblk

Just direct the output into a file and the move the file to some place where you can post the output here. I’m also working on an idea for a debugging tool to. But I’m not quite done yet. But its great you have a live boot system on the surface pro 4, that means there is some hope of success. (!!)

Wayne Workman

@george1421 said:

But its great you have a live boot system on the surface pro 4, that means there is some hope of success. (!!)

I want to understand how Ubuntu and other distributions can reliably access persistent storage for whatever device it’s installed onto. Maybe we should look into that? Is there some sort of intelligent code that determines options correctly, or are they just hard-coding for each unique piece of hardware? Or is the Live kernel just compiled with every single option enabled by default?

george1421

@Wayne-Workman The answer is easy why the live boot environment can run on a wide variety of hardware. Its much more difficult for an embedded environment because of the limited resources.

@sarge_212 Please excuse this post it will be off point of your issue.

The majority of the issue is trying to keep the kernel size below a specific size. I know when I was building kernels for an embedded device back in the 2.2 to 2.4.37 kernel days we had 512KB of nv storage with 1MB of RAM. The kernel and root file system all gzipped up had to fit on that 512K boot device. When booted syslinux would take that kernel build a ram drive and expand both the kernel and root fs into that 1MB of ram, and leave enough ram left over to run the device. (something similar today is the same process how dd-wrt is booted off consumer (home) internet routers). Well when you make these kernels you generally know the target hardware so you throw out drivers for hardware you know will never exist in your environment. (i.e. why include a driver for an IBM token ring board if you will never use it)

When you build a linux kernel you can decide to either statically link the drivers right into the kernel or build them as dynamically linked modules. How they are linked doesn’t matter (much) since the size of the driver itself is the same. In this case the driver will either be in the kernel itself (i.e. bzImage) or if dynamically linked in the root file system (init.xz).

The idea when pxe booting devices is you want to get the kernel and the root fs to the client in a moderately fast time. So you want the kernel and the inits as small as possible yet still remain functional. Plus the tftp process is not designed (really) to handle large file transfers. Transferring a 500MB file is kind of slow over tftp. That’s why the fog devs use http (one reason why they are using ipxe instead of the standard pxe boot code) to deliver the fog kernels and init to the target computers instead of tftp (there are a few other reasons, but speed is very important).

Just for comparison. On my ubuntu laptop the kernel is 5.8MB and the drivers (/lib/modules/<kernel version>) are almost 140MB. Compare that with FOG’s client OS where the kernel is 6M and the inits are about 16MB (understand 16MB is the kernel drivers plus any programs included in the FOG OS).

So for PXE booting you want the smallest kernel and root fs so it can be delivered quickly over the network. One way to get a small kernel is to only include drivers you expect to see in the target’s environment. So in the end its a balancing act between size, flexibility, and speed.

So ultimately the answer to the question is: Why can live boot OSs work where the FOG OS fails? Its because the live boot OSs have the luxury of a huge nv storage to house every driver the kernel supports, plus they really don’t care how fast the target computer boots. I really haven’t timed it, but the FOG OS appears boots in less than 15 seconds (once you get past all of the ipxe stuff).

Back on point of this thread: If we can identify what kernel drivers this new hardware uses, the devs can consider to include that “required” drive in the kernel or the inits. That is why it is important that if a live boot OS to capture what it sees and properly relay that to the devs for consideration. If a live boot OS fails to boot, the owner of that device is out of luck since the linux kernel itself does not support the hardware.

Wayne Workman

@george1421 But the kernels and inits are not loaded every time a computer network boots, they only load when there is a task. And just for the record, I’m fine with longer loading times if it means better support for larger range of systems. Specifically storage and network interface drivers.

Thanks for the explanation though. Very well written, and I understand the kernel and drivers a lot better now.

sarge_212

Okay FOG team,

Here is the output of those 2 commands on a Surface PRO 4, see attached…0_1453757795337_deviceinfo.txt

Thanks, I do have a live Ubuntu running now, let me know if there is anything else I can provide

george1421

@sarge_212 (this is a repost from another thread, but I want to see if it works with this hardware too)

If you are willing and have a general knowledge of linux, I would like you to try something.

I’ve been working on an idea about booting the FOG Client kernel (the bits of FOG code that run on the target computer to load or capture images) using a usb drive. Understand this is only for debugging purposes. Right now it is not clear in my mind if the booting issues with these newer efi based systems are with the PXE boot kernel, the FOG Client kernel, or the hand off between ipxe and the FOG kernel.

To test this I wrote a document to create a uefi boot media that will boot the fog kernel from USB. You can not capture or deploy using this kernel only access command line tools. But the point of this exercise is to see if it boots, period.

If you have a spare 2GB or larger usb flash drive, are comfortable with some linux commands and have a little time; could you follow the instructions outlined here: https://forums.fogproject.org/topic/6532/usb-boot-target-device-into-fog-debug-os/3

For the sake of the test, I only want you to execute what appears id method #3. Understand these instructions were written for ubuntu, but they should translate to Centos 7 without too much headache ( ) . At the bottom there is a copy and paste section where you can copy the commands paste them into a text editor, change the reference to the usb flash drive and then just execute the commands one after another through a remote terminal. It takes me less and 5 minutes to execute the commands once you have the rpm files installed.

For the flash drive start with a freshly formatted flash drive using a windows computer and format it fat32 (note the first step of my instructions tells linux to reformat the drive, any existing content will be lost). If you have any questions about this, please post back here.

What I want to test:

Does it boot?
IF it doesn’t boot is there an error message?
Does this image boot on any other device you have that is in uefi mode?

After this test we will add in the grub insmod commands and test again. But first lets see if you can build the boot drive and get it to boot on this table as is.

Sebastian Roth

@sarge_212 Would you be able to upload the full output of dmesg as a text file here if you get up your ubuntu live stick again. Or if you can get George’s FOG debug system (I really like this approach and hope that we can use this even more and maybe simplify it in the near future) that would be a great progress.

Looked throught the outputs you posted and I can’t see anything in particular that I would think is causing the issue.

sarge_212

@Sebastian-Roth I am in the process of making a live debug fog usb stick now. I’ll let you guys know of the status when I get there. Thanks for all the help so far!

sarge_212

Here is the dmesg output:

0_1453846418443_dmesg.txt I’m testing the FOG debugging tool now and seeing if it will boot.

sarge_212

Okay – here is the output of both the 32-bit and 64-bit debug kernel mode selections:

error: no suitable video mode found.
Booting to blind mode
_

Is there a way for me to make the debugging go on no video mode? It isn’t accepting keyboard input at this point so I think it stops here.

I can however hit “c” at the boot menu and get to grub command shell. If there is anything there I can run to provide insight, I’d be more than happy to do so.

george1421

@sarge_212 said:

Okay – here is the output of both the 32-bit and 64-bit debug kernel mode selections:

error: no suitable video mode found.
Booting to blind mode
_

Is there a way for me to make the debugging go on no video mode? It isn’t accepting keyboard input at this point so I think it stops here.

This is the same error I saw with my testing so this is good (kind of). Now I want you to edit the grub.cfg file and change the 64 bit section. to add the insmod command to add video support via grub.

menuentry "FOG 64-bit Debug Kernel" {
 insmod all_video

 linux  /boot/bzImage loglevel=7 init=/sbin/init root=/dev/ram0 rw ramdisk_size=127000 pcie_aspm=off consoleblank=0 isdebug=yes
 initrd /boot/init.xz
}

george1421

What we are trying to identify is if this issue with the Surface Pro 4 is with the FOG kernel or the hand off between ipxe and the fog kernel. Right now we are not sure which part is falling down. If we can usb boot the fog kernel and it works in debug mode then we are pretty sure where the problem isn’t.

Thank you for giving us a hand with this issue, its bigger than just the surface pro. The uefi booting is here to stay, so we have to work it out.

sarge_212

So,

It booted and gave a FOG menu with some items to choose from. This is further than I’ve got in a while. Here is a shot of what it looks like:

sarge_212

@george1421

No worries at all on helping out. I’m glad to provide some help in the development of FOG. FOG is a great tool and we love using it in our environment, and it really rocks, as do you gents!

george1421

@sarge_212 So for the fog menu info if you press enter a few times, you should end up at a command prompt.

This is GREAT! on the positive side the fog kernel does work for the surface pro 4. Now we need to identify why the hand-off from ipxe to the fog kernel is failing, but at least the devs have a direction!!

Thank you for your help with this.

sarge_212

Cool. I’m ready to run some command-fu on this thing and provide output as needed.

george1421

@sarge_212 I hate to keep going back to the well on this one. But do you have a working EFI system that does do the hand off between ipxe and the fog kernel without issue?

What I’m looking for is if you have one of these systems, does the fog debug flash drive boot without the insmod command? We are trying to see if the insmod video… fixes the grub boot. Where some systems don’t need this command and some do, and the ones that don’t need the insmod command boot correctly via ipxe into the fog kernel. (wow that sounding like a lot of words to explain a simple process). I can tell you for the Dell e6220 I have, it requires the insmod command and does not boot in efi via ipxe.

sarge_212

@george1421 I can certainly try it. I have some old hardware but not sure what to test on. We have several generations of Dell Latitudes if that could be of use. E6420, E6430 and the like. We also are using optiplex 9020s in our environs so I can test on those. Isn’t it possible to switch from BIOS to UEFI as well in the setup of the dell systems? I’ll have a look at that. So basically I’ll do the following:

EFI boot to see what happens on a system if it is the same behavior as the surface

I’ll check to see if the hand off works but I will need to switch back the kernels to the trunk kernels. || sidenote: when I needed to image a machine for whatever reason, the new kernels{bzImage, bzImage32, init.xz, init_32.xz}, didn’t work. I got some errors imaging with the new kernels on the 1.2.0 fog (it’s that bad karma of mixing kernels) I’m toying with the idea of creating another FOG server for TRUNK version to see what it would do and how I could image from that. However, for now I’ll mess with the UEFI settings and see how far I get.

george1421

@sarge_212 Don’t go through extra effort if you don’t have any systems you know of that pxe boot properly today in efi mode. I have the same systems in my test lab and will test them tomorrow with the latest fog build. Don’t disrupt your current production environment.

I’m just trying to track down a ipxe efi booting system that is successful today. Then try to boot the debug kernel without the insmod video command. This will tell us if the insmod video is the true issue or just happens to fix a different issue.

Surface Pro 4 won't get to registration menu

229

11.6k

17.1k

154.6k