Surface Pro 4 won't get to registration menu

[george1421]

@sarge_212 Just for clarity you are updating the bzImage as well as the inits at the same time?
[Edit] wait… do these kernels work on a regular computer and they are only blowing up on the surface? [/Edit]

[sarge_212]

@george1421 Yes. I downloaded and installed FOG trunk on a separate server. Copied the bzImage, bzImage32, init.xz, init_32.xz to the /var/www/html/service/ipxe folder. Kernels do work on regular computer, yes.

[Sebastian Roth]

@sarge_212 Do you still see the ‘cdc-ether…’ messages as well? This should not be the case if you are running the latest kernels!

Edit: Can you please show us what you see on your modded 1.2.0 server issuing this command: ls -al /var/www/html/service/ipxe/{bzImage,bzImage32,init.xz,init_32.xz}

[sarge_212]

@Sebastian-Roth Here is the output from the modded 1.2.0:

#  ls -al /var/www/html/fog/service/ipxe/{bzImage,bzImage32,init.xz,init_32.xz}
-rw-r--r--. 1 fog apache  6847456 Jan 21 15:20 /var/www/html/fog/service/ipxe/bzImage
-rw-r--r--. 1 fog apache  6759504 Jan 21 15:20 /var/www/html/fog/service/ipxe/bzImage32
-rw-r--r--. 1 fog apache 15448272 Jan 21 15:20 /var/www/html/fog/service/ipxe/init_32.xz
-rw-r--r--. 1 fog apache 16476332 Jan 21 15:20 /var/www/html/fog/service/ipxe/init.xzere

testing again to see if I still get the cdc first

[george1421]

@sarge_212 said:

@george1421 Kernels do work on regular computer, yes.

OK, great. I was afraid we broke you FOG server.

So the root of the issue is that the current kernels and inits are panicking the kernel. I’m wondering if these devices have a weird hard drive structure/ definition that is causing this issue. With that said, it would be interesting to know if we could live boot linux on one of these devices…

[sarge_212]

@george1421 no, no FOG broken here However, it is using the most current inits and kernels I believe as I am no longer getting the cdc-ether error @Sebastian-Roth . That’s a step in the right direction. And @george1421 I do believe they DO have a weird hard drive structure. I think Windows preloads 3 separate partitions on these surfaces:

A recovery partition, system partition, and the actual partition where the OS is.

That being said, I think I’m going to play with blowing away these partition schemes, USB image it, and then try and register the image with the FOG server. That might just do it. You guys have been so helpful!

[Sebastian Roth]

@sarge_212 I really don’t think that partition scheme comes into play at this early stage of loading the kernel! Feel free to do whatever you think is right but I doubt that a simple partition layout will help in loading the kernel any further.

That said you might want to try cranking up the kernel debugging level and maybe we see some more information then.

[sarge_212]

Well shucks.

[Sebastian Roth]

Are you happy to play with the code, as FOG 1.2.0 does not have this included in the WEB GUI yet (just another reason for FOG trunk). Take a look at /var/www/fog/lib/fog/BootMenu.class.php

Search for loglevel=4 and change all (three) to loglevel=7. Plus add debug as well. For example around line 442 would be:

				"osid=$osid",
				"loglevel=7",
				"debug",
				"consoleblank=0",

And line 247:

        print "$this->kernel loglevel=7 debug ".implode ...

[george1421]

@sarge_212 Don’t get discouraged here. You are doing pioneering work that hasn’t been done before with fog and the surface 4. “Some times you will get a little bloody walking on the bleeding edge of technology”

As Sebastian said, go into the fog settings and crank the debug level up to max and see if it will provide any additional information.

It would still be interesting to see if you can get a live linux to boot on this thing. That may be a path if the debugging doesn’t provide any more data. You can live boot linux from a usb flash drive so there won’t be anything installed on the surface. The live linux would give us a chance to run a few commands as well as tell use that booting linux on the tab is possible.

[sarge_212]

@george1421 You guys rock, and thanks for the encouragement. Don’t worry, I’m far from giving up! Sebastian, thanks for the tips on getting a higher level of debugging going, I will do that. Thanks FOG team!

[sarge_212]

I changed the debugging up to level 7 and there isn’t any more output on the surface after I try to register it. Any suggestions on a minimal ISO that I could drop on a USB stick and live boot it to run a few commands? Thanks all!

[george1421]

Similar issue in another thread. https://forums.fogproject.org/topic/6525/pxe-boot-hp-x2-210-hybrid-tablet-windows-10-pro/10

Not sure if they are related or not, but OP in that thread was able to live boot ubuntu.

[Sebastian Roth]

Too bad that we don’t get any more news with higher log level. Tried some google foo (wonder why I didn’t do this earlier) and came up with this:
http://superuser.com/questions/1005825/linux-on-new-surface-pro-4
https://www.reddit.com/r/SurfaceLinux/comments/3zytfw/status_of_arch_linux_on_sp4/
https://www.reddit.com/r/SurfaceLinux/comments/3xhmu7/just_installed_chakra_linux_on_surface_pro_4_i7/
https://www.reddit.com/r/SurfaceLinux/comments/3w8du2/outofthebox_support_for_linux_mint_173_cinnamon/ (see the EDITs on Pro 4)

There seam to be a lot of things that don’t work but I guess most of that is irelevant for cloning. By the way, have you disabled secure boot?

[sarge_212]

Disabling secure boot was one of the first steps as indicated on another forum post about imaging a surface pro 3 so I had done that already. I’ll look into live booting a linux OS on the surface.

[sarge_212]

Also,

What commands should I run when I get that live linux install booted up on the surface?

[sarge_212]

Awesome! I was able to make a USB ubuntu live image and I’ve booted into that and am more than happy to provide some command output. I’m not sure what you guys are looking for. Just let me know, thanks team!

[george1421]

It would be interesting to capture the output of
lspci -m
and
lsblk

Just direct the output into a file and the move the file to some place where you can post the output here. I’m also working on an idea for a debugging tool to. But I’m not quite done yet. But its great you have a live boot system on the surface pro 4, that means there is some hope of success. (!!)

[Wayne Workman]

@george1421 said:

But its great you have a live boot system on the surface pro 4, that means there is some hope of success. (!!)

I want to understand how Ubuntu and other distributions can reliably access persistent storage for whatever device it’s installed onto. Maybe we should look into that? Is there some sort of intelligent code that determines options correctly, or are they just hard-coding for each unique piece of hardware? Or is the Live kernel just compiled with every single option enabled by default?

[george1421]

@Wayne-Workman The answer is easy why the live boot environment can run on a wide variety of hardware. Its much more difficult for an embedded environment because of the limited resources.

@sarge_212 Please excuse this post it will be off point of your issue.

The majority of the issue is trying to keep the kernel size below a specific size. I know when I was building kernels for an embedded device back in the 2.2 to 2.4.37 kernel days we had 512KB of nv storage with 1MB of RAM. The kernel and root file system all gzipped up had to fit on that 512K boot device. When booted syslinux would take that kernel build a ram drive and expand both the kernel and root fs into that 1MB of ram, and leave enough ram left over to run the device. (something similar today is the same process how dd-wrt is booted off consumer (home) internet routers). Well when you make these kernels you generally know the target hardware so you throw out drivers for hardware you know will never exist in your environment. (i.e. why include a driver for an IBM token ring board if you will never use it)

When you build a linux kernel you can decide to either statically link the drivers right into the kernel or build them as dynamically linked modules. How they are linked doesn’t matter (much) since the size of the driver itself is the same. In this case the driver will either be in the kernel itself (i.e. bzImage) or if dynamically linked in the root file system (init.xz).

The idea when pxe booting devices is you want to get the kernel and the root fs to the client in a moderately fast time. So you want the kernel and the inits as small as possible yet still remain functional. Plus the tftp process is not designed (really) to handle large file transfers. Transferring a 500MB file is kind of slow over tftp. That’s why the fog devs use http (one reason why they are using ipxe instead of the standard pxe boot code) to deliver the fog kernels and init to the target computers instead of tftp (there are a few other reasons, but speed is very important).

Just for comparison. On my ubuntu laptop the kernel is 5.8MB and the drivers (/lib/modules/<kernel version>) are almost 140MB. Compare that with FOG’s client OS where the kernel is 6M and the inits are about 16MB (understand 16MB is the kernel drivers plus any programs included in the FOG OS).

So for PXE booting you want the smallest kernel and root fs so it can be delivered quickly over the network. One way to get a small kernel is to only include drivers you expect to see in the target’s environment. So in the end its a balancing act between size, flexibility, and speed.

So ultimately the answer to the question is: Why can live boot OSs work where the FOG OS fails? Its because the live boot OSs have the luxury of a huge nv storage to house every driver the kernel supports, plus they really don’t care how fast the target computer boots. I really haven’t timed it, but the FOG OS appears boots in less than 15 seconds (once you get past all of the ipxe stuff).

Back on point of this thread: If we can identify what kernel drivers this new hardware uses, the devs can consider to include that “required” drive in the kernel or the inits. That is why it is important that if a live boot OS to capture what it sees and properly relay that to the devs for consideration. If a live boot OS fails to boot, the owner of that device is out of luck since the linux kernel itself does not support the hardware.