another init.xz issue

Sebastian Roth

@bmaster001 Great to hear you are making progress. If I remember correctly George and Tom have changed some code in the init files lately. Please upgrade to the very latest trunk version and see if you still get that fatal error: unknown request type :: Null thing.

bmaster001

Ok, now I am very confused:

First I updated to the latest trunk version
I tried to choose the first grub-menu item on the FOS stick, with and without a capture task in the server, and both failed with the same error
I tried running “fog” from within the debug menu item -> with the same error
then I noticed that the MAC address that I see with ifconfig is totally different from what I have in the fog server, so I created a new host on the server, with that MAC address, but that doesn’t help either
Then I removed the USB stick, and re-enabled UEFI boot in the bios to check the MAC address, and suddenly the network boot works?!

Did you guys change something in the latest trunk version to make this thing boot correctly?
Is it normal that a pxe boot gives me a different MAC address than an USB-boot ?

Sebastian Roth

@bmaster001 said:

Did you guys change something in the latest trunk version to make this thing boot correctly?

Possibly a newer kernel that does handle this hardware properly? @Tom-Elliott?

Is it normal that a pxe boot gives me a different MAC address than an USB-boot ?

Should not be the case if both are using the same NIC! Would you mind posting both MACs here so we can have a look. Usually MAC addresses have a vendor part and we might shed a light on this if we see the two different addresses.

bmaster001

I already did a lookup for both addresses, and it’s the second one that’s strange because it couldn’t find a manufacturer:

00:40:fd:0a:41:a8 is the one I see when network-booting, which is LXE. That’s correct, because the manufacturer of this VM3 device was called LXE a few years ago
f6:c5:25:ec:b1:ff is the address I see when booting from the usb stick. No vendor can be found for this one… maybe it’s something virtual?

george1421

@bmaster001 said in another init.xz issue:

Ok, now I am very confused:

Did you guys change something in the latest trunk version to make this thing boot correctly?

Tom said that the kernels were updated last monday or tuesday to the latest release. My fos-usb.img file has the older kernels on it. If you built your own then it should have the latest kernels on it.

I did wonder myself if the fos client was picking the wrong network adapter as compared to what was captured during inventory.

bmaster001

I didn’t build the img myself. But apparently the newer kernels seem to help me a lot

The inventory task didn’t work either so I had to create the host manually in the fog server, and I used the MAC address I saw during network boot for that. No idea where the other address comes from. It seems to work though, when I ping from another pc to this device, I see that same mac address in my ARP table. Very strange. EDIT: If you want me to do more tests with your image, just let me know, I have this device on my desk for a few more days

Next problem: there seems to a problem with the NTFS partition on the device, so the capture doesn’t start. I have to boot to windows and run “chkdsk /f”. Can’t do that of course, since I want to capture the image as it is
Quick registration and Full Host Registration still don’t work. After init.xz the screen goes black
memtest gives me “Exec format error”

So we’re not entirely there yet…

bmaster001

I’m now on version 8185, and pxe boot hangs again on “init.xz…ok”. Weird, I’m sure it worked yesterday.
Disabled uefi again, booted the fos usb stick, and noticed that the MAC address is totally different as before. It seems that the mac address is different after each reboot. That makes this fos image pretty hard to use for deployment/capture tasks with fog because it gives the “fatal error: unknown request type :: Null” error each time. @george1421: any idea why this happens? I tried starting from the stich in debug mode, update the host on the server so that it matches the virtual mac, and ran ‘fog’ on the host, but that gave the same error. Maybe this is fixed when the real mac is used when booting … ?

george1421

@bmaster001 This is totally strange, I can’t understand why the mac address is coming from random places. I’m not going to rule out something strange going on in the FOS engine, but this hardware is the only one doing this (so far)

What I’m going to suggest is that you update the kernel and inits on that usb stick. Just download the files using these urls
https://fogproject.org/inits/init.xz
https://fogproject.org/inits/init_32.xz
https://fogproject.org/kernels/bzImage
https://fogproject.org/kernels/bzImage32

And replace the files in the /boot folder on the stick. This will put the latest kernels and inits on that stick.

Once that is done boot from the usb stick and select the debug boot. That will drop you to a command prompt on the device.

Then key in ip addr show and record the network mac addresses
Key in . /usr/share/fog/lib/funcs.sh
Key in set | grep mac
Record the value
Reboot
Test 2 more times. See if the mac address is dynamic (for some reason).

Hopefully some pattern will show up. The mac variable is what fog uses to identify the target to the FOG server.

I’ll tell you a secret for debugging the FOS engine. if you use the ip addr show you can find the IP address of the target. Then if you set a password for the logged in user (root) with passwd you can then use putty to ssh into the FOS engine. This makes it easier to copy / paste take screen shots of what is going on in the FOS environment.

bmaster001

Thanks for the “secret”

First run:

[Tue Jun 21 root@fogclient ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 06:a7:78:e5:c9:ed brd ff:ff:ff:ff:ff:ff
    inet 10.1.14.214/16 brd 10.1.255.255 scope global eth0
       valid_lft forever preferred_lft forever
[Tue Jun 21 root@fogclient ~]# /usr/share/fog/lib/funcs.sh
[Tue Jun 21 root@fogclient ~]# set |grep mac
SHELLOPTS=braceexpand:emacs:hashall:histexpand:history:interactive-comments:monitor:posix

I think the funcs.sh script doesn’t output anything mac-address-related into a variable. I looked in the script, and saw that it runs /sbin/ip which returns basically the same info as ip add show or ifconfig.

Second run:

[Tue Jun 21 root@fogclient ~]# ip add show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 96:c1:60:13:b0:09 brd ff:ff:ff:ff:ff:ff
    inet 10.1.14.248/16 brd 10.1.255.255 scope global eth0
       valid_lft forever preferred_lft forever

Third run:

[Tue Jun 21 root@fogclient ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether ce:82:f1:bb:00:6c brd ff:ff:ff:ff:ff:ff
    inet 10.1.14.190/16 brd 10.1.255.255 scope global eth0
       valid_lft forever preferred_lft forever

I think this confirms what I already saw: the mac address changes on each reboot

george1421

@bmaster001 The actual command for functs is . /usr/share/fog/lib/funcs.sh (dot space <path>). Without the preceding dot the variable are lost when the script quits running. Unfortunately this information is important for debugging since the mac variable is what is passed to FOG. But you are right the mac address of the eth0 changes at random. This is crazy.

what do you get from lspci -m and then lspci -k

bmaster001

oh, I thought that was a type, sorry for that. But it doesn’t help, the set command doesn’t give me anything mac-related:

[Tue Jun 21 root@fogclient ~]# set |grep mac
SHELLOPTS=braceexpand:emacs:hashall:histexpand:history:interactive-comments:monitor:posix
[Tue Jun 21 root@fogclient ~]#

And here is the lspci output:

[Tue Jun 21 root@fogclient ~]# lspci -m
00:00.0 "Host bridge" "Intel Corporation" "Atom Processor Z36xxx/Z37xxx Series SoC Transaction Register" -r11 "Intel Corporation" "Device 7270"
00:02.0 "VGA compatible controller" "Intel Corporation" "Atom Processor Z36xxx/Z37xxx Series Graphics & Display" -r11 "Intel Corporation" "Device 7270"
00:13.0 "SATA controller" "Intel Corporation" "Atom Processor E3800 Series SATA AHCI Controller" -r11 -p01 "Intel Corporation" "Device 7270"
00:17.0 "SD Host controller" "Intel Corporation" "Atom Processor E3800 Series eMMC 4.5 Controller" -r11 -p01 "Intel Corporation" "Device 7270"
00:1a.0 "Encryption controller" "Intel Corporation" "Atom Processor Z36xxx/Z37xxx Series Trusted Execution Engine" -r11 "Intel Corporation" "Device 7270"
00:1b.0 "Audio device" "Intel Corporation" "Atom Processor Z36xxx/Z37xxx Series High Definition Audio Controller" -r11 "Intel Corporation" "Device 7270"
00:1c.0 "PCI bridge" "Intel Corporation" "Atom Processor E3800 Series PCI Express Root Port 1" -r11 "" ""
00:1c.1 "PCI bridge" "Intel Corporation" "Atom Processor E3800 Series PCI Express Root Port 2" -r11 "" ""
00:1c.2 "PCI bridge" "Intel Corporation" "Atom Processor E3800 Series PCI Express Root Port 3" -r11 "" ""
00:1c.3 "PCI bridge" "Intel Corporation" "Atom Processor E3800 Series PCI Express Root Port 4" -r11 "" ""
00:1d.0 "USB controller" "Intel Corporation" "Atom Processor Z36xxx/Z37xxx Series USB EHCI" -r11 -p20 "Intel Corporation" "Device 7270"
00:1f.0 "ISA bridge" "Intel Corporation" "Atom Processor Z36xxx/Z37xxx Series Power Control Unit" -r11 "Intel Corporation" "Device 7270"
00:1f.3 "SMBus" "Intel Corporation" "Atom Processor E3800 Series SMBus Controller" -r11 "Intel Corporation" "Device 7270"
02:00.0 "Network controller" "Qualcomm Atheros" "AR9580 Wireless Network Adapter" -r01 "Qualcomm Atheros" "Device 3123"

george1421

@bmaster001 Interesting, I don’t see the ethernet adapter only wireless.

Wait, are you using a usb ethernet adapter? That may explain why lspci (look for built in hardware) is not showing the ethernet adapter. But that still doesn’t explain why it is creating a random mac address.

bmaster001

It’s not a usb-stick, but I think internally it’s a usb connection. Here’s the output for lsusb:

[Tue Jun 21 root@fogclient ~]# lsusb
Bus 001 Device 002: ID 8087:07e6
Bus 001 Device 001: ID 1d6b:0002
Bus 001 Device 003: ID 0424:2517
Bus 001 Device 004: ID 0424:9514
Bus 001 Device 005: ID 0eef:0001
Bus 001 Device 007: ID 0c2e:0c80
Bus 001 Device 006: ID 0a12:0000
Bus 001 Device 009: ID 0572:141f
Bus 001 Device 008: ID 0424:ec00
Bus 001 Device 010: ID 058f:6387
Bus 001 Device 011: ID 046d:c31c

I went to http://www.linux-usb.org/usb.ids to lookup if it knows the devices, and the 0424:ec00 is a “SMSC9512/9514 Fast Ethernet Adapter”. Don’t know if that helps?

EDIT: dmesg also mentions this device and the mac:

smsc95xx 1-1.2.1:1.0 eth0: register 'smsc95xx' at usb-0000:00:1d.0-1.2.1, smsc95xx USB 2.0 Ethernet, ce:82:f1:bb:00:6c

EDIT2: Check out http://lxr.free-electrons.com/source/drivers/net/usb/smsc95xx.c#L788 …

george1421

@bmaster001 It would be intersting to note, if the dmsg changed on each boot. More precisely the MAC address of the smsc controller.

Since that is a usb nic (in the dock) there is a kernel parameter that you could add to the grub menu to tell the kernel there is a usb nic involved. I can’t remember if that was posted here. I’m going to look back at the thread and see if its here.

Yes it was posted below has_usb_nic=1 Make sure that is on the line for debug and capture / deploy. Not sure if that will help or not. We may have to get one of the developers to look at this thread now that we know what ethernet adapter is in play.

Tom Elliott

@george1421 has_usb_nic=1

bmaster001

Whene I add that, it asks me to unplug and replug my device into the usb port and press enter. Can’t do that, so I just pressed enter

Anyway: the result is the same: another random mac address…

[Tue Jun 21 root@fogclient ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether a6:3a:48:bb:24:65 brd ff:ff:ff:ff:ff:ff
    inet 10.1.14.75/16 brd 10.1.255.255 scope global eth0
       valid_lft forever preferred_lft forever

Have you seen the link to some driver-source I posted above? http://lxr.free-electrons.com/source/drivers/net/usb/smsc95xx.c#L788

george1421

@bmaster001 I saw the kernel driver, but not sure what you wanted to tell me. I assume the driver is built into the current FOS kernel otherwise you would not get an ip address.

I think we need to get the kernel developers to look at this issue. They may need to check to see if the current kernel supports this specific nic or one that is close. The device (hex) ID you provided below will help. I think I’m at the end of my ability to debug this @Developers ??

bmaster001

@george1421 said in another init.xz issue:

@bmaster001 I saw the kernel driver, but not sure what you wanted to tell me. I assume the driver is built into the current FOS kernel otherwise you would not get an ip address.

In that source you can see that the driver tries to get a MAC address from the EEPROM but if that fails, it generates a random one. I thought that it might help to know that it’s the driver who does this, and not some weird config option somewhere…

Another thing that I’ve tried is to boot in debug mode, and manually add a host to the fog server with the random mac. Then schedule a capture task, and run ‘fog’ on the host. It fails with the “unknown request type :: Null” error. Maybe you know if there’s a way around that?

Thanks a lot for you help anyway, I’ve learned a lot! Let’s hope someone else with a deeper knowledge of this stuff is willing to look at this!

FYI: I’ll be on vacation from coming friday until the 3rd of july…

george1421

@bmaster001 Gotcha on the drive. I did not read it line for line. (that is a silly feature)

As for your hack around this. If you look in grub for the debug entry, there is a kernel parameter isdebug=yes If you copy and paste that onto the end of the capture / deploy line. Your capture deploy entry will be in debug mode. Do that boot, and then try your trick of updating the mac address in fog, schedule a capture/deploy. Then on the FOS console key in fog. That might just work.

bmaster001

That seems to work! I’m now stuck on the ntfs inconsistency again that’s not fog’s fault… I’ll have to find a way to fix that too.

Before doing what you described, I took a loog at the contents of /bin/fog . It starts like this:

### If USB Boot device we need a way to get the kernel args properly
if [[ $boottype == usb && ! -z $web ]]; then
    mac=$(getMACAddresses)
    wget -q -O /tmp/hinfo.txt "http://${web}service/hostinfo.php?mac=$mac"
    [[ -f /tmp/hinfo.txt ]] && . /tmp/hinfo.txt
fi $

The $web part seems to be a problem, because that variable doesn’t exist. So it skips the getMACAddresses-part…

another init.xz issue

157

12.0k

17.3k

155.2k