another init.xz issue

bmaster001

I didn’t build the img myself. But apparently the newer kernels seem to help me a lot

The inventory task didn’t work either so I had to create the host manually in the fog server, and I used the MAC address I saw during network boot for that. No idea where the other address comes from. It seems to work though, when I ping from another pc to this device, I see that same mac address in my ARP table. Very strange. EDIT: If you want me to do more tests with your image, just let me know, I have this device on my desk for a few more days

Next problem: there seems to a problem with the NTFS partition on the device, so the capture doesn’t start. I have to boot to windows and run “chkdsk /f”. Can’t do that of course, since I want to capture the image as it is
Quick registration and Full Host Registration still don’t work. After init.xz the screen goes black
memtest gives me “Exec format error”

So we’re not entirely there yet…

bmaster001

I’m now on version 8185, and pxe boot hangs again on “init.xz…ok”. Weird, I’m sure it worked yesterday.
Disabled uefi again, booted the fos usb stick, and noticed that the MAC address is totally different as before. It seems that the mac address is different after each reboot. That makes this fos image pretty hard to use for deployment/capture tasks with fog because it gives the “fatal error: unknown request type :: Null” error each time. @george1421: any idea why this happens? I tried starting from the stich in debug mode, update the host on the server so that it matches the virtual mac, and ran ‘fog’ on the host, but that gave the same error. Maybe this is fixed when the real mac is used when booting … ?

george1421

@bmaster001 This is totally strange, I can’t understand why the mac address is coming from random places. I’m not going to rule out something strange going on in the FOS engine, but this hardware is the only one doing this (so far)

What I’m going to suggest is that you update the kernel and inits on that usb stick. Just download the files using these urls
https://fogproject.org/inits/init.xz
https://fogproject.org/inits/init_32.xz
https://fogproject.org/kernels/bzImage
https://fogproject.org/kernels/bzImage32

And replace the files in the /boot folder on the stick. This will put the latest kernels and inits on that stick.

Once that is done boot from the usb stick and select the debug boot. That will drop you to a command prompt on the device.

Then key in ip addr show and record the network mac addresses
Key in . /usr/share/fog/lib/funcs.sh
Key in set | grep mac
Record the value
Reboot
Test 2 more times. See if the mac address is dynamic (for some reason).

Hopefully some pattern will show up. The mac variable is what fog uses to identify the target to the FOG server.

I’ll tell you a secret for debugging the FOS engine. if you use the ip addr show you can find the IP address of the target. Then if you set a password for the logged in user (root) with passwd you can then use putty to ssh into the FOS engine. This makes it easier to copy / paste take screen shots of what is going on in the FOS environment.

bmaster001

Thanks for the “secret”

First run:

[Tue Jun 21 root@fogclient ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 06:a7:78:e5:c9:ed brd ff:ff:ff:ff:ff:ff
    inet 10.1.14.214/16 brd 10.1.255.255 scope global eth0
       valid_lft forever preferred_lft forever
[Tue Jun 21 root@fogclient ~]# /usr/share/fog/lib/funcs.sh
[Tue Jun 21 root@fogclient ~]# set |grep mac
SHELLOPTS=braceexpand:emacs:hashall:histexpand:history:interactive-comments:monitor:posix

I think the funcs.sh script doesn’t output anything mac-address-related into a variable. I looked in the script, and saw that it runs /sbin/ip which returns basically the same info as ip add show or ifconfig.

Second run:

[Tue Jun 21 root@fogclient ~]# ip add show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 96:c1:60:13:b0:09 brd ff:ff:ff:ff:ff:ff
    inet 10.1.14.248/16 brd 10.1.255.255 scope global eth0
       valid_lft forever preferred_lft forever

Third run:

[Tue Jun 21 root@fogclient ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether ce:82:f1:bb:00:6c brd ff:ff:ff:ff:ff:ff
    inet 10.1.14.190/16 brd 10.1.255.255 scope global eth0
       valid_lft forever preferred_lft forever

I think this confirms what I already saw: the mac address changes on each reboot

george1421

@bmaster001 The actual command for functs is . /usr/share/fog/lib/funcs.sh (dot space <path>). Without the preceding dot the variable are lost when the script quits running. Unfortunately this information is important for debugging since the mac variable is what is passed to FOG. But you are right the mac address of the eth0 changes at random. This is crazy.

what do you get from lspci -m and then lspci -k

bmaster001

oh, I thought that was a type, sorry for that. But it doesn’t help, the set command doesn’t give me anything mac-related:

[Tue Jun 21 root@fogclient ~]# set |grep mac
SHELLOPTS=braceexpand:emacs:hashall:histexpand:history:interactive-comments:monitor:posix
[Tue Jun 21 root@fogclient ~]#

And here is the lspci output:

[Tue Jun 21 root@fogclient ~]# lspci -m
00:00.0 "Host bridge" "Intel Corporation" "Atom Processor Z36xxx/Z37xxx Series SoC Transaction Register" -r11 "Intel Corporation" "Device 7270"
00:02.0 "VGA compatible controller" "Intel Corporation" "Atom Processor Z36xxx/Z37xxx Series Graphics & Display" -r11 "Intel Corporation" "Device 7270"
00:13.0 "SATA controller" "Intel Corporation" "Atom Processor E3800 Series SATA AHCI Controller" -r11 -p01 "Intel Corporation" "Device 7270"
00:17.0 "SD Host controller" "Intel Corporation" "Atom Processor E3800 Series eMMC 4.5 Controller" -r11 -p01 "Intel Corporation" "Device 7270"
00:1a.0 "Encryption controller" "Intel Corporation" "Atom Processor Z36xxx/Z37xxx Series Trusted Execution Engine" -r11 "Intel Corporation" "Device 7270"
00:1b.0 "Audio device" "Intel Corporation" "Atom Processor Z36xxx/Z37xxx Series High Definition Audio Controller" -r11 "Intel Corporation" "Device 7270"
00:1c.0 "PCI bridge" "Intel Corporation" "Atom Processor E3800 Series PCI Express Root Port 1" -r11 "" ""
00:1c.1 "PCI bridge" "Intel Corporation" "Atom Processor E3800 Series PCI Express Root Port 2" -r11 "" ""
00:1c.2 "PCI bridge" "Intel Corporation" "Atom Processor E3800 Series PCI Express Root Port 3" -r11 "" ""
00:1c.3 "PCI bridge" "Intel Corporation" "Atom Processor E3800 Series PCI Express Root Port 4" -r11 "" ""
00:1d.0 "USB controller" "Intel Corporation" "Atom Processor Z36xxx/Z37xxx Series USB EHCI" -r11 -p20 "Intel Corporation" "Device 7270"
00:1f.0 "ISA bridge" "Intel Corporation" "Atom Processor Z36xxx/Z37xxx Series Power Control Unit" -r11 "Intel Corporation" "Device 7270"
00:1f.3 "SMBus" "Intel Corporation" "Atom Processor E3800 Series SMBus Controller" -r11 "Intel Corporation" "Device 7270"
02:00.0 "Network controller" "Qualcomm Atheros" "AR9580 Wireless Network Adapter" -r01 "Qualcomm Atheros" "Device 3123"

george1421

@bmaster001 Interesting, I don’t see the ethernet adapter only wireless.

Wait, are you using a usb ethernet adapter? That may explain why lspci (look for built in hardware) is not showing the ethernet adapter. But that still doesn’t explain why it is creating a random mac address.

bmaster001

It’s not a usb-stick, but I think internally it’s a usb connection. Here’s the output for lsusb:

[Tue Jun 21 root@fogclient ~]# lsusb
Bus 001 Device 002: ID 8087:07e6
Bus 001 Device 001: ID 1d6b:0002
Bus 001 Device 003: ID 0424:2517
Bus 001 Device 004: ID 0424:9514
Bus 001 Device 005: ID 0eef:0001
Bus 001 Device 007: ID 0c2e:0c80
Bus 001 Device 006: ID 0a12:0000
Bus 001 Device 009: ID 0572:141f
Bus 001 Device 008: ID 0424:ec00
Bus 001 Device 010: ID 058f:6387
Bus 001 Device 011: ID 046d:c31c

I went to http://www.linux-usb.org/usb.ids to lookup if it knows the devices, and the 0424:ec00 is a “SMSC9512/9514 Fast Ethernet Adapter”. Don’t know if that helps?

EDIT: dmesg also mentions this device and the mac:

smsc95xx 1-1.2.1:1.0 eth0: register 'smsc95xx' at usb-0000:00:1d.0-1.2.1, smsc95xx USB 2.0 Ethernet, ce:82:f1:bb:00:6c

EDIT2: Check out http://lxr.free-electrons.com/source/drivers/net/usb/smsc95xx.c#L788 …

george1421

@bmaster001 It would be intersting to note, if the dmsg changed on each boot. More precisely the MAC address of the smsc controller.

Since that is a usb nic (in the dock) there is a kernel parameter that you could add to the grub menu to tell the kernel there is a usb nic involved. I can’t remember if that was posted here. I’m going to look back at the thread and see if its here.

Yes it was posted below has_usb_nic=1 Make sure that is on the line for debug and capture / deploy. Not sure if that will help or not. We may have to get one of the developers to look at this thread now that we know what ethernet adapter is in play.

Tom Elliott

@george1421 has_usb_nic=1

bmaster001

Whene I add that, it asks me to unplug and replug my device into the usb port and press enter. Can’t do that, so I just pressed enter

Anyway: the result is the same: another random mac address…

[Tue Jun 21 root@fogclient ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether a6:3a:48:bb:24:65 brd ff:ff:ff:ff:ff:ff
    inet 10.1.14.75/16 brd 10.1.255.255 scope global eth0
       valid_lft forever preferred_lft forever

Have you seen the link to some driver-source I posted above? http://lxr.free-electrons.com/source/drivers/net/usb/smsc95xx.c#L788

george1421

@bmaster001 I saw the kernel driver, but not sure what you wanted to tell me. I assume the driver is built into the current FOS kernel otherwise you would not get an ip address.

I think we need to get the kernel developers to look at this issue. They may need to check to see if the current kernel supports this specific nic or one that is close. The device (hex) ID you provided below will help. I think I’m at the end of my ability to debug this @Developers ??

bmaster001

@george1421 said in another init.xz issue:

@bmaster001 I saw the kernel driver, but not sure what you wanted to tell me. I assume the driver is built into the current FOS kernel otherwise you would not get an ip address.

In that source you can see that the driver tries to get a MAC address from the EEPROM but if that fails, it generates a random one. I thought that it might help to know that it’s the driver who does this, and not some weird config option somewhere…

Another thing that I’ve tried is to boot in debug mode, and manually add a host to the fog server with the random mac. Then schedule a capture task, and run ‘fog’ on the host. It fails with the “unknown request type :: Null” error. Maybe you know if there’s a way around that?

Thanks a lot for you help anyway, I’ve learned a lot! Let’s hope someone else with a deeper knowledge of this stuff is willing to look at this!

FYI: I’ll be on vacation from coming friday until the 3rd of july…

george1421

@bmaster001 Gotcha on the drive. I did not read it line for line. (that is a silly feature)

As for your hack around this. If you look in grub for the debug entry, there is a kernel parameter isdebug=yes If you copy and paste that onto the end of the capture / deploy line. Your capture deploy entry will be in debug mode. Do that boot, and then try your trick of updating the mac address in fog, schedule a capture/deploy. Then on the FOS console key in fog. That might just work.

bmaster001

That seems to work! I’m now stuck on the ntfs inconsistency again that’s not fog’s fault… I’ll have to find a way to fix that too.

Before doing what you described, I took a loog at the contents of /bin/fog . It starts like this:

### If USB Boot device we need a way to get the kernel args properly
if [[ $boottype == usb && ! -z $web ]]; then
    mac=$(getMACAddresses)
    wget -q -O /tmp/hinfo.txt "http://${web}service/hostinfo.php?mac=$mac"
    [[ -f /tmp/hinfo.txt ]] && . /tmp/hinfo.txt
fi $

The $web part seems to be a problem, because that variable doesn’t exist. So it skips the getMACAddresses-part…

george1421

@bmaster001 When you enter in debug mode none of the fog scripts have run. So the $web won’t be set until . /usr/share/fog/lib/funcs.sh has been called. I know it is confusing, but I ran into the same issue when developing the code snippet you posted.

$web should point to your fog server when its set. Actually <fog_server_ip>/fog (this comes from the kernel parameter in the GRUB config file). The funcs.sh script extracts the kernel parameters and then sets the bash variables to match the kernel parameters. Then my code calls the fog server to pick up the other dynamic kernel parameters than can’t be supplied by the static grub.cfg file. That is where the hostinfo.php comes in. The type variable is set by the hostinfo.php script, fwiw.

bmaster001

It’s a difficult mechanism to understand. A lot of unfamiliar technologies, scripts and boot-arguments working together. It’ll take a while for me to better understand it. But it’s a very interesting subject!

The very first line in the fog script is the call to funcs.sh. when I run that on the command line, and do set, I see no web variable:

_=/usr/share/fog/lib/funcs.sh
acpi=off
boottype=usb
consoleblank=0
has_usb_nic=1
init=/sbin/init
isdebug=yes
ismajordebug=0
loglevel=7
name=has_usb_nic
oIFS='
'
ramdisk_size=127000
root=/dev/ram0
rootfstype=ext4
value=1
var=has_usb_nic=1

george1421

@bmaster001 hey that’s cool (not).

Lets confirm that your grub.cfg has this line.

menuentry "1. FOG Image Deploy/Capture" {
 echo loading the kernel
 linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=127000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 isdebug=yes
 echo loading the virtual hard drive
 initrd $myinits
 echo booting kernel...
}

Note the kerenel parameter web=$myfogip/fog/

[edit] note web will only be there when you are in capture/deploy menu mode not in the debug mode from the menu entry.

george1421

@bmaster001 said in another init.xz issue:

It’s a difficult mechanism to understand. A lot of unfamiliar technologies, scripts and boot-arguments working together. It’ll take a while for me to better understand it. But it’s a very interesting subject!

I understand this (linux) is a alien world to most windows folks. But you see there IS a lot of magic that goes into make FOG functional and universal.

You have the unfortunate situation of having very strange hardware (maybe one of a kind) that make imaging difficult to manage. FOG has a lot of instruments in the band, its takes a bit of cohering to get them to all play the same tune.

Sebastian Roth

@bmaster001 said:

In that source you can see that the driver tries to get a MAC address from the EEPROM but if that fails, it generates a random one. I thought that it might help to know that it’s the driver who does this, and not some weird config option somewhere…

Great find! This is really unusual. Have never seen this before! Why would if not be able to read the MAC I wonder?!? Random MAC is kind of breaking the whole FOG logic of identifying hosts via MAC address.

another init.xz issue

222

11.7k

17.2k

154.7k