SOLVED ipxe booting "waiting for the link to come up"


  • Hi,

    Recently installed fog server on my domain. I use my own dhcp (pfsense) and not fog’s dhcp. I was able to register and image a big number of different computers but today i was trying to register a tiny Toshiba Satellite pro.

    After pxe booting, fog’s menu shows up. Then i choose quick register and after that it shows “starting enp3s0 interface and waiting link to come up” after that its says “No link detectet on enp3s0m, skipping it. Failed to get an ip via DHCP!”
    Thats weird because the laptop was able to get an IP and boot to fog’s server menu.

    Any suggestion? im stuck there.

    thanks in advance.


  • @sebastian-roth Well, ive done it already!

    Thanks both of you for your help! I could never been able to figure it out without you! Thanks!

  • Senior Developer

    @kvothe88 said in ipxe booting "waiting for the link to come up":

    So my question now is…is it possible to download this kernel and make ir usable just for this laptop? and use the newest kernels on the rest of computers¿

    Great you found a kernel that works. Sure you can! Install the kernel you want to use for all your systems. Then manually download the old kernel from our website and put that into /var/www/html/fog/serive/ipxe/ directory on your FOG server. Just keep the name Kernel.TomElliott.4.18.11.64 as downloaded. Not exactly sure if this needs a 64 bit or 32 bit kernel. Do you remember if it loaded bzImage (64 bit) or bzImage32 (32 bit) when PXE booting into a task?

    Now edit the host settings of your Toshiba device and set option Host Kernel to Kernel.TomElliott.4.18.11.64 (or …32). When you boot the device up into a task you should clearly see it loading that kernel binary instead of the default bzImage(32).


  • @george1421 said in ipxe booting "waiting for the link to come up":

    e if downgrading your linu

    Hi Gorge, and @sebastian-roth

    thanks for your reply. I tried all 4.19.x kernels and never worked. Also tried 5.6.x kernel and never worked neither, but…i gave a last shoot to 4.18.11 kernel and BOOOOM! It worked!!

    So my question now is…is it possible to download this kernel and make ir usable just for this laptop? and use the newest kernels on the rest of computers¿

    Thanks for everything!

  • Senior Developer

    @kvothe88 @george1421 It’s been busy at work and so I only just saw all your posts. Interesting. From the very first picture you posted we see that enp3s0 is there but says NO-CARRIER and so it seems to not be able to detect the link to be up. Firmware blob does not seem to be an issue from the information posted so far.

    Let’s try disabling auto-negotiation for testing to see if that makes a difference. Either do that on the switch if you can or boot into a debug session and run ethtool -s enp3s0 autoneg off - then wait for a bit and check with ip a s to see if it the NO-CARRIER goes away…

  • Moderator

    @kvothe88 After much head scratching I understand what is going on here. I don’t have a solution atm, but at least I have an idea.

    Realtek has a generic network driver called the r8169. That supports (mostly) a large range of network adapters.

    Looking at the boot log of the file you sent me I see this:

    r8169 0000:03:00.0 eth0: RTL8106e, 60:02:92:3e:ab:7a, XID 449, IRQ 91
    

    So the real nic is an RTL8106 not an r8168/r8169.

    I also found a reference that there is a bug in the R8169 kernel after version 5.4 and the recommendation was to use the 4.19.x series kernel.

    ref: https://archived.forum.manjaro.org/t/realtek-rtl810xe-ethernet-not-working-after-kernel-5-4-31/135945/4

    it appears the issue was fixed in 5.4.33 (so it should be fixed in 5.6.18).

    So what can we do quickly? See if downgrading your linux kernel to 4.19.x allows the system to boot.


  • @george1421 hi George,

    Yes that’s the Mac. Interface is not able to get ip so that makes sense…

    Maybe 5, 6 years old

  • Moderator

    @kvothe88 Just to confirm this is the real mac address of the network adapter in question “60:02:92:3e:ab:7a” ?? The log file is saying the link is down.

    Also how old is this computer?


  • @george1421 hi George, thanks for your reply. Here you have the file:

    messages.txt

  • Moderator

    @kvothe88 Sorry I’m trying to do too many things this AM. Yes scp should have worked, but we have an issue with the NETWORK. So no network functions.

    OK will the updated realtek drives in the kernel I sent did not fix the issue. Its still pointing back to the firmware.

    If you have a usb flash drive we can use that to get the messages file out of the pxe booted computer.

    1. Insert the usb drive into the target computer.
    2. issue this command from the debug console
      lsblk
    3. I identify the usb flash drive by its size in the list. For the reset of this lets assume it shows up and /dev/sdb
    4. Now let make a directory where we can mount that usb drive.
      mkdir /ext
    5. Lets connect the usb drive to that directory
      mount /dev/sdb1 /ext
    6. That command will mount the first partition on that usb stick to the /ext directory. Issue this command to see the contents of the usb drive
      ls -la /ext
    7. If you see what is expected then copy the log file to the use flash drive directory.
      cp /var/log/messages /ext
    8. When done unmount that usb drive with
      unmount /ext
    9. Now you will be able to remove the usb drive. Don’t remove it before you unmount it and the command completes.

  • @george1421 said in ipxe booting "waiting for the link to come up":

    bzImageRT3

    Hi George! Thanks again.

    So, i was not able to move the file to my fog server, that was the error:

    aee40515-2de8-4c83-9713-23fcc8dbfe59.jpg

    Again, i was able to register the host on the fogserver UI. I copied the kernel where you told me and i booted from pxe. Tried the compatibility mode and the bzImageRT3 was using was OK. Got some errors about dell_smbios but unfortunatly the link error showed again:

    f421a513-280b-4ba0-97ad-188cf122981e.jpg

  • Moderator

    @kvothe88 I think I would like to personally see that log file. You can use the scp command on the pxe booting client computer to copy the file to you fog server from there upload it to this thread with a .txt file extension. The command is similar to this scp /var/log/messages root@192.168.1.10:/images Of course change the user and IP address to match your environment. The command will drop the file into /images on the fog server.

    ok then lets try one of the one-off kernels that have the updated realtek driver in it.

    https://drive.google.com/file/d/1vSu5L-DAZYK7VYiJtFfCYrrqJb963cMg/view?usp=sharing

    Download that file from the link and save it as bzImageRT3 in /var/www/html/fog/service/ipxe directory.

    Manually register this host with the fog server using the fog web ui then in the host registration page, set the kernel field to bzImageRT3. Save the settings and then pxe boot the target computer into the hardware compatibility check. See if it detects the NIC there.

    Also, just so we know the scope, approx how old is this laptop?


  • @george1421 hi Gorge,

    got nothing from that file. No /Failed strings shown, no /failed, no /error or /firmware either with capital letter

    3593a64e-f558-4a4c-8f11-fe83bc85c003.jpg 548e42a3-6baf-4d92-bcb5-548dfa1c4ec4.jpg

  • Moderator

    @kvothe88 said in ipxe booting "waiting for the link to come up":

    On toshiba, on the same consolore where i ran the debug task, there is not a syslog file

    Yes on pxe booting computer there should be a log file in /var/log its either called syslog or messages. I can’t remember at the moment which.

    I see that the right kernel driver is being used. “r8169”. So its either missing the proper firmware (need info from above) or I need to send you an updated FOS Linux kernel that has the current realtek driver. Lets find out about the firmware bit first.


  • @george1421

    here you have all i could get from your question:

    ac1b412c-c780-4442-bdc0-2ac085ba08b3.jpg

    227459d6-d431-4b27-a7a6-be8d85e86e6a.jpg


  • @george1421 said in ipxe booting "waiting for the link to come up":

    vi /var/log/syslog

    vi /var/log/syslog yu mean in the Toshiba laptop or in my fog server? On toshiba, on the same consolore where i ran the debug task, there is not a syslog file

  • Moderator

    @kvothe88 The driver should be in the FOS Linux kernel because it is a very common one. Sometimes the nic card requires a specific driver.

    This is a bit harder way to look if you don’t know vi commands but key in vi /var/log/syslog

    Then key in /Failed and press Enter (case is important). That will take you to the first occurrence of the word “Failed” in the log. If that is not it press / and enter again to go to the next “Failed” word. If you searched through the whole log and did not find an mention of missing fw (firmware) then that isn’t the problem. To exit vi key in ESC q!

    This one will give us a bit more detail on the lspci command key in the following: lspci -nn -k | more Scroll down to where you see the line starting with 03:00 There should be two additional lines below that we are interested in. Does it list a kernel module being used?

    Here is an example of what I’m looking for. This is from my FOG server of course, the kernel values is what is needed.

    0b:00.0 Ethernet controller [0200]: VMware VMXNET3 Ethernet Controller [15ad:07b
    0] (rev 01)
            Subsystem: VMware VMXNET3 Ethernet Controller [15ad:07b0]
            Kernel driver in use: vmxnet3
            Kernel modules: vmxnet3
    

  • @george1421

    Hi Gorge, could not find nothing similar to this in that log. Its very big and i could miss something but I checked it twice and saw nothing similar…

    Could those drivers been missing? thanks

  • Moderator

    @kvothe88 Realtek nic 10ec:8136 first added to the linux kernel in version 3.3 so the kernel fog uses (even an old one) should support this nic. I’m going to think that FOS Linux is missing a specific firmware patch for that nic.

    Run this command from the debug console dmesg | less Look through that log file for specifically things that say “failed to find fw file”. It should list a file path and file name. Send us a screen shot of the missing firmware file.


  • @sebastian-roth said in ipxe booting "waiting for the link to come up":

    nd might come ba

    hi, thanks for the fast reply. Here you have the image. For the first command i had no results:

    0e07fa46-7dbd-4e59-9e1f-4b622c8b34fd.jpg

345
Online

8.2k
Users

15.0k
Topics

141.5k
Posts