bnx2x fails to load firmware on Dell R430



  • Hello,

    I’m having a error after the Fog Menu Register Host, I think its with the kernel, everything works fine on other macines and the Dell R430 pxe boots and loads the menu’s fine, when its trying to do the kernel pxe it keeps saying cannot load bnx2x firmware and then cycles through all the ethernet adapters on the server failing to load firmware. I was on RC9 and I found a post that said they fixed it in the next alpha release but the fix doesn’t work for me. any help would be appreciated, as we have a few of these servers.

    Thu Sep 15, 2016 18:44 pm
    Running Version 1.3.0-RC-10
    SVN Revision: 5955

    Official Published Kernels
    Kernel - 4.7.3 TomElliott
    Date : September 13, 2016
    Version : 4.7.3
    FOG Type: TomElliott
    Arch Type: (x86_64)


  • Moderator

    @Sebastian-Roth You are right that firmware is for the fiber (10G) adapter. While it did fix the error in the log its not adding any real value for the OP or for FOG in general. But too, I’m still not seeing FOG as a good tool for cloning servers because of the other mix of hardware (you won’t find in desktops). I would say remove the firmware and have the OP test again now that its working in his environment (because the other roadblocks have been removed).


  • Developer

    @grey said:

    Toms fix on the firmware of the drivers and Georges portfast suggestion solved the last of my network issues.

    I don’t think the added firmware binaries made any difference in this case. As we can see in the dmesg output further down the connected NIC eth5 is managed by the tg3 driver…
    @Tom-Elliott How much did those drivers add to the kernel size? We might think about removing those again as I have never seen anyone PXE booting on fiber cards. What do you think? @george1421?


  • Senior Developer

    @george1421 I’m fairly sure they will both pass.



  • @george1421

    I will do that as soon as the servers return, they are all out with customers now and I have no access.


  • Moderator

    @grey Before you loose access to this server pxe boot it again and run the compatibility test from the FOG iPXE menu. I’m interested to see if both networking and storage passes (you will be too).



  • @grey

    oh btw, thanks tons guys for all your help



  • @Tom-Elliott said in bnx2x fails to load firmware on Dell R430:

    @george1421 Well it goes a bit further, if a link is up, but fails to get an ip, it will continue on as well, I believe.

    Networking works!

    I can confirm that, mine runs through interfaces 1-5 and now now acquires a IP!, i’m unsure if there is a arbitrary limit. when 5 was failing to get DHCP it also tried 6 and 7. Toms fix on the firmware of the drivers and Georges portfast suggestion solved the last of my network issues. Unfortunately I won’t have access to the servers for a week or so, but once they get back i’ll see how the imaging goes. at this point it should work as the rest is similar to my older servers setup the same way.


  • Moderator

    @Tom-Elliott That one I can test by setting the interface to a vlan without a dhcp server. I will prove that one out for sure. The other one I’m not sure if I can have a “unplugged” interface but have it appear in linux with ESXi.


  • Senior Developer

    @george1421 Well it goes a bit further, if a link is up, but fails to get an ip, it will continue on as well, I believe.


  • Moderator

    @Tom-Elliott said in bnx2x fails to load firmware on Dell R430:

    @george1421 It tries everything until it finds a link up I believe.

    Great, I have never tried. But I just thought I can test this by creating a vm with 6 network adapters with eth5 being the only one connected. But I’m not sure if I can have it defined but not having the link up. I may have to play in my test lab later tonight.


  • Senior Developer

    @george1421 It tries everything until it finds a link up I believe.


  • Moderator

    Also pondering this a bit more since your first (usable) interface is eth5 I can see another issue.

    @Developers Does the FOS engine stop enumerating the network adapter at eth1 or will it keep going until it finds a network adapter with a link up state to use? OR Is there a kernel parameter we can pass that says to use ethX as the FOS network interface?



  • @george1421 said in bnx2x fails to load firmware on Dell R430:

    OK this is good promising news. eth5 is getting a link up but no IP address in FOS but you can pxe boot and iPXE is getting an IP address because the FOS kernel (bzImage) and VHD (init.xz) are being sent to the target computer. When we see this we typically see that spanning tree is turned on at the network switch. Spanning tree takes about 27 seconds for the port to start forwarding data. The FOS engine is so fast that it has already given up by the time spanning tree starts to forward data.

    On these ports you need to enable one of the fast spanning tree prototocols (port fast, fast stp, rstp, or what ever your switch mfg calls it). A quick check to see if its a spanning tree issue is to put an unmanaged switch between the building switch and the target computer. This unmanaged switch keeps the port on the building switch from winking as the iPXE kernel hands off the network interface to the FOS engine.

    its a cisco so its port fast, i’ll check right now,


  • Moderator

    OK this is good promising news. eth5 is getting a link up but no IP address in FOS but you can pxe boot and iPXE is getting an IP address because the FOS kernel (bzImage) and VHD (init.xz) are being sent to the target computer. When we see this we typically see that spanning tree is turned on at the network switch. Spanning tree takes about 27 seconds for the port to start forwarding data. The FOS engine is so fast that it has already given up by the time spanning tree starts to forward data.

    On these ports you need to enable one of the fast spanning tree prototocols (port fast, fast stp, rstp, or what ever your switch mfg calls it). A quick check to see if its a spanning tree issue is to put an unmanaged switch between the building switch and the target computer. This unmanaged switch keeps the port on the building switch from winking as the iPXE kernel hands off the network interface to the FOS engine.



  • @Sebastian-Roth

    ok PNX2X is the fiber ports so we can disregard them, oddly they where enumerated first so thats part of the problem as i thought the ethernet ports would be first, the TG3 driver is the ethernet port and the ones we are interested in. eth5 is the port thats active and does have link, although its not getting a DHCP assigment ( statically assigned to that mac ), even though at pexe boot it is.

    Settings for eth1: PNX2X
    Supported ports: [ FIBRE ]
    Supported link modes: 1000baseT/Full
    10000baseT/Full
    Supported pause frame use: Symmetric Receive-only
    Supports auto-negotiation: No
    Advertised link modes: 10000baseT/Full
    Advertised pause frame use: No
    Advertised auto-negotiation: No
    Speed: Unknown!
    Duplex: Unknown! (255)
    Port: FIBRE
    PHYAD: 1
    Transceiver: internal
    Auto-negotiation: off
    Supports Wake-on: g
    Wake-on: d
    Current message level: 0x00000000 (0)

    Link detected: no
    

    Settings for eth2:PNX2X
    Supported ports: [ FIBRE ]
    Supported link modes: 1000baseT/Full
    10000baseT/Full
    Supported pause frame use: Symmetric Receive-only
    Supports auto-negotiation: No
    Advertised link modes: 10000baseT/Full
    Advertised pause frame use: No
    Advertised auto-negotiation: No
    Speed: Unknown!
    Duplex: Unknown! (255)
    Port: FIBRE
    PHYAD: 1
    Transceiver: internal
    Auto-negotiation: off
    Supports Wake-on: g
    Wake-on: d
    Current message level: 0x00000000 (0)

    Link detected: no
    

    Settings for eth3:PNX2X
    Supported ports: [ FIBRE ]
    Supported link modes: 1000baseT/Full
    10000baseT/Full
    Supported pause frame use: Symmetric Receive-only
    Supports auto-negotiation: No
    Advertised link modes: 10000baseT/Full
    Advertised pause frame use: No
    Advertised auto-negotiation: No
    Speed: Unknown!
    Duplex: Unknown! (255)
    Port: FIBRE
    PHYAD: 1
    Transceiver: internal
    Auto-negotiation: off
    Supports Wake-on: g
    Wake-on: d
    Current message level: 0x00000000 (0)

    Link detected: no
    

    Settings for eth4:
    Supported ports: [ TP ]
    Supported link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Half 1000baseT/Full
    Supported pause frame use: No
    Supports auto-negotiation: Yes
    Advertised link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Half 1000baseT/Full
    Advertised pause frame use: Symmetric
    Advertised auto-negotiation: Yes
    Speed: Unknown!
    Duplex: Unknown! (255)
    Port: Twisted Pair
    PHYAD: 1
    Transceiver: internal
    Auto-negotiation: on
    MDI-X: Unknown
    Supports Wake-on: g
    Wake-on: d
    Current message level: 0x000000ff (255)
    drv probe link timer ifdown ifup rx_err tx_err
    Link detected: no


    Settings for eth5:
    Supported ports: [ TP ]
    Supported link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Half 1000baseT/Full
    Supported pause frame use: No
    Supports auto-negotiation: Yes
    Advertised link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Half 1000baseT/Full
    Advertised pause frame use: Symmetric
    Advertised auto-negotiation: Yes
    Link partner advertised link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Full
    Link partner advertised pause frame use: No
    Link partner advertised auto-negotiation: Yes
    Speed: 1000Mb/s
    Duplex: Full
    Port: Twisted Pair
    PHYAD: 2
    Transceiver: internal
    Auto-negotiation: on
    MDI-X: off
    Supports Wake-on: g
    Wake-on: d
    Current message level: 0x000000ff (255)
    drv probe link timer ifdown ifup rx_err tx_err
    Link detected: yes


    Settings for eth6:
    Supported ports: [ TP ]
    Supported link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Half 1000baseT/Full
    Supported pause frame use: No
    Supports auto-negotiation: Yes
    Advertised link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Half 1000baseT/Full
    Advertised pause frame use: Symmetric
    Advertised auto-negotiation: Yes
    Speed: Unknown!
    Duplex: Unknown! (255)
    Port: Twisted Pair
    PHYAD: 1
    Transceiver: internal
    Auto-negotiation: on
    MDI-X: Unknown
    Supports Wake-on: g
    Wake-on: d
    Current message level: 0x000000ff (255)
    drv probe link timer ifdown ifup rx_err tx_err
    Link detected: no


    Settings for eth7:
    Supported ports: [ TP ]
    Supported link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Half 1000baseT/Full
    Supported pause frame use: No
    Supports auto-negotiation: Yes
    Advertised link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Half 1000baseT/Full
    Advertised pause frame use: Symmetric
    Advertised auto-negotiation: Yes
    Speed: Unknown!
    Duplex: Unknown! (255)
    Port: Twisted Pair
    PHYAD: 2
    Transceiver: internal
    Auto-negotiation: on
    MDI-X: Unknown
    Supports Wake-on: g
    Wake-on: d
    Current message level: 0x000000ff (255)
    drv probe link timer ifdown ifup rx_err tx_err
    Link detected: no


  • Developer

    @grey said:

    ok, re-installed fog rebooted on the new kernel, drivers loaded fine but all drivers say there is no link, although its running on a pxe kernel :-p.

    As we can see from your earlier posts you have four NICs in that one server. Which one is connected and how often do you see the message “no link”??


  • Moderator

    @grey Correct it will stay in debug capture until the capture completes or you cancel the task. its kind of handy for debugging your hardware.

    As for the no link issue I would go back to the error logs on the FOS client. Also the lspci command (one of the switches) should show what kernel driver is managing which device.


  • Senior Developer

    @grey The phandle messages are non-impacting, and you can probably turn back down the loglevel. The drivers with no link simply means the kernel can’t find a corresponding device for the driver being loaded.



  • @Tom-Elliott said in bnx2x fails to load firmware on Dell R430:

    You mind rerunning the installer? I’ve added the missing firmware to the kernel build.

    ok, re-installed fog rebooted on the new kernel, drivers loaded fine but all drivers say there is no link, although its running on a pxe kernel :-p. all the firmware errors are gone. I’m running out of time here today, but I’ll jump into tomorrow and see what the link issue is about. the debug kernel selection is still loading by default so i guess it will do that until i delete the task.


Log in to reply
 

438
Online

38955
Users

10706
Topics

101580
Posts

Looks like your connection to FOG Project was lost, please wait while we try to reconnect.