Multiple NIC Hosts

[cspence]

I’m experiencing problems with some of the SVN revisions after 1.2.0 on machines with two NICs. I’m using the NIC that tends to register as eth1.

Here’s some base information:

FOG Versions Tested: 1.2.0, SVN 2948, and SVN 3053
On-Board NIC: Intel 82579LM Gigabit (Registers as eth1 with Linux)
PCI-Express NIC: Broadcom NetXtreme BCM5722 Gigabit (Registers as eth0 with Linux)

First of all, version 1.2.0 works fine. [B]This bug is experienced in SVN 2948 and 3053.[/B]

When I try to register a host or capture an image from a host, SVN 2948 and 3053 do not power on the On-Board NIC (eth1) while attempting to get a DHCP lease. Switching the cable to the PCI-E NIC immediately leases an IP. Removing the PCI-E NIC allows registration and imaging to work on the On-Board NIC.

It appears that some point between 1.2.0 and SVN 2948, the behavior was changed to only use eth0 rather than find the working interface.

[Sebastian Roth]

Thank you guys for working on this and making it all run smoothly!

[cspence]

@Tom Elliott, post: 43130, member: 7271 said:

It’s all good, you should see the issues I forget about, like using my mac address implicitly in the inventory and wondering why everybody else is getting repeated inventory errors.

Now that’s a good one.

[Tom Elliott]

It’s all good, you should see the issues I forget about, like using my mac address implicitly in the inventory and wondering why everybody else is getting repeated inventory errors.

[cspence]

@Tom Elliott, post: 43128, member: 7271 said:

There’s one more issue

Shouldn’t the cat be reading a file, not the directory?

Yes. Fixed again. I really should have slept more last night.

[Tom Elliott]

There’s one more issue

Shouldn’t the cat be reading a file, not the directory?

[cspence]

@Tom Elliott, post: 43117, member: 7271 said:

Also, shouldn’t the tr -d be tr -d ‘@’ not tr -d ‘0’?

Sorry, I was copying off code from my cellphone and wasn’t thinking about what I was typing. Also caught another typo on the udhcpc_opt statement.

Fixed the typos in the original post.

[Tom Elliott]

Also, shouldn’t the tr -d be tr -d ‘@’ not tr -d ‘0’?

[Tom Elliott]

Bad statement.

linkstate=$(/bin/cat /sys/class/net/$iface

Should be
linkstate=$(/bin/cat /sys/class/net/$iface)

[cspence]

# Enable all interfaces
ifaces=$(ls -1 /sys/class/net | tr -d '@')
for iface in $ifaces; do
    /sbin/ip link set $iface up
done
 
# Provide time for interfaces to detect their state
sleep 5
 
echo "auto lo" > /etc/network/interfaces
echo "iface lo inet loopback" >> /etc/network/interfaces
 
for iface in $ifaces; do
    linkstate=$(/bin/cat /sys/class/net/$iface/carrier)
    if [[ "x$linkstate" = "x1" -a "x$iface" != "xlo" ]]; then
        echo "auto $iface" >> /etc/network/interfaces
        echo "iface $iface inet dhcp" >> /etc/network/interfaces
        echo -e "\tudhcpc_opts -t 100 -T 20\n" >> /etc/network/interfaces
    fi
    /sbin/ip link set $iface down
done

This solution only requires one sleep rather than five-seconds of sleep time per interface.

Edit: Fixed typos.

[cspence]

Finally got a chance to play with it, and I have a solution.

Insert a 5-second sleep between setting the interface up and checking the link status (we can’t just check before it gets a chance to find out if it is connected to a cable). Here’s the snippet as modified.

...
/sbin/ip link set $iface up
# Provide time for the interface to update linkstate
sleep 5
linkstate=$(/bin/cat /sys/class/net/$iface/carrier)
...

Works on my end.

…and if you wait just a second, I’ll have an optimized version…

[Tom Elliott]

Would the problems we’re seeing be related to the fact we bring the interface up, but we don’t bring it back down, so when the interface is added and the loop happens, it can’t get dhcp because it believes the link is up?

[Tom Elliott]

Updated to reflect the change. Pushed to inits as well.

[cspence]

@Tom Elliott, post: 43023, member: 7271 said:

What’s weird, though, is the /etc/network/interfaces appears to get the proper values set (without the -a) but even without eth0 should still be found with little issue, and without these tweaks (reverting back to only having iface eth0) everything functions properly.

Actually, I was trying to go off memory. The flag is [B]-i $iface[/B]. You don’t want to use -a. Sorry about that.

[Tom Elliott]

What’s weird, though, is the /etc/network/interfaces appears to get the proper values set (without the -a) but even without eth0 should still be found with little issue, and without these tweaks (reverting back to only having iface eth0) everything functions properly.

[cspence]

By default, udhcpc uses eth0. You will probably need to add -a $iface to udhcpc_opts. I was unsure if that would be taken care of by /etc/network/interfaces.

[Sebastian Roth]

Sure I am happy to assist as much as I can!! Looking at the script in current SVN I noticed that you create ‘/etc/network/interfaces’ new from scratch (by using ‘>’) when ‘lo’ is detected. Normally that would be a good idea but when running this in my FOG test environment (QEMU) I end up with this:

auto lo
iface lo inet loopback

Why you ask?? Because ‘lo’ does not necessarily come first. Actually on most systems ls is sorted by alphabet and ‘e’ comes before ‘l’. Do you know what I mean?

Fix: Use “>> /etc/network/interfaces” everywhere instead of “> /etc/network/interfaces”. If you want to make sure that it’s empty then do “echo > /etc/network/interfaces” before the for loop.

[Tom Elliott]

so apparently this isn’t working properly.

I’ve changed the for loop to only add if the interfaces found does not match lo and eth0. My testing side seems to work properly, but other’s are seeing really strange things, particularly that the nic’s aren’t getting IPs. I don’t know why, maybe I can get some help over here?

[Tom Elliott]

It shouldn’t pose problems unless the nic is unsupported, in which case they’d already be having problems regardless of with or without these tweaks.

[Sebastian Roth]

@cspence, post: 42950, member: 28749 said:

Careful, that’s not an “L”, that’s a “One.” He set it up to do one file/directory per line.
That’s right, thanks for clarifying. And thank you Tom for adding this straight into the project. I really hope that it’s not causing any trouble for other users…