Changing IP address post fog install is problematic

[george1421]

Once fog is installed it is difficult to change the IP address of the fog server.

I updated the /opt/fog/.fogsettings with the new IP address and then reran the installer script.

The first issue I ran into was the background wouldn’t load for the iPXE menu. This was addressed by updating the fog setting FOG_WEB_HOST to the new IP address.

The second issue I ran into was not being able to connect to the NFS share. This was corrected by updating the default storage node, default member, IP address.

Also I see FOG_TFTP_HOST setting still points to the old IP address

I would be great if the installer script would check these values to see if they were different then update them to reflect the new setting in the .fogsettings file.

[Tom Elliott]

While I do understand the issues it’s not exactly that simple to make those adjustments. This, you should remember, is because storage nodes do the same type of file. While I do understand what you’re saying, if the ip/hostname changes how does the installer know what the original values were? In the case of of a setup with multiple nodes and potentially a separate server, how would you propose this be done? How can fog be told the original from the new? Especially if you migrate the original db to the new server?

[george1421]

I also understand what you are saying, but what would the impact be if the installer just blindly updated the current values (what ever they are) to the settings found in the .fogsettings file?

However this could be also addressed with a wiki page. Once the settings were found the fog server did work as intended at the new IP address. In the end this was my own fault for being lazy and letting dhcp setup the OS and not remembering to put it at the final IP address before fog was installed.

[Tom Elliott]

@george1421 The impact would be more that if you have multiple storage nodes (multiple ip address and what not) how will we know which one to update?

[george1421]

@Tom-Elliott The probably the easiest solution is to create a procedure (wiki page) that if you need to update your fog server you need to do:

1. Update the settings in the /opt/fog/.fogsettings
2. Rerun the installer
3. Update the IP address for the storage node on the FOG system where you changed the IP address
4. Update the IP address on a any master storage node that may reference this FOG server
5. Update the FOG_WEB_HOST value
6. update the FOG_TFTP_HOST value

I can I’ll marked this as addressed (Solved), since its not a technical solution but a procedural one that resolves the issue.
[Edit] I would, but it appears I can no longer mark topics as solved[/Edit]

[Wayne Workman]

@george1421 said:

However this could be also addressed with a wiki page.

I’m actually working on this. Here’s links for future readers:
https://wiki.fogproject.org/wiki/index.php?title=Change_FOG_Server_IP_Address

I’m going to work on that and a few other articles over my 2 week x-mas vacation.

wiki

[sudburr]

In its simplest form that we run, some of our FOG servers are stationary, others move around. For us to have a single solution build that works for any location, all of our FOG servers are configured to grab an IP via external DHCP reservation.

I copy and paste the following to the CLI to edit rc.local:

cp -f /etc/rc.local /etc/rc.local.old
if [ -f /etc/centos-release ]; then
echo ' ' >> /etc/rc.local
echo 'make_fog_portable' >> /etc/rc.local
echo ' ' >> /etc/rc.local
else
sed -i "s|exit 0|make_fog_portable &|g" /etc/rc.local
echo ' ' >> /etc/rc.local
echo 'exit 0' >> /etc/rc.local
fi
chmod 755 /etc/rc.local
echo '#!/bin/bash' > /bin/make_fog_portable
echo '#' >> /bin/make_fog_portable
echo '# make_fog_portable &' >> /bin/make_fog_portable
echo '#' >> /bin/make_fog_portable
echo '# This script is expected to be run as a job from /etc/rc.local' >> /bin/make_fog_portable
echo '# It will wait until an IP address is found, then use that IP' >> /bin/make_fog_portable
echo '# address to configure the FOG Server for that site.' >> /bin/make_fog_portable
echo '#' >> /bin/make_fog_portable
echo ' ' >> /bin/make_fog_portable
echo 'exit 0' >> /bin/make_fog_portable
chmod 755 /bin/make_fog_portable
vim /bin/make_fog_portable

At the now open file ‘make_fog_portable’ Insert the following, before “exit 0” ; [ESC]:wq to write/quit

# Wait for an IP address
IP=`ip addr list eth0 | grep "inet " |cut -d" " -f6|cut -d/ -f1`
while [ -z $IP ]
do
	echo "Waiting :05 for an IP Address" > /dev/kmsg
	sleep 5
	IP=`ip addr list eth0 | grep "inet " |cut -d" " -f6|cut -d/ -f1`
done

# Make FOG Server Portable
sleep 6
echo "Updating IP address for FOG_TFTP_HOST to be $IP [`date`]" > /dev/kmsg
mysql --user=root -e "UPDATE \`globalSettings\` SET \`settingValue\` = '$IP' WHERE \`settingKey\` ='FOG_TFTP_HOST';" fog
echo "Updating IP address for FOG_WEB_HOST to be $IP [`date`]" > /dev/kmsg
mysql --user=root -e "UPDATE \`globalSettings\` SET \`settingValue\` = '$IP' WHERE \`settingKey\` ='FOG_WEB_HOST';" fog
echo "Updating IP address for FOG_WOL_HOST to be $IP [`date`]" > /dev/kmsg
mysql --user=root -e "UPDATE \`globalSettings\` SET \`settingValue\` = '$IP' WHERE \`settingKey\` ='FOG_WOL_HOST';" fog
echo "Updating IP address for Storage Node DefaultMember to be $IP [`date`]" > /dev/kmsg
mysql --user=root -e "UPDATE \`nfsGroupMembers\` SET \`ngmHostname\` = '$IP' WHERE \`ngmMemberName\` ='DefaultMember';" fog
echo "Updating IP address in file .fogsettings to be $IP [`date`]" > /dev/kmsg
sed -i "s|ipaddress=\".*\"|ipaddress=\"$IP\"|" /opt/fog/.fogsettings
echo "Updating IP address in file default.ipxe to be $IP [`date`]" > /dev/kmsg
sed -i "s|http://\([^/]\+\)/|http://$IP/|" /tftpboot/default.ipxe
sed -i "s|http:///|http://$IP/|" /tftpboot/default.ipxe

echo "Sleeping 10 seconds before releasing script [`date`]" > /dev/kmsg
sleep 10
echo "releasing script [`date`]" > /dev/kmsg

Complete the generalization with this; you will also need to run this after any FOG trunk update:

if [ -f /etc/debian_version ]; then
cd /var/www/fog/lib/fog
fi

if [ -f /etc/centos-release ]; then
cd /var/www/html/fog/lib/fog
fi

cp -f config.class.php config.class.php.old
sed -i "s|\".*\..*\..*\..*\"|\$_SERVER['SERVER_ADDR']|" config.class.php
reboot

Why a job at startup? In the case of a power failure, the FOG server itself whether it be physical or virtual will almost invariably be available long before the site’s switches finish their POSTs and the network is available again. The loop to check for an IP prevents the FOG server coming up without an IP or in the case of a mobile server, forces the new IP into FOG’s configuration before it has a chance to start.

This solution even allows us to change the subnet entirely and the server will always, automatically reconfigure itself according to the new DHCP reservation.

Works like a charm.

This works on Ubuntu 14-, 15+, Debian 8+ and CentOS 7.

It is into this make_fog_portable job that I would also add any code for restarting critical services:

# Ubuntu 14-
sleep 6
echo "Restarting tftpd-hpa [`date`]" > /dev/kmsg
service tftpd-hpa restart
sleep 6
echo "Restarting mysql [`date`]" > /dev/kmsg
service mysql restart
sleep 6
echo "Restarting FOGMulticastManager [`date`]" > /dev/kmsg
service FOGMulticastManager restart

# Debian 8+, Ubuntu 15+
if [ -f /etc/debian_version ]; then
echo "Restarting Critical FOG Services [`date`]" > /dev/kmsg
systemctl restart tftp* mysql* FOG* apache*
fi

[Wayne Workman]

@sudburr now that is sweet! This is great work. It would have taken me several long hours to get to the point where you’ve come to. From here, it’s a hop, skip, and a jump to mix this with dnsmasq.

[george1421]

I agree this is great! I’m glad we have the community involvement in the FOG community to help everyone solve these basic problems. Well done @sudburr !!

The only thing that jumps out that may be an issue in the script is that for the Centos 7 OS the ethernet adapters are no longer just eth0, eth1 and so on. The Intel NUC I’m working on the default ethernet adapter is enp3s0.

I just checked and this value interface="enp3s0" can be extracted from the /opt/fog/.fogsettings file. This would then tie into what interface FOG is using too. There may be a case where there is a system with one or more network interfaces that the script would have to take into account.

[Wayne Workman]

@george1421 said:

The only thing that jumps out that may be an issue in the script is that for the Centos 7 OS the ethernet adapters are no longer just eth0, eth1 and so on. The Intel NUC I’m working on the default ethernet adapter is enp3s0.

If the interface name is known, you can use this to get the IP in CentOS 7.

interface1ip="$(/sbin/ip addr show | grep $interface1name | grep -o "inet [0-9]*\.[0-9]*\.[0-9]*\.[0-9]*" | grep -o "[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*")"

This is a piece from a little project I’m working on:
https://sourceforge.net/p/openvpnrouter/code/HEAD/tree/identifyInterfaces.sh#l26

[Wayne Workman]

I’m going to take a stab at making all of that below into one single cron-friendly script - and have it run via Cron every 5 minutes.

I’ll add logic to it so that it’ll only run if the IP has changed since the last run. It’ll use settings set in the FOG config file where appropriate.

I’ll also work dnsmasq into it as well, along with service restarts for dnsmasq.

[Wayne Workman]

100% NOT tested

#---- Notes ----#

# This script is for changing the FOG Server's IP address and configure dnsmasq automatically.

# Thanks to forums.fogproject.org @sudburr for doing a ton of initial work.

# Updated by forums.fogproject.org @Wayne-Workman

# This is an early copy, stay tuned for updates.

# December 22th, 2015.


#---- Get interface name and last IP from .fogsettings ---#

interface="$(/usr/bin/grep 'interface=' /opt/fog/.fogsettings | /usr/share/awk -F'"' '{$0=$2}1')"
fogsettingsIP="$(/usr/bin/grep 'ipaddress=' /opt/fog/.fogsettings | /usr/share/awk -F'"' '{$0=$2}1')"


#---- Wait for an IP address ----#

IP=`/sbin/ip addr list ${interface} | /usr/bin/grep "inet " |/usr/bin/cut -d" " -f6|/usr/bin/cut -d/ -f1`

while [ -z $IP ]

do
    sleep 5
    IP=`ip addr list ${interface} | grep "inet " |cut -d" " -f6|cut -d/ -f1`
done




if [[ "$IP" != "$fogsettingsIP" ]]; then
#If the actual IP doesn't match the .fogsettings IP

        #Update .fogsettings IP
        sed -i "s|ipaddress=\".*\"|ipaddress=\"$IP\"|" /opt/fog/.fogsettings

        #Get MySQL credentials
        snmysqluser="$(/usr/bin/grep 'snmysqluser=' /opt/fog/.fogsettings | /usr/share/awk -F'"' '{$0=$2}1')"
        snmysqlpass="$(/usr/bin/grep 'snmysqlpass=' /opt/fog/.fogsettings | /usr/bin/cut -d \' -f2 )"

        #---- SQL ----#

        #These are the SQL statements to run against the DB
        statement1="UPDATE \`globalSettings\` SET \`settingValue\` = '$IP' WHERE \`settingKey\` ='FOG_TFTP_HOST';"
        statement2="UPDATE \`globalSettings\` SET \`settingValue\` = '$IP' WHERE \`settingKey\` ='FOG_WOL_HOST';"
        statement3="UPDATE \`nfsGroupMembers\` SET \`ngmHostname\` = '$IP' WHERE \`ngmMemberName\` ='DefaultMember';"

        if [ "$snmysqlpass" != "" ]; then
                #If there is a password set
                /usr/bin/mysql --user=$snmysqluser --password=$snmysqlpass --database='fog'  << EOF
                $statement1
                $statement2
                $statement3
                EOF

        elif [ "$snmysqluser != "" ]; then
                #If there is a user set but no password
                /usr/bin/mysql --user $snmysqluser --database='fog' << EOF
                $statement1
                $statement2
                $statement3
                EOF

        else
                #If there is no user or password set
                /usr/bin/mysql --database='fog' << EOF
                $statement1
                $statement2
                $statement3
                EOF

        fi


        #Updating IP address in file default.ipxe
        sed -i "s|http://\([^/]\+\)/|http://$IP/|" /tftpboot/default.ipxe
        sed -i "s|http:///|http://$IP/|" /tftpboot/default.ipxe


        #---- Backup config.class.php and then updae IP ----#
        cp -f /var/www/html/fog/lib/fog/config.class.php /var/www/html/fog/lib/fog/config.class.php.old
        sed -i "s|\".*\..*\..*\..*\"|\$_SERVER['SERVER_ADDR']|" /var/www/html/fog/lib/fog/config.class.php



fi

[george1421]

@Wayne-Workman This is going to be a critical review of the code. Don’t take it personally (my guys hate me when I do this type of review, but the finished product is generally better).

Some lines use the full path to the application and some only use the application name. (i.e. /usr/share/awk)

Since you are using the full path to the application you assume that is the proper location. I just checked on my ubuntu based system awk is in /usr/bin/awk under Centos 7 its again /usr/bin/awk but in your script it references /usr/share/awk which I assume would cause your script to fail on these systems.

What would happen if interface or fogsettingsIP returned nothing instead of a value what would happen to your script?

The same question if the mysql credentials were not returned.

In sudburr’s post also references a wol host field to be updated, is this important?

What happens if config.class.php.old already exists when the script is run a second or third time. Will anything important be lost?

Do any running fog services maintain a cached copy of the server’s IP address or do the fog services need to be stopped when you update these settings to make them stick?

[Wayne Workman]

@george1421 All great points - The paths in there are for Fedora 23.

I’ll see about using the $PATH variable instead of hard coded paths - and I’ll work on better variable handling.

[george1421]

@Wayne-Workman

Some other things I just thought of while walking about.

How does dhcp lease time come into play in regards to script execution. Especially if this will be a cron job. Consider what is a shortest duration lease and a typical duration lease time.

should the physical location of this script be location in a fog related directory like /opt/fog/utils?

Is there any way to trap an interface going up/down and then to call your script? hint: http://www.cyberciti.biz/tips/how-do-i-run-firewall-script-as-soon-as-eth0-interface-brings-up.html

Does the interface go down and then up on a dhcp renewal.

We would assume that the path statement points correctly for bash to run without full paths on the called applications especially if they are system type applications.

[Wayne Workman]

@george1421 My thought was just to setup a cron job that runs every 5 minutes.

If the actual IP is the same as what’s in .fogsettings then nothing is done - and to find that out, it’s not much resources used.

[mrayzies]

@Wayne-Workman
I think there should be an option in .fogsettings to skip this altogether, not because of system resources, but occasionally I might want to poke at the system (e.g. add a temporary VIP for testing something else) and unless the coding is air-tight, it might kick off some horrible changes.

[Wayne Workman]

@mrayzies Are you talking about the cron job? We can add a simple setting into the .fogsettings file like keepIPupdated="1" / "0"

What I’m working on is a total custom thing - it’ll likely never be implemented into FOG Trunk officially.

[mrayzies]

I was talking about the CRON job.

And OK – sorry to barge in on the conversation.

[Wayne Workman]

@mrayzies By all means, give your thoughts and even contribute. FOG is a community project.