Script to install Samba with settings for FOG

[Wayne Workman]

OK! so…

Good news and bad news…

GOOD NEWS:
Tom integrated CIFS support into the inits and kernels within a matter of HOURS… wow!

When I turn OFF nfs on my FOG server, and then do a “debug download”
I can successfully issue a mount command via CIFS to the /images directory.

I can then go into that directory and see my images, make files, delete files, etc.

BAD NEWS:
The script changes in my earlier post did not work…

So… I hard coded everything… into this file:
[CODE]/svn/trunk/src/buildroot/package/fog/scripts/bin/fog.checkin[/CODE]

and I was using this command for mounting, more or less:
[CODE]mount -t cifs -o username=root,password=PASSWORDHERE //10.0.0.3/images /images 2>/tmp/mntfail;[/CODE]

and it would SEEM that it’s not using that command to mount… the error says “failed to mount on 10.0.0.3:/images blah blah” and I’m thinking that error is generated from the $storage variable… and isn’t actually the output from my actual command to mount.

So… this begs the question… why can I issue the command to mount inside a debug download, but the regular download task fails?

I’m convinced that somehow it’s not using the commands that I wrote into the aforementioned file.

Here’s the file as it is… I just changed my password. Note that those mounting commands DO work if I issue them manually.

[CODE]#!/bin/bash
. /usr/share/fog/lib/funcs.sh
RUN_CHKDSK=“”;
HOSTNAME_EARLY=“0”;
OS_ID_WIN7=“5”;
OS_ID_WIN8=“6”;
for arg in cat /proc/cmdline; do
case “$arg” in
initsh)
ash -i;
;;
nombr)
nombr=1;
;;
*)
;;
esac
done
clear;
displayBanner;
#setupDNS $dns;
osname=“”;
mbrfile=“”;
determineOS “$osid”;
macWinSafe=echo $mac|sed 's/://g';
cores=$(grep “core id” /proc/cpuinfo|sort -u|wc -l);
sockets=$(grep “physical id” /proc/cpuinfo|sort -u|wc -l);
cores=$((cores * sockets));
arch=$(uname -m);
if [ “$cores” == “0” ]; then
cores=1;
fi
if [ “$chkdsk” == “1” ]; then
RUN_CHKDSK=“-x”;
fi
if [ “$hostearly” == “1” ]; then
HOSTNAME_EARLY=“1”;
fi
if [ “$mc” == “yes” ]; then
method=“UDPCAST”;
elif [ “$mc” == “bt” ]; then
method=“Torrent-Casting”;
else
method=“NFS”;
fi
debugPause;
#fdisk -l &> /tmp/fdisk-before;
echo “”;
dots “Checking Operating System”
echo $osname;
dots “Checking CPU Cores”
echo $cores
echo “”;
dots “Send method”
echo $method
blGo=“0”;
nfsServerName=“”;
if [ “$mode” == “clamav” ]; then
dots “Checking In”;
queueinfo=wget -q -O - "http://${web}service/Pre_Stage1.php?mac=$mac&avmode=$avmode" 2>/dev/null;
echo “Done”;
debugPause;
dots “Mounting Clamav”;
if [ ! -d “/opt/fog/clamav” ]; then
mkdir -p /opt/fog/clamav 2>/dev/null;
fi
#mount -o nolock,proto=tcp,rsize=32768,wsize=32768,intr,noatime $clamav /opt/fog/clamav;
mkdir /opt
mkdir /opt/fog
mkdir /opt/fog/clamav
mount -t cifs -o username=root,password=PASSWORDHERE //10.0.0.3/images /opt/fog/clamav;
echo “Done”;
debugPause;
dots “Adding clamav to path”;
if [ -d “/opt/fog/clamav/bin” ] && [ -d “/opt/fog/clamav/sbin” ]; then
export PATH=$PATH:/opt/fog/clamav/bin:/opt/fog/clamav/sbin 2>/dev/null;
else
handleError “Cannot find clamav binaries to run task.”;
fi
echo “Done”;
debugPause;
fi
if [ “$type” == “up” ]; then
dots “Checking In”
queueinfo=wget -q -O - "http://${web}service/Pre_Stage1.php?mac=$mac&type=$type" 2>/dev/null;
echo “Done”;
debugPause;
dots “Mounting File System”
mkdir /images 2>/dev/null;
#mount -o nolock,proto=tcp,rsize=32768,wsize=32768,intr,noatime $storage /images &> /dev/null;
mkdir /images
mount -t cifs -o username=root,password=PASSWORDHERE //10.0.0.3/images /images &> /dev/null;
if [ “$?” == 0 ]; then
echo “Done”;
else
echo “Failed”;
handleError “Unable to mount NFS”;
fi
debugPause;
elif [ “$type” == “down” ] && [ “$capone” != “1” ]; then
mac64=getMACAddresses | base64;
dots “Attempting to send inventory”;
doInventory 2>/dev/null;
poststring=“mac=${mac64}&sysman=${sysman64}&sysproduct=${sysproduct64}&sysversion=${sysversion64}&sysserial=${sysserial64}&systype=${systype64}&biosversion=${biosversion64}&biosvendor=${biosvendor64}&biosdate=${biosdate64}&mbman=${mbman64}&mbproductname=${mbproductname64}&mbversion=${mbversion64}&mbserial=${mbserial64}&mbasset=${mbasset64}&cpuman=${cpuman64}&cpuversion=${cpuversion64}&cpucurrent=${cpucurrent64}&cpumax=${cpumax64}&mem=${mem64}&hdinfo=${hdinfo64}&caseman=${caseman64}&casever=${casever64}&caseserial=${caseserial64}&casesasset=${casesasset64}”;
invres=“”;
while [ “$invres” == “” ]; do
invres=wget -O - --post-data="$poststring" "http://${web}service/inventory.php" 2>/dev/null;
echo “$invres”;
done
debugPause;
dots “Checking In”;
while [ “$blGo” == “0” ]; do
if [ “$capone” != “1” ]; then
if [ “$mc” != “yes” -a “$mc” != “bt” ]; then
queueinfo=wget -q -O - "http://${web}service/Pre_Stage1.php?mac=$mac&type=$type" 2>/dev/null;
blPass=echo $queueinfo|grep "##";
waittime=0;
while [ ! -n “$blPass” ]; do
echo -n " * $queueinfo (“;
sec2String “$waittime”;
echo “)”
queueinfo=wget -q -O - "http://${web}service/Pre_Stage1.php?mac=$mac&type=$type" 2>/dev/null;
blPass=echo $queueinfo | grep "##";
sleep 5;
waittime=$(expr $waittime “+” 5);
done
echo “Done”;
debugPause;
directive=”${queueinfo:3}“;
if [ ! “$directive” = “GO” ]; then
tmpStorageIp=echo $queueinfo|cut -d'@' -f2 2>/dev/null;
tmpStorage=echo $queueinfo|cut -d'@' -f3 2>/dev/null;
tmpName=echo $queueinfo|cut -d'@' -f4 2>/dev/null;
if [ “$tmpStorage” != “” -a “$tmpStorageIp” != “” ]; then
storage=$tmpStorage;
storageip=$tmpStorageIp;
nfsServerName=$tmpName;
else
handleError “Error determining storage server!”;
exit 1;
fi
dots “Using Storage Node”
echo “$nfsServerName”
debugPause;
fi
else
queueinfo=wget -q -O - "http://${web}service/mc_checkin.php?mac=$mac&type=$type" 2>/dev/null;
blPass=echo $queueinfo|grep "##";
echo “Done”;
waittime=0;
while [ ! -n “$blPass” ]; do
echo -n " * $queueinfo (”;
sec2String “$waittime”
echo “)”
queueinfo=wget -q -O - "http://${web}service/mc_checkin.php?mac=$mac&type=$type" 2>/dev/null;
blPass=echo $queueinfo | grep "##";
sleep 5;
waittime=$(expr $waittime “+” 5);
done
if [ “$mc” == “bt” ]; then
dots “Using image”
# download $img.torrent file
wget -q -O /tmp/$img.torrent http://${web}/service/torrent.php?torrent=$img;
ctorrent /tmp/$img.torrent -x > /tmp/filelist.txt;
torrentDownloadSize=cat /tmp/filelist.txt|grep "Total:*"|awk '{print $2}';
echo “$img”;
dots “Size of image to download”
echo “$torrentDownloadSize MB”;
debugPause;
fi
fi
dots “Mounting File System”;
mkdir /images $debugstring 2>/dev/null;
#mount -o nolock,proto=tcp,rsize=32768,intr,noatime $storage /images 2>/tmp/mntfail;
mkdir /images
mount -t cifs -o username=root,password=PASSWORDHERE //10.0.0.3/images /images 2>/tmp/mntfail;
mntRet=“$?”;
if [ ! “$mntRet” == “0” ] && [ ! -f “/images/.mntcheck” ]; then
blame=wget -q -O - "http://${web}service/blame.php?mac=$mac&type=$type" 2>/dev/null;
if [ ! “$blame” == “##” ]; then
echo “Failed”;
echo “”;
echo “Error during failure notification: $blame”;
while [ ! “$blame” == “##” ]; do
blame=wget -q -O - "http://${web}service/blame.php?mac=$mac&type=$type" 2>/dev/null;
if [ ! “$blame” == “##” ]; then
echo $blame;
fi
sleep 5;
done
else
echo “Failed”;
echo “”;
cat /tmp/mntfail;
echo “”;
fi
sleep 5;
else
echo “Done”;
blGo=“1”;
fi
debugPause;
fi
done
else
echo “Done”;
dots “Mounting File System”;
mkdir /images $debugstring 2>/dev/null;
#mount -o nolock,proto=tcp,rsize=32768,intr,noatime $storage /images 2>/tmp/mntfail;
mount -t cifs -o username=root,password=PASSWORDHERE //10.0.0.3/images /images 2>/tmp/mntfail;
echo “Done”;
fi

Local Variables:

indent-tabs-mode: t

sh-basic-offset: 4

sh-indentation: 4

tab-width: 4

End:[/CODE]

[Wayne Workman]

So I’ve decided the mount validation probably just doesn’t like the output from my mount command or something, and it THINKS it failed… when infact it succeeded…

So i’m gonna jimmy-rig this script so that it’s impossible to fail… rip out everything that has anything to do with “failing”

And THEN we will see if it fails or not…

[Wayne Workman]

GOOD NEWS AND BAD NEWS… AGAIN!!!

Bad news:

did a debug download, was fiddling around with mounthing…

did this:
[CODE]rm -rf /images[/CODE]
before this:
[CODE]umount /images[/CODE]

and all of my images and data … GONE!!! MOTHER F@&*$#

Good news:

Restored my images from backup… was a process…

Ran another debug task.

created the /images directory manually at CLI
[CODE]mkdir /images[/CODE]

Mounted to the remote images directory via CLI (ensured NFS was NOT running first):
[CODE]mount -t cifs -o username=root,password=PASSWORDHERE //10.0.0.3/images /images[/CODE]

Issued the fog command:
[CODE]fog[/CODE]

and BADA BING bada BOOM

mounting passed and imaging finished without incident.

So… Conclusion… something is going wrong with mounting using the fog.checkin script. I don’t know what it is… I removed all the failure code and replaced it with the success code for EVERY section!

When I do the mount BEFORE the fog command, when the fog command tries to mount, I suppose it errors out, but is still somehow able to succeed?? Maybe because I made failing impossible??? I HAVE NO IDEA

BUT,

I JUST IMAGED USING SMB !!!

WOOOOOOOOT :d

Now, as far as SPEED goes, I was running through a 1Gbps switch.

The source HDD was SATA 2 (3Gbps) and destination was the same (I think). The target host has a 2.93Ghz core 2 Duo processor with I think DDR 2 RAM.

I saw speeds at roughly 3.25 GB / min in the partclone window.

According to Google:
3.25 (gigabytes / minute) =
0.433333333 Gbps

Using the EXACT same hardware, but running the image download via NFS (ensuring SMB is turned OFF)

I saw the same sustained speeds of 3.25ish GB / min.

Could others please validate that there are no performance hits?
I’m using OLD equipment to test with.

[cspence]

Just in case you’re interested: https://github.com/cspenceiv/fog-imager

I have been building a simplified set of imaging scripts. They’ll be fairly similar to what is in use now, but hopefully much easier to read and understand. I’m attempting to get away from a lot of things we currently do.

As of right now, I only have the upload script functional (on an experimental basis). That upload script does not support xfs and jfs (and others that aren’t supported officially by FOG yet). Additionally, it only does multi-disk, multi-partition creates for everything on a system.

Resizability is something I’ll look at later once the basics are taken care of here.

Right now, my test platform is a Arch live disk I built specifically for this testing (that way I’m not testing the buildroot image at the same time). Of course, this is also why I don’t have xfs and jfs support right now (big whoop for this testing).

…and of course, I’m just using samba shares.

[Wayne Workman]

@cspence Very nice work. Have you seen any performance hits during your testing?

[cspence]

@Wayne-Workman said:

@cspence Very nice work. Have you seen any performance hits during your testing?

At this point, it’s all about building a working prototype with VMs. But my other testing didn’t show any slow down using samba. Then again, I’m just using plain SATA drives.

[Wayne Workman]

This doesn’t rely on an internet connection to return the default external IP.

default_info=$(ip route list | awk '/^default/ {print $5}')
default_info=$(ip -o -f inet addr show $default_info | awk '{print $4}' | cut -f1 -d"/")
echo $default_info

[Wayne Workman]

Topic moved to Tutorials simply because of the Samba setup script in the OP.

[Wayne Workman]

This gets the IP of eth0 and sticks it into a variable.

eth0IP="$(ip addr show | grep eth0 | grep -o "inet [0-9]*\.[0-9]*\.[0-9]*\.[0-9]*" | grep -o "[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*")"

[Wayne Workman]

I had to update the script.
The ftp password is now surrounded by single quotes instead of double quotes. Fixed the extraction for that. Also I fixed a typo for ‘share modes’ and I also fixed some warnings by moving the security and passdb backend parameters to global instead of sectional.

I also added some output at the bottom of the script that tells you the username and password used.

installsamba.sh

Here’s the code:

# Last Modified:
# 10-15-2015
#
# Author: Wayne Guy Workman
# Feel free to share, just give credit. :-)
#
# Install samba and samba client.
#
yum install -y samba samba-client
#
# Move the old samba configuration file.
#
mv /etc/samba/smb.conf /etc/samba/smb.conf.old
#
# Make a new config file, then fill it with settings.
#
touch /etc/samba/smb.conf
#beware, below line overwrites anything in /etc/samba/smb.conf
echo '#' > /etc/samba/smb.conf
#below lines append to the end of /etc/samba/smb.conf
echo '#This file was generated by an automated installation script' >> /etc/samba/smb.conf
echo '#for FOG 1.3.0 and higher to share the default /images directory.' >> /etc/samba/smb.conf
echo '#Original Author: Wayne Workman' >> /etc/samba/smb.conf
echo '#' >> /etc/samba/smb.conf
echo 'security = user' >> /etc/samba/smb.conf
echo 'passdb backend = tdbsam' >> /etc/samba/smb.conf
echo '[images]' >> /etc/samba/smb.conf
echo 'path = /images' >> /etc/samba/smb.conf
echo 'read only = no' >> /etc/samba/smb.conf
echo 'unix charset = utf-8' >> /etc/samba/smb.conf
echo 'dos charset = cp932' >> /etc/samba/smb.conf
#
#
# The bleow bit extracts the out-facing IP. Only works if there is one interface.
# This is for extra security, prevents an IP not from your network getting in, even if they know the username / password.
#
# the last part determines the number of octects set for "hosts allow". -f 1-1 is one, -f 1-2 is two, -f 1-3 is three
#
ServerIP="$( ip route get 8.8.8.8 | awk 'NR==1 {print $NF}' | cut -d '.' -f 1-2 )"
#
#
echo '# The below line defines what IP ranges are allowed. They are space delemeted.' >> /etc/samba/smb.conf
echo '# For instance, if you wanted local loopback address, the 10.0.0. range,' >> /etc/samba/smb.conf
echo '# and the 192.168.1 range, and a specifc public IP of 50.50.50.50,' >> /etc/samba/smb.conf
echo '# It would be this:' >> /etc/samba/smb.conf
echo '# hosts allow = 127.0.0.1 10.0.0. 192.168.1. 50.50.50.50' >> /etc/samba/smb.conf
echo "hosts allow = "$ServerIP"." >> /etc/samba/smb.conf
#
# Continue with building the smb.conf file...
#
echo 'create mode = 0777' >> /etc/samba/smb.conf
echo 'directory mode = 0777' >> /etc/samba/smb.conf
echo 'share modes = yes' >> /etc/samba/smb.conf
echo 'writable = yes' >> /etc/samba/smb.conf
#
# Below gets the ftp user & pass from /opt/fot/.fogsettings and "assumes" it matches the local linux user
# Credit for trying?
#
user="$(grep 'storageftpuser=' /opt/fog/.fogsettings | awk -F'"' '{$0=$2}1')"
pass="$(grep 'storageftppass=' /opt/fog/.fogsettings | cut -d \' -f2 )"
#
# Set the samba user with the credentials.
#
(echo "$pass"; echo "$pass") | smbpasswd -s -a $user
systemctl enable smb.service
systemctl restart smb.service
echo " "
echo _______________________________
echo "You might want to note the below info."
echo "Your SMB Username is: " $user
echo "Your SMB Password is: " $pass
echo "If you want to make custom changes to shares,"
echo "The config file is /etc/samba/smb.conf"
echo _______________________________

[Wayne Workman]

I’ve turned this script into a project on SourceForge: https://sourceforge.net/p/samba-for-fog/svn/HEAD/tree/

[Wayne Workman]

svn checkout svn://svn.code.sf.net/p/samba-for-fog/svn/ samba-for-fog-svn
cd samba-for-fog-svn
./installsamba.sh

[Gilou]

Well… if you really want to protect your images, you’d also need to secure the access to the imaging process… It’s true that if /images requires auth, it makes it harder for someone to leak your images, however you’d need to make sure your attacker can’t just fake the MAC of a to-be-imaged computer and just retrieve your fog image with the credentials…

I’d say using samba/cifs is a bit overkill, especially if you don’t have proper security on layer2… But I see the point

[Wayne Workman]

@Gilou Setting up Samba started as a test for imaging through Samba.

But now, it’s mostly just for easy backup and transfer using Windows. Ultimately the Samba solution can be used by people however they see fit.