Mac support

[Tom S]

So here are the changes made from 2124. I have been told in the past to omit comments in my diffs in other projects I have contributed to. Please let me know if you need more comments.

So this is a serious hack. Most of the SQL calls do not use the fog api rather they use the php mysql commands. The Fog api kept denying my request to create tasks because of invalid MAC address issues. Also the main post function includes some sql “Alter” structure commands in order to edit the tables to fit the needs of the script.

I am not proud of the overall structure of the patch files so if someone knows how to do it with the fog api…awesome.

Please let me know if you have questions.

Thank you,

TS

[url=“/_imported_xf_attachments/1/1254_diff.zip?:”]diff.zip[/url]

[Tom Elliott]

Hey Tom,

I’ve taken a look at these, and some have been relatively simple to add, but there’s a lot of confusion in the files.

For example, When I look at the diff file for fog.checkin here’s what I see:
[code]— new/SubMenu.class.php2014-08-04 10:10:50.000000000 -0400
+++ old/SubMenu.class.php2014-07-10 18:11:12.000000000 -0400
@@ -192,7 +192,6 @@
$this->subMenu[$this->node][‘search’] = $this->foglang[‘NewSearch’];
$this->subMenu[$this->node][‘list’] = sprintf($this->foglang[‘ListAll’],$this->foglang[‘Images’]);
$this->subMenu[$this->node][‘add’] = sprintf($this->foglang[‘CreateNew’],$this->foglang[‘Image’]);
-$this->subMenu[$this->node][‘Multicast’] = sprintf($this->foglang[‘Multicast’],$this->foglang[‘Image’]);
if ($_REQUEST[‘id’])
{
$this->subMenu[$this->node][‘id’][$linkformat.‘#image-gen’] = $this->foglang[‘General’];
[/code]
fog.download:
[code]— new/fog.checkin2014-08-05 18:11:58.000000000 -0400
+++ old/fog.checkin2014-07-24 14:32:42.000000000 -0400
@@ -61,7 +61,7 @@
mkdir /images 2>/dev/null;
mount -o nolock,proto=tcp,rsize=32768,wsize=32768,intr,noatime $storage /images;
echo “Done”;
-elif [ “$type” == “down” ] && [ “$capone” != “1” ] && [ “$joinmc” != “1” ]; then
+elif [ “$type” == “down” ] && [ “$capone” != “1” ]; then
mac64=ifconfig | grep HWaddr | head -n1 | base64;
dots “Attempting to send inventory”;
doInventory 2>/dev/null;
@@ -153,81 +153,6 @@
debugPause;
fi
done
-elif [ “$joinmc” == “1” ] && [ “$mc” == “yes” ]; then
-echo “”;
-echo " #############################################################################“;
-echo " # #”;
-echo " # Joining Multicast Session! #“;
-echo " # #”;
-echo " #############################################################################";
-echo “”;
-canGo=“no”;
-while [ “$mSession” != “exit” ] && [ “$canGo” != “yes” ]; do
-echo -n “Enter in Session Name:[Type “exit” to quit]”;
-echo “”;
-read mSession;
-multicastValues=wget -O - "http://${web}service/mcTask.php?task=join&mSession=$mSession" 2>/dev/null
-if [ “$multicastValues” != “” ]; then
-arr=$(echo $multicastValues | tr “;” “\n”);
-i=0;
-for x in $arr
-do

• array[i]=$x;
  -i=$i+1;
  -done
  -img=${array[0]};
  -imgTypeId=${array[1]};
  -osid=${array[2]};
  -port=${array[3]};
  -storageip=${array[4]};
  -sessionPwd=${array[5]};
  -if [ “$imgTypeId” = “0” ]; then
  -imgType=“n”;
  -fi
  -if [ “$imgTypeId” = “2” ]; then
  -imgType=“mps”;
  -fi
  -if [ “$imgTypeId” = “3” ]; then
  -imgType=“mpa”;
  -fi
  -if [ “$sessionPwd” != “” ]; then
  -echo “”;
  -echo -n " This Session Requires A Password To Join,";
  -didAuth=“0”;
  -for I in 1 2 3;do
  -echo “Please enter in password:”;
  -read password;
  -if [ “$password” == “$sessionPwd” ]; then
  -echo “Password Accepted!!”;
  -didAuth=“1”;
  -break;
  -else
  -echo “Incorrect Password, Try again”;
  -fi
  -done
  -if [ “$didAuth” != “1” ]; then
  -echo “Incorrect password. To many attempts made, now rebooting”;
  -exit 0;
  -fi
  -fi
  -determineOS $osid;
  -canGo=“yes”;

•       echo -n "Multicast Configuration Complete";
  

• echo "";
  

•     echo "Joining Session: $mSession
  

-Using Image: $img
-OS: $osname
-Image Type: $imgType
-Session Port: $port";
-dots “Mounting File System”;
-mkdir /images $debugstring 2>/dev/null;
-mount -o nolock,proto=tcp,rsize=32768,intr,noatime $storage /images 2>/tmp/mntfail;
-echo “Done”;
-else
-echo -n “No mulicast sesion with that name was found!! Try again.\n”;
-fi
-done

else
echo “Done”;
dots “Mounting File System”;[/code]
fog.mcupdate:
[code]#!/bin/sh
IFS=’
';
web=$1;
mSession=$2;
sleep 3;
while true; do
fle=“/tmp/status.fog”
status=tail -n 2 $fle 2>/dev/null | head -n 1 2>/dev/null
cat /dev/null > $fle 2>/dev/null

status=echo $status | base64;
res=wget -T 3 -O - --post-data="status=${status}" "http://${web}service/mcTask.php?task=progress&mSession=${mSession}" 2>/dev/null;
sleep 3;
done[/code]
ImageManagementPage.class.php:
[code]<?php

@error_reporting(0);
require(‘…/commons/base.inc.php’);

$conn = @mysql_connect( DATABASE_HOST, DATABASE_USERNAME, DATABASE_PASSWORD);

if ( $conn ){
if ( ! @mysql_select_db( DATABASE_NAME, $conn ) ) die( mysql_error() );

if( isset($_GET[‘task’]) && isset($_GET[‘mSession’])){
$task = $_GET[‘task’];
$mSession = $_GET[“mSession”];

if($task==“join”){
$sql = “select max(msID) as msID from multicastSessions where msName = ‘$mSession’”;
$res = mysql_query( $sql, $conn ) or die( mysql_error() );
while( $ar = mysql_fetch_array( $res ) ){
$msID = $ar[“msID”];
}
$storageNodeID=“”;
$storageNodeIP=“”;
$sql = “select * from multicastSessions where msID = ‘$msID’”;
$res = mysql_query( $sql, $conn ) or die( mysql_error() );
if($res !=“”){
while( $ar = mysql_fetch_array( $res ) ){
$msImage = $ar[“msImage”];
$msIsDD = $ar[“msIsDD”];
$port = $ar[“msBasePort”];
$sessionPwd = $ar[“sessionPwd”];
$storageNodeID=$ar[‘msNFSGroupID’];
}
$imageType=“”;
$sql = “select * from images where imageID = ‘$msImage’”;
$res = mysql_query( $sql, $conn ) or die( mysql_error() );
while( $ar = mysql_fetch_array( $res ) ){
$hostOS = $ar[“imageOSID”];
$imageType = $ar[‘imageTypeID’];
$msImage = $ar[‘imageName’];

}
$sql = “select * from nfsGroupMembers where ngmID = ‘$storageNodeID’”;
$res = mysql_query( $sql, $conn ) or die( mysql_error() );
while( $ar = mysql_fetch_array( $res ) ){
$storageNodeIP = $ar[“ngmHostname”];

}

echo “$msImage”.“;”.“$imageType”.“;”.“$hostOS”.“;”.“$port”.“;”.“$storageNodeIP”.“;”.“$sessionPwd”;

}
}else if($task==“progress”){
$str = explode(‘@’,base64_decode($_REQUEST[‘status’]));
$percent=trim($str[5]);
mysql_query("UPDATE multicastSessions set msPercent=‘$percent’,msState=‘3’ WHERE msName=‘$mSession’ ");
}else if($task==“finish”){
$sql = “select * from multicastSessions where msName = ‘$mSession’”;
$res = mysql_query( $sql, $conn ) or die( mysql_error() );
$clientCount=0;
while( $ar = mysql_fetch_array( $res ) ){
$clientCount = $ar[“msClients”];
}
if($clientCount>1){
$clientCount–;
mysql_query("UPDATE multicastSessions set msClients=‘$clientCount’ WHERE msName=‘$mSession’ ");
}else{
mysql_query("DELETE FROM tasks WHERE taskName=‘$mSession’ ");
mysql_query("DELETE FROM hosts WHERE hostName=‘$mSession’ ");
$sql = “SELECT * FROM multicastSessions WHERE msName=‘$mSession’ “;
$res = mysql_query( $sql, $conn );
$thisMSID=””;
while( $row = mysql_fetch_array( $res ) ){
$thisMSID= $row[‘msID’];
}
mysql_query("DELETE FROM multicastSessions WHERE msName=‘$mSession’ ");
mysql_query("DELETE FROM multicastSessionsAssoc WHERE msID=‘$mSession’ ");
}
echo “##”;
}

}else{
echo “invalid syntax”;
}

}else{
echo “connecting to DB”;
}
?>
[/code]
mcTask.php appears to be a binary file.
MulticastTask.class.php:
[code]— new/funcs.sh2014-08-02 23:36:10.000000000 -0400
+++ old/funcs.sh2014-07-15 17:16:38.000000000 -0400
@@ -508,9 +508,6 @@
osname=“Windows 8.1”;
mbrfile=“/usr/share/fog/mbr/win8.mbr”;
defaultpart2start=“368050176B”;
-elif [ “$1” = “8” ]; then
-osname=“Apple Mac OS”;
-mbrfile=“”;
elif [ “$1” = “50” ]; then
osname=“Linux”;
mbrfile=“”[/code]
fog.checkin is the diff for SubMenu.class.php, fog.download is the diff for fog.checkin, fog.mcupdate might be correct? ImageManagementPage.class.php is supposed to be mcTask.php? mcTask appears to be a binary file. MulticastTask.class.php appears to be funcs.sh.

It also is backwards in the diff, It’s using the old files as yours and new files as the ones from svn.

I imagine, you could also make this a single diff with:
[code]diff -rupN <originalTrunk> <yourEditedTrunk> > mychanged.diff[/code]

This will give a single file and give all the changes compared to what mine looks like.

[Tom S]

Sorry Tom. I did diff -u on my Mac. Maybe that’s the issue. I will reupload tonight

[fractal13]

[quote=“Tom S, post: 34775, member: 25305”]

I did want to talk about the Fog service for Mac support. In my current profession I have written scripts that do what I can tell the Windows Fog service does, but for OS X. They are primarily bash scripts that can easily be piped from a C or another language. My question to everyone here is this:

What language do we want the service built on???

TS[/quote]

TS: The FOG service for Windows is under active development in C#. So, we won’t be using that code for OS X/Linux. I would encourage you to consider C. That will be at least a portable as Java, if we static compile it. It will be easy to write a simple service like you’re talking about being a wrapper for shell scripts. I could give you some boiler-plate networking code and give advice on invoking shell scripts from within, if you want it. This is something I’ve done in previous projects many times.

Thanks for all of your help.

[Tom Elliott]

I’ve begun work on this and added the necessary GUI elements already.

I can create the session and cancel the session. There is no need for the storage group or the os parameters in scheduling this as the group and os are stored with the image.

I’m not going to create the fog.mcupdate or mcTask.php file as I can do all of this from the ipxe menu.

This is just a checklist for me to follow, the multicast stuff is not operational from a name joining setup yet.

[Tom S]

Ok so here is the requested diff file. I had to zip it in order to upload the file. So I hope this helps. I look forward in seeing the final product!!

[url=“/_imported_xf_attachments/1/1260_joinMC.diff.zip?:”]joinMC.diff.zip[/url]

[Tom S]

[quote=“fractal13, post: 34855, member: 24300”]TS: The FOG service for Windows is under active development in C#. So, we won’t be using that code for OS X/Linux. I would encourage you to consider C. That will be at least a portable as Java, if we static compile it. It will be easy to write a simple service like you’re talking about being a wrapper for shell scripts. I could give you some boiler-plate networking code and give advice on invoking shell scripts from within, if you want it. This is something I’ve done in previous projects many times.
Thanks for all of your help.[/quote]
Fractal13
I get what you are saying about using C. My only point was if I where to write it in java and got a working app, adding the windows version would a piece of cake and then fog would have a single executable for all os types one Maven or eclipse file rather than a VS project and an xcode project and a another for the various flavors of nix. I know a lot of people are not very happy with java so thats why I asked.

Nonetheless I can write it in C and keep you all informed. Is an xCode project acceptable for submittal?

I feel this forum proves why open source Rocks!!!

Thanks,
TS

[Tom Elliott]

ONly problem I see with using the java approach is it would assume all users always have a version of java installed. Issue I foresee, beyond windows admins who don’t allow java because of security concerns, is what about linux non-gui installs? How would the app function then? Couldn’t the linux/mac service just be a series of scripts that are controlled through at or crontab?

[Tom Elliott]

I’ve now got it to a workable state where you can join existing sessions based on the session name. You can delete the task from the link, and added db updates to add the session password and maxwait fields where appropriate.

I haven’t added, yet, checking on the session password as I just want to make sure the menu system is working.

[Tom S]

@Tom Elliot
Your Java statements are 100% correct and valid. One thing Mac does well is supplying a shell acceptable way to manipulate almost every aspect of the machine’s configuration. Since the arrival of ARD, you can find scripts for almost everything. As far as using cron jobs for the fog service on os x and nix, I would steer more toward daemons. In os x, there are 4 levels of daemon. Ones that run when machine boots (via launchd) and ones that run at user login (launchctl). The naming, binding, and other administrative tasks would run as root in a daemon. The gui would run at user login.

Would wget with https work for communication between server and client? If not, apple does supply some safari based C libraries.

Printers. Since both nix and os x use cups, this should be pretty straight forward. I would suggest users supply a package file with the drivers or the actual ppd/gz file through the web gui. Install package or use wget to download ppd to printer/ppd folder and then run script to install printer.

So if you all are ok with a bash daemon that periodically probes the fog server cool, I can start adapting my scripts to work with your current service pages on fog.

Side note… same kind of service could install software too. Just sayin

TS

[Tom Elliott]

If you’d be willing TS, Can you try svn of fog?

Test it out. There are a few bugs I’m aware of right now, such as the FOG Service stuff. Also, session password setting doesn’t work either. I’m still working that out. WHen I get it working all should be well.

But give it a go. Most of my implementations have been based solely on adding the ability to join multicast sessions. This joining is only relevant from the session generated from the Image Management Page. You can only add hosts, right now, from the ipxe menu. There is a login required to get to the Multicast Join field to add some layer of security, but hopefully this will be okay. I don’t think we need the session password, necessarily, if we have the login parameter in effect. What do you think?

[Tom S]

[quote=“Tom Elliott, post: 34911, member: 7271”]If you’d be willing TS, Can you try svn of fog?

Test it out. There are a few bugs I’m aware of right now, such as the FOG Service stuff. Also, session password setting doesn’t work either. I’m still working that out. WHen I get it working all should be well.

But give it a go. Most of my implementations have been based solely on adding the ability to join multicast sessions. This joining is only relevant from the session generated from the Image Management Page. You can only add hosts, right now, from the ipxe menu. There is a login required to get to the Multicast Join field to add some layer of security, but hopefully this will be okay. I don’t think we need the session password, necessarily, if we have the login parameter in effect. What do you think?[/quote]
@Tom Elliott
I agree with using the hide menu function in ipxe (username and password) if someone wants to password protect it. The name in a way is a password if you think about it.

I will definitely check out the svn a little later. Does the current code allow for non-registered clients to join?

Thanks,
TS

[Tom Elliott]

No it doesn’t though I suppose I could add that feature.

[Tom Elliott]

For right now it’s only registered hosts. I have to think of the logic to get the non-registered hosts to join the task as I have to falsify a tasking much the same way capone falsify’s a tasking.

[Tom Elliott]

Before I go adding support for “non-registered” hosts.

I’m going to first add a field to allow groups or host to join previously created sessions so you can schedule registered hosts from the gui. Don’t know why you’d want to do this method of multicast, but if this is worth doing, it’s worth overdoing (mythbusters is my favorite).

[Tom Elliott]

A caveat to all of this information.

Right now, although it’s only enabled for registered hosts, it does not need any modification to the init files. Though I will probably need to add support for the OS type of Mac OS, if you just need to get it working, you can test with windows or linux. Hopefully, soon, I can remove even the need for the OSID as all it’s used for is to determine partition layout on the Resizable image, and even then, it’s only specific to Windows Partitioning layout. It’s not used for MPS,MPA, or Raw image types.

So if you have a Mac OS image, partclone already is aware of the file type. The OSID has no impact on how the system will run if it’s MultiPart single/all disk, So you can essentially assign linux os to mac os image and all should work properly. We create a copy of the GPT/MBR information which is what get’s restored to the hdd on download. So things should all work properly anyway.

[Tom S]

[quote=“Tom Elliott, post: 34918, member: 7271”]For right now it’s only registered hosts. I have to think of the logic to get the non-registered hosts to join the task as I have to falsify a tasking much the same way capone falsify’s a tasking.[/quote]
Call me crazy but could you…
Select Join Multicast from ipxe. ipxe sends mac address to php. If not registered, php sends back to ipxe. ipxe asks for computer name. Enter in computer name and maybe os type. ipxe then asks for confirmation, then send back to php. Php creates host, then creates task for multicast session. Maybe protect the final step with fog username and password. Or possibly send client to fog.man.reg then onto multicast.

I think this sort of registration process would be pretty cool!!

When multicasting a large organization, as I am sure you know, knowing EXACTLY what machines will join the multicast session is impossible without touching machines multiple times. So being able to join any machine to multicast is I think key…

Side note, as you said earlier the funcs.sh file still needs editing. If os is unknown, if I remember correctly it will fail!!

Anyways awesome work Tom!!

TS

[Tom Elliott]

[quote=“Tom S, post: 34924, member: 25305”]Call me crazy but could you…
Select Join Multicast from ipxe. ipxe sends mac address to php. If not registered, php sends back to ipxe. ipxe asks for computer name. Enter in computer name and maybe os type. ipxe then asks for confirmation, then send back to php. Php creates host, then creates task for multicast session. Maybe protect the final step with fog username and password. Or possibly send client to fog.man.reg then onto multicast.

I think this sort of registration process would be pretty cool!!

When multicasting a large organization, as I am sure you know, knowing EXACTLY what machines will join the multicast session is impossible without touching machines multiple times. So being able to join any machine to multicast is I think key…

Side note, as you said earlier the funcs.sh file still needs editing. If os is unknown, if I remember correctly it will fail!!

Anyways awesome work Tom!!

TS[/quote]

One of the coolest features of ipxe is we already know whether or not the host is registered when it first starts. While we originally toyed with the idea of having ipxe auto register the host, the reason we went against it is because of compatibility issues within the OS layer of FOG.

I’ve added the Apple Mac OS to the list of OS’s but I have not incorporated it into fog quite yet.

I think I have another method around having a non-registered system join an existing multicast task. The method will not require generating a task in and of itself and should still work, if the host is compatible with fog. I will look into doing such incorporation with the ipxe system for un-registered hosts. However, I may wait a couple of days if you’re willing to wait as well. Only reason, this incorporation today took a lot out of me mentally speaking. I just need a day away from all the testing and multicast stuff. Personally, it’s not one of my favorite areas.

[Tom S]

TAKE A BREAK.
You have done a lot on Fog over the last few months!! I think we have some good areas to start testing with. Machine types, multicasting and everything else for macs. I am pulling all of my scripts together and comparing the logic between the Windows service and how it talks to fog. It will take me a few days to get things worked out but I can at least get the scripts all compliant with fog and then we can just worry about the service wrapper and gui!!!

I hope that you are getting the praise you deserve for picking up this project and starting the fire under the fog community’s butt. Given the talents of the people using fog…skies the limit!!!

Thank you Tom,

TS

[Tom Elliott]

To be fair,

I’d highly recommend you don’t use the current FOGService files to compare against. We’re in the process of testing a new build for this. So much of what’s in those folders will soon be irrelevant as we’re modularizing it.

If you need the links we use to get the services and how to interpret them, I can help you there. I’d highly recommend that you start from scratch rather than trying to follow the current code base as they’re to completely different systems.