Building USB Booting FOS Image

[george1421]

Part 1

The idea for this solution is to build a usb bootable FOS client that will handle the basics of imaging from target registration, suitability checking, debugging, and image capture and deployment.

This solution is only mean to replace pxe booting where the target can’t pxe boot effectcivly or the network adapter is not supported by the firmware for pxe booting such as with uefi firmware and a usb 2.0 network adapter. The only caveat here is the network adapter MUST be supported by the FOS linux kernel. The usb bootable FOS client also should remove any incompatibilities we’ve seen with the iPXE kernel and and the hardware.

The USB FOS Client creation must be done on a linux computer. The operating system doesn’t matter much. What ever the FOG Server supports these instructions should work on. The only caveat here is that what ever operating system you use, the OS must use GRUB as the boot loader. I can say for both Centos and Ubuntu that grub is already installed and used.

The working solution for this solution is based on the work creating the FOSL (FOS Live) usb boot debugging tutorial https://forums.fogproject.org/topic/6532/usb-boot-target-device-into-fog-os-live-fosl-for-debugging and the support of Tom (senior developer) with his quick updates to the trunk build that tied all of the bits together nicely. That brings up the point that this FOS USB Boot drive will only work on/with trunk builds post r8050. If you have an older trunk build than 8050 you must update to get the supporting environment.

If you are interested in the development process of this product you can review this link: https://forums.fogproject.org/topic/7656/pxe-less-booting-fos-client-os

Now lets get down to business in Part 2

[george1421]

place holder

0_1476207284967_BOOTIA32.EFI
0_1476207446133_BOOTX64.EFI
0_1476207451596_grub.efi

[george1421]

Part 2b

These steps and script works correctly on a Cento 7 system.

1. For Centos 7 you must install the following packages
   yum install dosfstools parted kpartx grub2-install grub2-efi-modules grub2-efi -y
2. Centos 7 is a x64 only OS so if you have a i386 uefi system you MUST USE either Centos 6 or Ubuntu 14.04 LTS to build a usb boot drive that supports i386-efi systems. This is not a big issue because there are not many systems that are both IA32 and efi only.
3. The second thing to note about Centos 7 it uses grub2 instead of grub (version 1). So the script commands have changed a little.
4. For Centos 7 paste the following script into mk.fos-usb

#!/bin/bash

if [ -f /tmp/fogkern.img ]; then
    echo Nuking old FOG Debug image
    rm -f /tmp/fos-usb.img
fi

echo Make a blank 150MB disk image
dd if=/dev/zero of=/tmp/fos-usb.img bs=1M count=150
 
echo Make the partition table, partition and set it bootable.
parted --script /tmp/fos-usb.img mklabel msdos mkpart p fat32 1 128 set 1 boot on
 
echo Map the partitions from the image file
kpartx -a -s /tmp/fos-usb.img
LOOPDEV=$(losetup -a | grep "/tmp/fos-usb.img" | grep -o "loop[0-9]*")
 
echo Make an vfat filesystem on the first partition.
mkfs -t vfat -n GRUB /dev/mapper/${LOOPDEV}p1
 
echo Mount the filesystem via loopback
mount /dev/mapper/${LOOPDEV}p1 /mnt

echo Install GRUB
grub2-install --removable --no-nvram --efi-directory=/mnt --boot-directory=/mnt/boot --target=x86_64-efi
grub2-install --removable --no-floppy --boot-directory=/mnt/boot --target=i386-pc /dev/${LOOPDEV}

echo Download the FOG kernels and inits
wget -P /mnt/boot/ https://github.com/FOGProject/fos/releases/latest/download/bzImage
wget -P /mnt/boot/ https://github.com/FOGProject/fos/releases/latest/download/bzImage32
wget -P /mnt/boot/ https://github.com/FOGProject/fos/releases/latest/download/init.xz
wget -P /mnt/boot/ https://github.com/FOGProject/fos/releases/latest/download/init_32.xz
wget -P /mnt/boot/ https://github.com/FOGProject/fogproject/blob/dev-branch/packages/tftp/ipxe.krn
wget -P /mnt/boot/ https://github.com/FOGProject/fogproject/blob/dev-branch/packages/tftp/ipxe.efi

echo Create the grub configuration file
cat > /mnt/boot/grub2/grub.cfg << 'EOF'

set myfogip=http://192.168.1.100
set myimage=/boot/bzImage
set myinits=/boot/init.xz
set myloglevel=4
set timeout=-1
insmod all_video

menuentry "1. FOG Image Deploy/Capture" {
 echo loading the kernel
 linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4
 echo loading the virtual hard drive
 initrd $myinits
 echo booting kernel...
}

menuentry "2. Perform Full Host Registration and Inventory" {
 echo loading the kernel
 linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=manreg
 echo loading the virtual hard drive
 initrd $myinits
 echo booting kernel...
}

menuentry "3. Quick Registration and Inventory" {
 echo loading the kernel
 linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=autoreg
 echo loading the virtual hard drive
 initrd $myinits
 echo booting kernel...
}

menuentry "4. Client System Information (Compatibility)" {
 echo loading the kernel
 linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=sysinfo
 echo loading the virtual hard drive
 initrd $myinits
 echo booting kernel...
}

menuentry "5. Run Memtest86+" {
 linux /boot/memdisk iso raw
 initrd /boot/memtest.bin
}

menuentry "6. FOG Debug Kernel" {
 echo loading the kernel
 linux  $myimage loglevel=7 init=/sbin/init root=/dev/ram0 rw ramdisk_size=275000 keymap= boottype=usb consoleblank=0 rootfstype=ext4 isdebug=yes
 echo loading the virtual hard drive
 initrd $myinits
 echo booting kernel...
}

menuentry "7. FOG iPXE Jumpstart BIOS" {
 echo loading the kernel
 linux16  /boot/ipxe.krn
 echo booting iPXE...
}

menuentry "8. FOG iPXE Jumpstart EFI" {
 echo chain loading the kernel
 insmod chain 
 chainloader /boot/ipxe.efi
 echo booting iPXE-efi...
}

EOF
 
echo Unmount the loopback
umount /mnt
 
echo Unmap the image
kpartx -d /tmp/fos-usb.img
 
# Write the file to flash drive
# sudo dd bs=1M if=/tmp/fos-usb.img of=/dev/sdX

For the rest of the build follow the Part 2a build.

[george1421]

Part 2a

If you have any question about the following script you can review the creation process in this tutorial: https://forums.fogproject.org/topic/6532/usb-boot-target-device-into-fog-os-live-fosl-for-debugging

The following script creates a complete boot image that can be moved to a flash drive with dd, rawwrite, win32diskimager or most other “image to disk” tools.

Just a note on USB Flash drive size. The entire FOS Client image is only 150MB in size. So any size flash driver larger than 150MB will work. Please understand that any usb flash drive space beyond the FOS client will not be usable even as extra storage. This process will create a single partition on the flash drive of ~150MB. The rest of the (extra) space on your usb flash drive will be unreachable. The point is don’t use a 64GB usb flash drive for FOS cleint booting since almost all of the space will be wasted. I would recommend any size usb 2GB or smaller (for 32 bit reasons). If you can still find them, a 256 or 512 MB flash drive will work just fine for this task.

Note: You may need to install a few utilities used in the script if they are not installed by your OS install.

sudo apt-get install grub-efi-amd64 grub-efi-ia32 parted kpartx

1. On your FOG server or other linux workstation create a file
   touch mk.fos-usb
2. Change its mode to make it executable
   chmod 755 mk.fos-usb
3. Open the mk.fos-usb with your favorate text editor and paste in the following
   NOTE: These instructions were created using my Zorin 9 (Ubuntu 14.04 LTS) based laptop they do not work for Centos 7. See part 2b for Centos 7 instructions

#!/bin/bash

if [ -f /tmp/fogkern.img ]; then
    echo Nuking old FOG Debug image
    rm -f /tmp/fos-usb.img
fi

echo Make a blank 128MB disk image
dd if=/dev/zero of=/tmp/fos-usb.img bs=1M count=128
 
echo Make the partition table, partition and set it bootable.
parted --script /tmp/fos-usb.img mklabel msdos mkpart p fat32 1 128 set 1 boot on
 
echo Map the partitions from the image file
kpartx -a -s /tmp/fos-usb.img
LOOPDEV=$(losetup -a | grep "/tmp/fos-usb.img" | grep -o "loop[0-9]*")
 
echo Make an vfat filesystem on the first partition.
mkfs -t vfat -n GRUB /dev/mapper/${LOOPDEV}p1
 
echo Mount the filesystem via loopback
mount /dev/mapper/${LOOPDEV}p1 /mnt

echo Install GRUB
grub-install --removable --no-nvram --no-uefi-secure-boot --efi-directory=/mnt --boot-directory=/mnt/boot --target=i386-efi
grub-install --removable --no-nvram --no-uefi-secure-boot --efi-directory=/mnt --boot-directory=/mnt/boot --target=x86_64-efi
grub-install --removable --no-floppy --boot-directory=/mnt/boot --target=i386-pc /dev/${LOOPDEV}

echo Download the FOG kernels and inits
wget -P /mnt/boot/ https://github.com/FOGProject/fos/releases/latest/download/bzImage
wget -P /mnt/boot/ https://github.com/FOGProject/fos/releases/latest/download/bzImage32
wget -P /mnt/boot/ https://github.com/FOGProject/fos/releases/latest/download/init.xz
wget -P /mnt/boot/ https://github.com/FOGProject/fos/releases/latest/download/init_32.xz
wget -P /mnt/boot/ https://github.com/FOGProject/fogproject/blob/dev-branch/packages/web/service/ipxe/memdisk
wget -P /mnt/boot/ https://github.com/FOGProject/fogproject/blob/dev-branch/packages/web/service/ipxe/memtest.bin
wget -P /mnt/boot/ https://github.com/FOGProject/fogproject/blob/dev-branch/packages/tftp/ipxe.krn
wget -P /mnt/boot/ https://github.com/FOGProject/fogproject/blob/dev-branch/packages/tftp/ipxe.efi

echo Create the grub configuration file
cat > /mnt/boot/grub/grub.cfg << 'EOF'

set myfogip=http://192.168.1.100
set myimage=/boot/bzImage
set myinits=/boot/init.xz
set myloglevel=4
set timeout=-1
insmod all_video

menuentry "1. FOG Image Deploy/Capture" {
 echo loading the kernel
 linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4
 echo loading the virtual hard drive
 initrd $myinits
 echo booting kernel...
}

menuentry "2. Perform Full Host Registration and Inventory" {
 echo loading the kernel
 linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=manreg
 echo loading the virtual hard drive
 initrd $myinits
 echo booting kernel...
}

menuentry "3. Quick Registration and Inventory" {
 echo loading the kernel
 linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=autoreg
 echo loading the virtual hard drive
 initrd $myinits
 echo booting kernel...
}

menuentry "4. Client System Information (Compatibility)" {
 echo loading the kernel
 linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=275000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=sysinfo
 echo loading the virtual hard drive
 initrd $myinits
 echo booting kernel...
}

menuentry "5. Run Memtest86+" {
 linux /boot/memdisk iso raw
 initrd /boot/memtest.bin
}

menuentry "6. FOG Debug Kernel" {
 echo loading the kernel
 linux  $myimage loglevel=7 init=/sbin/init root=/dev/ram0 rw ramdisk_size=275000 keymap= boottype=usb consoleblank=0 rootfstype=ext4 isdebug=yes
 echo loading the virtual hard drive
 initrd $myinits
 echo booting kernel...
}

menuentry "7. FOG iPXE Jumpstart BIOS" {
 echo loading the kernel
 linux16  /boot/ipxe.krn
 echo booting iPXE...
}

menuentry "8. FOG iPXE Jumpstart EFI" {
 echo chain loading the kernel
 insmod chain 
 chainloader /boot/ipxe.efi
 echo booting iPXE-efi...
}

EOF
 
echo Unmount the loopback
umount /mnt
 
echo Unmap the image
kpartx -d /tmp/fos-usb.img
 
# Write the file to flash drive
# sudo dd bs=1M if=/tmp/fos-usb.img of=/dev/sdX

4. Save and exit the text editor
5. Run the file with sudo ./mk.fos-usb
6. IF the build was successful you should end up with a USB image file in /tmp called /tmp/fos-usb.img

For linux
7. Insert your USB flash drive into the FOG server
8. Use the lsblk command to properly identify your flash drive (get it wrong here and you could overwrite the wrong drive {like your FOG boot drive} so be sure. If you have questions about identifying your USB drive review the FOSL build instructions).
9. Use the linux dd command to write that image to the usb drive
10. Key in the following sudo dd bs=1M if=/tmp/fos-usb.img of=/dev/sdX where /dev/sdX is the drive you identified using the lsblk command. If you have any questions refer to the above link for detailed instructions and warnings.
11. When the write is complete remove the USB flash drive from the FOS server
12. Done.

For windows
7. Move the /tmp/fos-usb.img to your windows computer using putty or what ever method works best for you
8. Insert the blank usb flash drive into your windows computer
9. Use one of the above mentioned windows software to write the the fos-usb.img to your flash drive
10. When the write is complete remove the flash drive (note windows will try to mount it as soon as the image is done downloading. Don’t let windows reformat the drive if it doesn’t understand the format. All is good)

The final steps you MUST complete is to customize the FOS usb boot drive for your site. You must edit the grub.cfg file in the /boot/grub(2) folder. You can do this in linux or windows. If you use windows I recommend editing the file with Notepad++ or Write (not notepad). Both of the previous text editors will handle linux files well.

In the grub.cfg file you must edit the very first line with set myfogip=http://192.168.1.100 and replace 192.168.1.100 with the IP address of your fog server. Once that is done save the grub.cfg file and now the FOS USB boot drive is ready for use.

[Wayne Workman]

#wiki worthy

George, do you want to scrap the “usb bootable media” article in the wiki for this or… what? Is this a separate thing? Up to you, it’s all your work.

[george1421]

@Wayne-Workman Not just yet since that article is the path how we got here. Maybe once this method is accepted then the other one is not relevant any more.

[aplaisted]

Thanks for putting this together. It’s a lot less kludgy than the boot drive’s I’d made for clients that don’t support pxe boot. I did run into one tiny issue with your script. I’m running centos 7, but this shoud apply to anyone running bash. The
cat > /mnt/boot/grub2/grub.cfg << EOF command will evaluate all of the variables in the grub script instead of writing them out literally. Putting single quotes around the EOF will tell it not to evaluate anything.

cat > /mnt/boot/grub2/grub.cfg << 'EOF'

Other wise all of the variables in the grub script are written out as empty strings.

[george1421]

@aplaisted Great catch on the single quotes, I’ll update the script.

[Wayne Workman]

@george1421 do you want wiki access so you can start putting this stuff in there yourself?

[george1421]

@george1421 Bump…

Can I bump my own thread??

[mwarner]

Thank you for this guide! Your script works very well - a lot better than cloning the (much older and more obscure) mac-boot Github repo. We ran into an issue with DHCP leasing but it’s certainly a lot farther than we got with previous attempts on our test Mac.

[george1421]

@mwarner Can you provide an example of the troubles? A picture snapped with a mobile phone is helpful to tell the context of the error.

[mwarner]

@george1421 I did not change the $myfogip variable in the script, so I will snap a picture if changing that does not work.

[george1421]

@mwarner that is probably the root of your issue if it tries 3 times to get dhcp, acts like it gets dhcp and then tries again. FOS does a web call to the FOG server to see if its up. It uses that call to identify if the network is working or not.

[mwarner]

@george1421 Yep, that’s exactly what was happening. I changed it and ran a full host registration successfully. I will try an image capture next.

[mwarner]

I just performed a capture and deploy and it worked flawlessly. Again, thank you so much.

I do have one question: would it be possible to mount this on the partition of a hard drive? Or would that likely mess something up?

[george1421]

@mwarner said in Building USB Booting FOS Image:

mount this on the partition of a hard drive? Or would that likely mess something up

Please define ‘this’

[mwarner]

@george1421 the .img file that your script generates (found in /tmp/fos-usb.img according to your script)

[george1421]

@mwarner and what do you need to do with it on a hard drive? Sorry I’m not seeing the intended use you are planning.

[mwarner]

@george1421 we are using FOG in a business environment (~120 employees), and a few of our users have Mac Pro’s. That’s why we were looking for this USB stick as a solution. But instead of having those two people relying on a USB stick, I was wondering if we could simply mount this .img as a partition on their device’s hard drive so it would be easier for them when they want to backup their device.