Building USB Booting FOS Image


  • Moderator

    Part 1

    The idea for this solution is to build a usb bootable FOS client that will handle the basics of imaging from target registration, suitability checking, debugging, and image capture and deployment.

    This solution is only mean to replace pxe booting where the target can’t pxe boot effectcivly or the network adapter is not supported by the firmware for pxe booting such as with uefi firmware and a usb 2.0 network adapter. The only caveat here is the network adapter MUST be supported by the FOS linux kernel. The usb bootable FOS client also should remove any incompatibilities we’ve seen with the iPXE kernel and and the hardware.

    The USB FOS Client creation must be done on a linux computer. The operating system doesn’t matter much. What ever the FOG Server supports these instructions should work on. The only caveat here is that what ever operating system you use, the OS must use GRUB as the boot loader. I can say for both Centos and Ubuntu that grub is already installed and used.

    The working solution for this solution is based on the work creating the FOSL (FOS Live) usb boot debugging tutorial https://forums.fogproject.org/topic/6532/usb-boot-target-device-into-fog-os-live-fosl-for-debugging and the support of Tom (senior developer) with his quick updates to the trunk build that tied all of the bits together nicely. That brings up the point that this FOS USB Boot drive will only work on/with trunk builds post r8050. If you have an older trunk build than 8050 you must update to get the supporting environment.

    If you are interested in the development process of this product you can review this link: https://forums.fogproject.org/topic/7656/pxe-less-booting-fos-client-os

    Now lets get down to business in Part 2


  • Moderator

    @mwarner I really had to think about the answer to your task as to what is the right direction.

    The 100% right answer is for when the iPXE guys are able to get the iPXE kernel to boot with the updated firmware that Apple has released. Then there will be no need for the USB drive. The MacPros will just PXE boot into the FOG iPXE menu. We are using the USB stick as only a stop-gap measure.

    (Just me thinking out loud here) For FOG to backup a target computer the target computer’s OS must be stopped and the FOG linux (FOS) must be running on the target computer. So in “theory” if the MacPro was a uefi system, and you could create an at least 128MB partition formatted as FAT32, you could copy the contents of the usb drive to that partition. At this point all you would have to do is have the macpro boot from that partition. Once booted then FOG would take over.

    It may be easier to just buy a handful of sandisk cruzer-fit install FOS on that and leave the fits plugged into the USB ports for when you need to image.



  • @george1421 we are using FOG in a business environment (~120 employees), and a few of our users have Mac Pro’s. That’s why we were looking for this USB stick as a solution. But instead of having those two people relying on a USB stick, I was wondering if we could simply mount this .img as a partition on their device’s hard drive so it would be easier for them when they want to backup their device.


  • Moderator

    @mwarner and what do you need to do with it on a hard drive? Sorry I’m not seeing the intended use you are planning.



  • @george1421 the .img file that your script generates (found in /tmp/fos-usb.img according to your script)


  • Moderator

    @mwarner said in Building USB Booting FOS Image:

    mount this on the partition of a hard drive? Or would that likely mess something up

    Please define ‘this’



  • I just performed a capture and deploy and it worked flawlessly. Again, thank you so much.

    I do have one question: would it be possible to mount this on the partition of a hard drive? Or would that likely mess something up?



  • @george1421 Yep, that’s exactly what was happening. I changed it and ran a full host registration successfully. I will try an image capture next.


  • Moderator

    @mwarner that is probably the root of your issue if it tries 3 times to get dhcp, acts like it gets dhcp and then tries again. FOS does a web call to the FOG server to see if its up. It uses that call to identify if the network is working or not.



  • @george1421 I did not change the $myfogip variable in the script, so I will snap a picture if changing that does not work.


  • Moderator

    @mwarner Can you provide an example of the troubles? A picture snapped with a mobile phone is helpful to tell the context of the error.



  • Thank you for this guide! Your script works very well - a lot better than cloning the (much older and more obscure) mac-boot Github repo. We ran into an issue with DHCP leasing but it’s certainly a lot farther than we got with previous attempts on our test Mac.


  • Moderator

    @george1421 Bump…

    Can I bump my own thread??


  • Moderator

    @george1421 do you want wiki access so you can start putting this stuff in there yourself?


  • Moderator

    @aplaisted Great catch on the single quotes, I’ll update the script.



  • Thanks for putting this together. It’s a lot less kludgy than the boot drive’s I’d made for clients that don’t support pxe boot. I did run into one tiny issue with your script. I’m running centos 7, but this shoud apply to anyone running bash. The
    cat > /mnt/boot/grub2/grub.cfg << EOF command will evaluate all of the variables in the grub script instead of writing them out literally. Putting single quotes around the EOF will tell it not to evaluate anything.

    cat > /mnt/boot/grub2/grub.cfg << 'EOF'
    

    Other wise all of the variables in the grub script are written out as empty strings.


  • Moderator

    @Wayne-Workman Not just yet since that article is the path how we got here. Maybe once this method is accepted then the other one is not relevant any more.


  • Moderator

    #wiki worthy

    George, do you want to scrap the “usb bootable media” article in the wiki for this or… what? Is this a separate thing? Up to you, it’s all your work.


  • Moderator

    Part 2a

    If you have any question about the following script you can review the creation process in this tutorial: https://forums.fogproject.org/topic/6532/usb-boot-target-device-into-fog-os-live-fosl-for-debugging

    The following script creates a complete boot image that can be moved to a flash drive with dd, rawwrite, win32diskimager or most other “image to disk” tools.

    Just a note on USB Flash drive size. The entire FOS Client image is only 150MB in size. So any size flash driver larger than 150MB will work. Please understand that any usb flash drive space beyond the FOS client will not be usable even as extra storage. This process will create a single partition on the flash drive of ~150MB. The rest of the (extra) space on your usb flash drive will be unreachable. The point is don’t use a 64GB usb flash drive for FOS cleint booting since almost all of the space will be wasted. I would recommend any size usb 2GB or smaller (for 32 bit reasons). If you can still find them, a 256 or 512 MB flash drive will work just fine for this task.

    Note: You may need to install a few utilities used in the script if they are not installed by your OS install.

    sudo apt-get install grub-efi-amd64 grub-efi-ia32 parted kpartx

    1. On your FOG server or other linux workstation create a file
      touch mk.fos-usb
    2. Change its mode to make it executable
      chmod 755 mk.fos-usb
    3. Open the mk.fos-usb with your favorate text editor and paste in the following
      NOTE: These instructions were created using my Zorin 9 (Ubuntu 14.04 LTS) based laptop they do not work for Centos 7. See part 2b for Centos 7 instructions
    #!/bin/bash
    
    if [ -f /tmp/fogkern.img ]; then
        echo Nuking old FOG Debug image
        rm -f /tmp/fos-usb.img
    fi
    
    echo Make a blank 128MB disk image
    dd if=/dev/zero of=/tmp/fos-usb.img bs=1M count=128
     
    echo Make the partition table, partition and set it bootable.
    parted --script /tmp/fos-usb.img mklabel msdos mkpart p fat32 1 128 set 1 boot on
     
    echo Map the partitions from the image file
    kpartx -a -s /tmp/fos-usb.img
    LOOPDEV=$(losetup -a | grep "/tmp/fos-usb.img" | grep -o "loop[0-9]*")
     
    echo Make an vfat filesystem on the first partition.
    mkfs -t vfat -n GRUB /dev/mapper/${LOOPDEV}p1
     
    echo Mount the filesystem via loopback
    mount /dev/mapper/${LOOPDEV}p1 /mnt
    
    echo Install GRUB
    grub-install --removable --no-nvram --no-uefi-secure-boot --efi-directory=/mnt --boot-directory=/mnt/boot --target=i386-efi
    grub-install --removable --no-nvram --no-uefi-secure-boot --efi-directory=/mnt --boot-directory=/mnt/boot --target=x86_64-efi
    grub-install --removable --no-floppy --boot-directory=/mnt/boot --target=i386-pc /dev/${LOOPDEV}
    
    echo Download the FOG kernels and inits
    wget -P /mnt/boot/ https://fogproject.org/inits/init.xz
    wget -P /mnt/boot/ https://fogproject.org/inits/init_32.xz
    wget -P /mnt/boot/ https://fogproject.org/kernels/bzImage
    wget -P /mnt/boot/ https://fogproject.org/kernels/bzImage32
    wget -P /mnt/boot/ https://github.com/FOGProject/fogproject/blob/dev-branch/packages/web/service/ipxe/memdisk
    wget -P /mnt/boot/ https://github.com/FOGProject/fogproject/blob/dev-branch/packages/web/service/ipxe/memtest.bin
    wget -P /mnt/boot/ https://github.com/FOGProject/fogproject/blob/dev-branch/packages/tftp/ipxe.krn
    wget -P /mnt/boot/ https://github.com/FOGProject/fogproject/blob/dev-branch/packages/tftp/ipxe.efi
    
    echo Create the grub configuration file
    cat > /mnt/boot/grub/grub.cfg << 'EOF'
    
    set myfogip=192.168.1.100
    set myimage=/boot/bzImage
    set myinits=/boot/init.xz
    set myloglevel=4
    set timeout=-1
    insmod all_video
    
    menuentry "1. FOG Image Deploy/Capture" {
     echo loading the kernel
     linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=127000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4
     echo loading the virtual hard drive
     initrd $myinits
     echo booting kernel...
    }
    
    menuentry "2. Perform Full Host Registration and Inventory" {
     echo loading the kernel
     linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=127000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=manreg
     echo loading the virtual hard drive
     initrd $myinits
     echo booting kernel...
    }
    
    menuentry "3. Quick Registration and Inventory" {
     echo loading the kernel
     linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=127000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=autoreg
     echo loading the virtual hard drive
     initrd $myinits
     echo booting kernel...
    }
    
    menuentry "4. Client System Information (Compatibility)" {
     echo loading the kernel
     linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=127000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=sysinfo
     echo loading the virtual hard drive
     initrd $myinits
     echo booting kernel...
    }
    
    menuentry "5. Run Memtest86+" {
     linux /boot/memdisk iso raw
     initrd /boot/memtest.bin
    }
    
    menuentry "6. FOG Debug Kernel" {
     echo loading the kernel
     linux  $myimage loglevel=7 init=/sbin/init root=/dev/ram0 rw ramdisk_size=127000 keymap= boottype=usb consoleblank=0 rootfstype=ext4 isdebug=yes
     echo loading the virtual hard drive
     initrd $myinits
     echo booting kernel...
    }
    
    menuentry "7. FOG iPXE Jumpstart BIOS" {
     echo loading the kernel
     linux16  /boot/ipxe.krn
     echo booting iPXE...
    }
    
    menuentry "8. FOG iPXE Jumpstart EFI" {
     echo chain loading the kernel
     insmod chain 
     chainloader /boot/ipxe.efi
     echo booting iPXE-efi...
    }
    
    EOF
     
    echo Unmount the loopback
    umount /mnt
     
    echo Unmap the image
    kpartx -d /tmp/fos-usb.img
     
    # Write the file to flash drive
    # sudo dd bs=1M if=/tmp/fos-usb.img of=/dev/sdX
    
    1. Save and exit the text editor
    2. Run the file with sudo ./mk.fos-usb
    3. IF the build was successful you should end up with a USB image file in /tmp called /tmp/fos-usb.img

    For linux

    1. Insert your USB flash drive into the FOG server
    2. Use the lsblk command to properly identify your flash drive (get it wrong here and you could overwrite the wrong drive {like your FOG boot drive} so be sure. If you have questions about identifying your USB drive review the FOSL build instructions).
    3. Use the linux dd command to write that image to the usb drive
    4. Key in the following sudo dd bs=1M if=/tmp/fos-usb.img of=/dev/sdX where /dev/sdX is the drive you identified using the lsblk command. If you have any questions refer to the above link for detailed instructions and warnings.
    5. When the write is complete remove the USB flash drive from the FOS server
    6. Done.

    For windows

    1. Move the /tmp/fos-usb.img to your windows computer using putty or what ever method works best for you
    2. Insert the blank usb flash drive into your windows computer
    3. Use one of the above mentioned windows software to write the the fos-usb.img to your flash drive
    4. When the write is complete remove the flash drive (note windows will try to mount it as soon as the image is done downloading. Don’t let windows reformat the drive if it doesn’t understand the format. All is good)

    The final steps you MUST complete is to customize the FOS usb boot drive for your site. You must edit the grub.cfg file in the /boot/grub(2) folder. You can do this in linux or windows. If you use windows I recommend editing the file with Notepad++ or Write (not notepad). Both of the previous text editors will handle linux files well.

    In the grub.cfg file you must edit the very first line with set myfogip=192.168.1.100 and replace 192.168.1.100 with the IP address of your fog server. Once that is done save the grub.cfg file and now the FOS USB boot drive is ready for use.


  • Moderator

    Part 2b

    These steps and script works correctly on a Cento 7 system.

    1. For Centos 7 you must install the following packages
      yum install dosfstools parted kpartx grub2-install grub2-efi-modules grub2-efi -y
    2. Centos 7 is a x64 only OS so if you have a i386 uefi system you MUST USE either Centos 6 or Ubuntu 14.04 LTS to build a usb boot drive that supports i386-efi systems. This is not a big issue because there are not many systems that are both IA32 and efi only.
    3. The second thing to note about Centos 7 it uses grub2 instead of grub (version 1). So the script commands have changed a little.
    4. For Centos 7 paste the following script into mk.fos-usb
    #!/bin/bash
    
    if [ -f /tmp/fogkern.img ]; then
        echo Nuking old FOG Debug image
        rm -f /tmp/fos-usb.img
    fi
    
    echo Make a blank 150MB disk image
    dd if=/dev/zero of=/tmp/fos-usb.img bs=1M count=150
     
    echo Make the partition table, partition and set it bootable.
    parted --script /tmp/fos-usb.img mklabel msdos mkpart p fat32 1 128 set 1 boot on
     
    echo Map the partitions from the image file
    kpartx -a -s /tmp/fos-usb.img
    LOOPDEV=$(losetup -a | grep "/tmp/fos-usb.img" | grep -o "loop[0-9]*")
     
    echo Make an vfat filesystem on the first partition.
    mkfs -t vfat -n GRUB /dev/mapper/${LOOPDEV}p1
     
    echo Mount the filesystem via loopback
    mount /dev/mapper/${LOOPDEV}p1 /mnt
    
    echo Install GRUB
    grub2-install --removable --no-nvram --efi-directory=/mnt --boot-directory=/mnt/boot --target=x86_64-efi
    grub2-install --removable --no-floppy --boot-directory=/mnt/boot --target=i386-pc /dev/${LOOPDEV}
    
    echo Download the FOG kernels and inits
    wget -P /mnt/boot/ https://fogproject.org/inits/init.xz
    wget -P /mnt/boot/ https://fogproject.org/inits/init_32.xz
    wget -P /mnt/boot/ https://fogproject.org/kernels/bzImage
    wget -P /mnt/boot/ https://fogproject.org/kernels/bzImage32
    wget -P /mnt/boot/ https://github.com/FOGProject/fogproject/blob/dev-branch/packages/tftp/ipxe.krn
    wget -P /mnt/boot/ https://github.com/FOGProject/fogproject/blob/dev-branch/packages/tftp/ipxe.efi
    
    echo Create the grub configuration file
    cat > /mnt/boot/grub2/grub.cfg << 'EOF'
    
    set myfogip=192.168.1.100
    set myimage=/boot/bzImage
    set myinits=/boot/init.xz
    set myloglevel=4
    set timeout=-1
    insmod all_video
    
    menuentry "1. FOG Image Deploy/Capture" {
     echo loading the kernel
     linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=127000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4
     echo loading the virtual hard drive
     initrd $myinits
     echo booting kernel...
    }
    
    menuentry "2. Perform Full Host Registration and Inventory" {
     echo loading the kernel
     linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=127000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=manreg
     echo loading the virtual hard drive
     initrd $myinits
     echo booting kernel...
    }
    
    menuentry "3. Quick Registration and Inventory" {
     echo loading the kernel
     linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=127000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=autoreg
     echo loading the virtual hard drive
     initrd $myinits
     echo booting kernel...
    }
    
    menuentry "4. Client System Information (Compatibility)" {
     echo loading the kernel
     linux  $myimage loglevel=$myloglevel initrd=init.xz root=/dev/ram0 rw ramdisk_size=127000 keymap= web=$myfogip/fog/ boottype=usb consoleblank=0 rootfstype=ext4 mode=sysinfo
     echo loading the virtual hard drive
     initrd $myinits
     echo booting kernel...
    }
    
    menuentry "5. Run Memtest86+" {
     linux /boot/memdisk iso raw
     initrd /boot/memtest.bin
    }
    
    menuentry "6. FOG Debug Kernel" {
     echo loading the kernel
     linux  $myimage loglevel=7 init=/sbin/init root=/dev/ram0 rw ramdisk_size=127000 keymap= boottype=usb consoleblank=0 rootfstype=ext4 isdebug=yes
     echo loading the virtual hard drive
     initrd $myinits
     echo booting kernel...
    }
    
    menuentry "7. FOG iPXE Jumpstart BIOS" {
     echo loading the kernel
     linux16  /boot/ipxe.krn
     echo booting iPXE...
    }
    
    menuentry "8. FOG iPXE Jumpstart EFI" {
     echo chain loading the kernel
     insmod chain 
     chainloader /boot/ipxe.efi
     echo booting iPXE-efi...
    }
    
    EOF
     
    echo Unmount the loopback
    umount /mnt
     
    echo Unmap the image
    kpartx -d /tmp/fos-usb.img
     
    # Write the file to flash drive
    # sudo dd bs=1M if=/tmp/fos-usb.img of=/dev/sdX
    

    For the rest of the build follow the Part 2a build.


Log in to reply
 

457
Online

38949
Users

10700
Topics

101523
Posts

Looks like your connection to FOG Project was lost, please wait while we try to reconnect.