PXE image looping

[Tom Elliott]

@ManofValor the settings look fine. This is particularly why I love teamviewer though. I can’t find an error on my side namely because. I don’t have a uefi system to work with. I’m pretty certain I’ve screwed up somewhere but I haven’t a clue what or where .

[Wayne Workman]

@Tom-Elliott said:

@ManofValor the settings look fine. This is particularly why I love teamviewer though. I can’t find an error on my side namely because. I don’t have a uefi system to work with. I’m pretty certain I’ve screwed up somewhere but I haven’t a clue what or where .

Just going to go ahead and vouch for anyone here with a moderator or developer (or Senior Developer) tag by their name. These people are trustworthy.

I mean, if you’re running fog trunk, you’re quite literally running thousands and thousands of lines of code written by the developers on a server inside your organization. Not trusting the developers via team viewer is like owning a Ford vehicle, and then rejecting assistance from Ford’s senior chief engineer (who happens to be your brother too).

[ManofValor]

Update, output of a debug capture
fog r6116
CentOS 7
HP z440

[Tue Jan 26 root@fogclient ~]# fog

   +------------------------------------------+
   |     ..#######:.    ..,#,..     .::##::.  |
   |.:######          .:;####:......;#;..     |
   |...##...        ...##;,;##::::.##...      |
   |   ,#          ...##.....##:::##     ..:: |
   |   ##    .::###,,##.   . ##.::#.:######::.|
   |...##:::###::....#. ..  .#...#. #...#:::. |
   |..:####:..    ..##......##::##  ..  #     |
   |    #  .      ...##:,;##;:::#: ... ##..   |
   |   .#  .       .:;####;::::.##:::;#:..    |
   |    #                     ..:;###..       |
   |                                          |
   +------------------------------------------+
   |      Free Computer Imaging Solution      |
   +------------------------------------------+
   |  Credits: http://fogproject.org/Credits  |
   |       http://fogproject.org/Credits      |
   |       Released under GPL Version 3       |
   +------------------------------------------+
   Version: 6116
 * Press [Enter] key to continue

 * Verifying network interface configuration.........Done
 * Press [Enter] key to continue

 * Checking Operating System.........................Windows 7
 * Checking CPU Cores................................8
 * Send method.......................................NFS
 * Attempting to check in............................Done
 * Press [Enter] key to continue

 * Mounting File System..............................Done
 * Press [Enter] key to continue

 * Checking Mounted File System......................Done
 * Press [Enter] key to continue

 * Checking img variable is set......................Done
 * Press [Enter] key to continue

 * Preparing to send image file to server
 * Preparing backup location.........................Done
 * Press [Enter] key to continue

 * Setting permission on /images/ecb1d756281d........Done
 * Press [Enter] key to continue

 * Removing any pre-existing files...................Done
 * Press [Enter] key to continue

 * Using Image: MCWP82
 * Looking for Hard Disk.............................Done
 * Press [Enter] key to continue

 * Reading Partition Tables..........................Done
 * Press [Enter] key to continue

 * Using Hard Disk: /dev/sda
 * Press [Enter] key to continue

 * Now FOG will attempt to upload the image using Partclone
 * Press [Enter] key to continue

 * Checking for fixed partitions.....................Done
 * Press [Enter] key to continue

 * Getting Windows/Linux Partition Count.............Done
 * Press [Enter] key to continue

 * NTFS Partition count of: 2
 * Press [Enter] key to continue

 * EXTFS Partition count of: 0
 * Press [Enter] key to continue

 * Setting up any additional fixed parts
 * Saving original partition table...................Done
 * Press [Enter] key to continue

 * Shrinking Partitions on disk
 * Press [Enter] key to continue

 * Clearing part (/dev/sda1).........................Failed
 * Press [Enter] key to continue

 * Failed to mount partition to clear
 * Mounting partition (/dev/sda1)....................Failed
 * Press [Enter] key to continue

 * Could not mount to location
 * Not shrinking (/dev/sda1) fixed size
 * Press [Enter] key to continue

 * Clearing part (/dev/sda2).........................Not NTFS partition
 * Press [Enter] key to continue

 * Not an NTFS file system
 * Press [Enter] key to continue

 * Not shrinking (/dev/sda2) fixed size
 * Press [Enter] key to continue

 * Clearing part (/dev/sda3).........................Done
 * Press [Enter] key to continue

 * Mounting partition (/dev/sda3)....................Done
 * Press [Enter] key to continue

 * Removing page file................................Done
 * Press [Enter] key to continue

 * Removing hibernate file...........................Done
 * Press [Enter] key to continue

 * Possible resize partition size: 35124723 k
 * Running resize test /dev/sda3.....................Done
 * Press [Enter] key to continue

 * Resize test was successful
 * Press [Enter] key to continue

 * Resizing filesystem...............................Done
 * Press [Enter] key to continue

 * Resizing partition /dev/sda3......................Done
 * Clearing ntfs flag................................Done
 * Press [Enter] key to continue

 * Saving shrunken partition table
 * Press [Enter] key to continue

 * Saving Partition Tables (GPT).....................Done
 * Press [Enter] key to continue

 * Processing Hard Disk: /dev/sda
 * Processing Partition: /dev/sda1 (1)
 * Press [Enter] key to continue

 * Using partclone.ntfs
 * Press [Enter] key to continue

##############################################################################
#                                                                            #
#                         An error has been detected!                        #
#                                                                            #
##############################################################################
Failed to complete upload (savePartition)
 * Press [Enter] key to continue

[Tue Jan 26 root@fogclient ~]#

output of lsblk

[Tue Jan 26 root@fogclient ~]# lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    1 931.5G  0 disk 
|-sda1   8:1    1   100M  0 part 
|-sda2   8:2    1   128M  0 part 
`-sda3   8:3    1  36.3G  0 part 
[Tue Jan 26 root@fogclient ~]# 

[Tom Elliott]

@ManofValor while I’m sure it doesn’t seem helpful to you at least I now have a point where I can look at a problem. Based on what I can see though, what size disk created this image and what size disk is this trying to go onto. Based on what I see it feels like the files are unable to create a viable partition layout possibly because the receiving drive is smaller than what is expected.

[ManofValor]

@Tom-Elliott That makes sense. I will look into that. I’m leaving early today so I will look at it tomorrow. Thanks.

[Wayne Workman]

Makes zero sense to me.

FYI, the drive in the source host is 1TB, and the /dev/sda3 partition was properly re-sized. It re-sized down to roughly 34GB. This was a debug capture/upload.

I’d be really surprised if that didn’t fit onto the fog server.
But none the less, @ManofValor can you run this command on your FOG server and provide the outupt? df -h

[ManofValor]

@Wayne-Workman FOG Server output:

[fogadmin@localhost /]$ df -h
Filesystem               Size  Used Avail Use% Mounted on
/dev/mapper/centos-root   50G  4.7G   46G  10% /
devtmpfs                 1.9G     0  1.9G   0% /dev
tmpfs                    1.9G  152K  1.9G   1% /dev/shm
tmpfs                    1.9G  8.9M  1.9G   1% /run
tmpfs                    1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/sda1                497M  211M  287M  43% /boot
/dev/mapper/centos-home  412G  389M  411G   1% /home
tmpfs                    379M   28K  379M   1% /run/user/1000
[fogadmin@localhost /]$

[Wayne Workman]

@ManofValor The way you have partitioning setup (the CentOS 7 Defaults), and using /images as your image directory, you are limited to 50GB worth of images.

In the Fedora 21 Server article in the wiki, it shows a better method. The best method is to give /images it’s own partition.

[ManofValor]

@Wayne-Workman Cool. I’ll look at that.

[ManofValor]

@Wayne-Workman Not sure if im using fdisk right. I run:

[root@localhost ~]# fdisk /dev/sdb1

and get

fdisk: cannot open /dev/sdb1: No such file or directory

Output of:

[root@localhost ~]# sudo fdisk -l
Disk /dev/sda: 500.1 GB, 500074307584 bytes, 976707632 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00018c75
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 1026047 512000 83 Linux
/dev/sda2 1026048 976707583 487840768 8e Linux LVM

I have not used fdisk in like 12 years and that was in windows. I don’t want to screw something up.

[ManofValor]

@Wayne-Workman I’m an idiot. I was following the instructions on the wiki and was trying to use “sdb” instead of “sda” on my system. So did i do it right and why can’t I continue?

[root@localhost /]# fdisk /dev/sda1
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x5cca602a.

Command (m for help): m
Command action
   a   toggle a bootable flag
   b   edit bsd disklabel
   c   toggle the dos compatibility flag
   d   delete a partition
   g   create a new empty GPT partition table
   G   create an IRIX (SGI) partition table
   l   list known partition types
   m   print this menu
   n   add a new partition
   o   create a new empty DOS partition table
   p   print the partition table
   q   quit without saving changes
   s   create a new empty Sun disklabel
   t   change a partition's system id
   u   change display/entry units
   v   verify the partition table
   w   write table to disk and exit
   x   extra functionality (experts only)

Command (m for help): g
Building a new GPT disklabel (GUID: 002FA017-DD70-4CA2-A4E0-41D4B51E68ED)


Command (m for help): p

Disk /dev/sda1: 524 MB, 524288000 bytes, 1024000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: gpt


#         Start          End    Size  Type            Name

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 22: Invalid argument.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
[root@localhost /]# fdisk -l

Disk /dev/sda: 500.1 GB, 500074307584 bytes, 976707632 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00018c75

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     1026047      512000   83  Linux
/dev/sda2         1026048   976707583   487840768   8e  Linux LVM

Disk /dev/mapper/centos-root: 53.7 GB, 53687091200 bytes, 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/centos-swap: 4160 MB, 4160749568 bytes, 8126464 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/centos-home: 441.6 GB, 441630851072 bytes, 862560256 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

[root@localhost /]# fdisk l
fdisk: cannot open l: No such file or directory
[root@localhost /]# fdisk /dev/sda
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): g
Building a new GPT disklabel (GUID: B59077CE-6FB8-4EA4-B6DA-92E14E33A412)


Command (m for help): p

Disk /dev/sda: 500.1 GB, 500074307584 bytes, 976707632 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: gpt


#         Start          End    Size  Type            Name

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
[root@localhost /]# mkfs.ext4 /dev/sda
mke2fs 1.42.9 (28-Dec-2013)
/dev/sda is entire device, not just one partition!
Proceed anyway? (y,n) y
/dev/sda is apparently in use by the system; will not make a filesystem here!
[root@localhost /]# mkfs.ext4 /dev/sda1
mke2fs 1.42.9 (28-Dec-2013)
/dev/sda1 is mounted; will not make a filesystem here!
[root@localhost /]# 

[Wayne Workman]

@ManofValor Whether you’re able to shrink the /home directory and then expand the / directory depends on how the partitions are laid on the disk. For a partition to be expanded, the actual physical location on the HDD that it’s expanding into must be available. Does that make sense?

And while some Linux magic might exist to do it, I’ll always opt to do it using the OS Installer’s GUI, and if I wanted to possibly expand a partition in the future, I’d probably not even bother with bare metal, and use a VM and assign two HDDs before I installed the OS, and then I’d follow @george1421 's tutorial on expanding a virtual HDD.

So what I might tell you is… if you have a ton of space in /home, then maybe it’d be easier to just move your images to there. There is a good (and recent) thread on this here: https://forums.fogproject.org/topic/6580/changing-the-directory-where-fog-images-are-stored-question/27

[Tom Elliott]

@ManofValor when formatting a disk, it cannot be used during or before the time of the run.

You mount, from what I can tell, is already established for /dev/sda1, so run:
umount /dev/sda1 (I don’t know what folder it’s mounted to)

Then make your changes to the partition layout. Also don’t try to format a disk. (/dev/sda), you should really only ever format a partition.

[Sebastian Roth]

@ManofValor Ouch! I really think you should look into how fdisk and partitioning in general works before using this tool. From what I can see you’ve totally screwed the partition scheme on your FOG server and you will run into big trouble sooner or later. You might not see any issue yet as your disk was in use when you changed the partitions. But rebooting the machine will finally kill it I am pretty sure. So if you still have access try backing up things over the network or onto an external disk before it is too late!

Then you probably need to start from scratch. Install CentOS and probably best make sure you add a good size partition for /images right then. So you don’t have to move things around later on. Then install FOG trunk again.

[george1421]

@Sebastian-Roth said:

Then you probably need to start from scratch. Install CentOS and probably best make sure you add a good size partition for /images right then. So you don’t have to move things around later on. Then install FOG trunk again.

FWIW: Unless you specifically go into the hard drive layout during centos install Centos 7 will divide the hard drive between the root partition and the /home partition. Not really helpful for an application server configuration. If you are setting up fog on a VM create the initial FOG disk in the 16 to 20GB size, then after centos is installed add a new vmdk file that is 50-100GB and mount this new disk to the /opt directory before you install fog. Then during fog install tell fog to install its images in /opt/fog/images. This will keep all of the big files on the /opt disk without the risk of filling up the root partition on linux.

[Wayne Workman]

@george1421 said:

Unless you specifically go into the hard drive layout during centos install

That’s the only way I ever do it

I actually give /images it’s very own partition, and I seriously reduce the size of all other partitions.

[ManofValor]

@george1421 I knew I was going to screw it up. I want to make the images file as big as possible since this is strictly for image backup. So after install I could make it like 350G in size, right?

[george1421]

@ManofValor If you do as I suggest just create a 20GB vmdk for the OS. The OS only really needs about 5GB, but leave room for updates and such. Once that is done go back in and create a new vmdk file for your storage. Make it what ever size you need today. By putting the images in their own vmdk file you can dynamically grow that disk as you need to as time goes by.

Once you have that second vmdk file (will probably be listed as /dev/sdb) you will need to partition it with fdisk and then format it as ext4. Once you have that disk you need to attach it over/to the /opt directory so it mounts every time (using /etc/fstab file).

Once you have that new vmdk attached, then you can install fog and tell it to put its images in /opt/fog/images

This will be a great learning experience for you. Google is your friend.

[Wayne Workman]

@george1421 Solid advice, absolutely, and would work fine too.

I would prefer though to mount the 2nd hdd to /images because /opt/fog/images is a deviation from the expected. And because /images would be the expected location for all of our documentation, all of our forums posts, and any future advice we give out, and any tech with previous fog experience coming into an environment to take care of FOG would expect the images to be in that location, and the installer uses this location as default.

But again, nothing mechanically or logically wrong with moving it to /opt/fog/images, it’d be perfectly functional and equally as expandable in the future.

[ManofValor]

@Wayne-Workman What partitioning scheme do you recommend? I’ve been trying to look up the difference between the four options I have (standard, btrfs, LVM, and LVM Thin Provisioning). I was thinking standard for the OS and LVM thin for the images?

This is question for all.