Failure to expand shrunken resizeable image from Linux machines

[Xipher]

Server

• FOG Version: 1.4.4
• OS: CentOS 7 X64

Client

• Service Version: ?
• OS: Linux Mint 18.1

Description

I’ve been trying to capture an image of this system (it has about 5~gb of used space on a 50gb drive) and then cast it back to two more systems, one with a 30gb drive, one with a 100gb drive.

The image I captured is marked resizeable, and during the capture the drive is successfully shrunk to about the same 5gb of space thats on it, however when its ‘cast’ back onto any system, the drive is not expanded, it stays shrunk leaving massive amounts of free space.

Even if I edit the “fixed size partitions” to tell it that the partition in question shouldn’t be a fixed size, it won’t realize it.

So far every linux capture/cast I’ve tried has worked this same way, none of them ‘re-expand’ to fill the ‘free space’, I’ve tried with and without using LVM on the machines I’m capturing from, etc.

I’ve got no idea what I’m doing wrong here… I’m going to venture that this is some sort of bug?

[Sebastian Roth]

@Xipher Please run the following commands (while replacing <IMAGENAME> with the actual name of the image) on your FOG server and post all the output here:

cat /images/<IMAGENAME>/d1.fixed_size_partitions
cat /images/<IMAGENAME>/d1.partitions
cat /images/<IMAGENAME>/d1.minimum.partitions
cat /images/<IMAGENAME>/d1.original.fstypes

[Xipher]

Here is the output after recapturing this morning per your request:

[root@localhost images]# cat /images/Minttest1/d1.fixed_size_partitions
:2:5
[root@localhost images]# cat /images/Minttest1/d1.partitions
label: dos
label-id: 0xb67edb1f
device: /dev/sda
unit: sectors

/dev/sda1 : start= 2048, size= 102760448, type=83, bootable
/dev/sda2 : start= 102764542, size= 2091010, type=5
/dev/sda5 : start= 102764544, size= 2091008, type=82
[root@localhost images]# cat /images/Minttest1/d1.minimum.partitions
label: dos
label-id: 0xb67edb1f
device: /dev/sda
unit: sectors

/dev/sda1 : start= 2048, size= 11838052, type=83, bootable
/dev/sda2 : start= 102764542, size= 2091010, type=5
/dev/sda5 : start= 102764544, size= 2091008, type=82
[root@localhost images]# cat /images/Minttest1/d1.original.fstypes
/dev/sda1 extfs

[Tom Elliott]

For this host, can you add to the Host Kernel Args:

isdebug=yes ismajordebug=1

Then run a deploy, and capture pictures of the “sfdisk output” after running fog on the machine?

You will have to step through a lot of this, but it will likely glean more information as to why it’s not expanding.

[Xipher]

@tom-elliott I think this is what you are looking for?

Here’s a little folder with more errors I captured during the process…

https://rareroute.com/files/

[Tom Elliott]

@xipher No, I want to see it as FOG is trying to apply it, before/after, inbetween.

[Xipher]

@tom-elliott Not sure I follow, sorry!

I didn’t see the output you were looking for during the process, I did capture just about the whole thing in the rest of the pictures included in the link below the picture of my prior reply.

I’m not /super/ familiar with FOG yet…

[Tom Elliott]

@Xipher

For this host, can you add to the Host Kernel Args:
isdebug=yes ismajordebug=1
Then run a deploy, and capture pictures of the “sfdisk output” after running fog on the machine?
You will have to step through a lot of this, but it will likely glean more information as to why it’s not expanding.

[Xipher]

@tom-elliott I captured every step this time, here is a small album of a screenshot for each step in the imaging process from before, to during, to after

http://imgur.com/a/LLH0m

[Xipher]

Not sure what else to provide to assist with this, I’ve moved on from testing with VM images to mass physical image testing, however when I capture from a ‘live’ physical machine then cast the image to another machine with a larger drive, there is still no expansion performed…

Was thinking FOG would be my silver bullet, but I’m totally at a loss for what I’m doing wrong after reading everything I can on the subject, as far as I can tell it’s a bug? Unintended functionality failure…?

It’s very frustrating to image a few systems with renewed confidence that a change I’ve made might have resolved it, only to see a partition barley a few gb in size, almost 100% used, and 99% of the drive free and not partitioned

[george1421]

@xipher There has to be something unique with your configuration since image resizing on deployment does work very well. We just need to identify what is unique about your configuration that is keeping expansion from working.

While this isn’t a solution only a stop gap measure. Before FOG supported single disk resizable we would build our image on a 40GB vm. This size was picked because it was smaller than any disks we would deploy to. Then when we deployed to a 60GB drive (for example) the image would deploy at 40GB, and then we used the windows diskpart command to expand the drive to the size of the physical disk. This ‘expand’ command would be called from the setupcomplete.cmd batch file. This worked very, very well with fog 1.2.0 (the same could be said with 1.4.4).

[george1421]

@xipher Have you confirmed that both your reference image and the target hard drive are in good shape? If by chance your reference image was corrupted because of bad sectors on the master hard drive, or the target drive has errors the deployment could fail.

Do you have a virtualization environment available. The idea is just create a 30GB disk, dump a 1 or 2 gb file on it and then capture and deploy that image to another vm. You don’t need a huge disk for the capture.

FOG does work and it works great.

[Xipher]

@george1421 your Windows centric response actually begs a question.

Since I’m deploying Linux images, is there less support for doing this than there is for Windows?

I’ve built images with and without using lvm on the captured host, thinking it was that, but now I’m starting to think that the support for this feature and people talking about it are deploying Windows, not Linux…?

[Xipher]

@george1421 all drives are AOK, SMART checked, etc, I’ve tried it after building a dozen or so images now and probably pulled it down a hundred times across a plethora of devices, from SSD’s to good ol’ spinning rust.

I’ve tried to create custom partition schemes that are just big enough to build the image for capture, I’ve tried to leave it using 100% of the drive

I have a VM infrastructure at home when I’m working on this there to try and resolve the issue, but no access to one when working on the problem in the field.

The only commonality at the moment is that it’s Linux Mint 18.2 x64 being captured, and always with a swap at the start of the drive, then two logical partitions on a single extent after that.

[george1421]

@xipher said in Failure to expand shrunken resizeable image from Linux machines:

@george1421 your Windows centric response actually begs a question.

Since I’m deploying Linux images, is there less support for doing this than there is for Windows?

I’ve built images with and without using lvm on the captured host, thinking it was that, but now I’m starting to think that the support for this feature and people talking about it are deploying Windows, not Linux…?

Well my apologies, since 90% of the people deploy windows computers, I assumed without verifying.

[george1421]

@xipher said in Failure to expand shrunken resizeable image from Linux machines:

The only commonality at the moment is that it’s Linux Mint 18.2 x64 being captured, and always with a swap at the start of the drive, then two logical partitions on a single extent after that

I will spin this test up in my home VM lab. I’m actually writing this post on on a LM 18.2 OS. I should be able to confirm that this is an issue later tonight.

[Xipher]

@george1421 keen to hear what you find! I’ll take some pictures of what I run into myself, source material and what I get in the end.

Also, didn’t mean to sound off color with the Windows comment, just genuine curiosity if I had the wrong idea on things, bad tone on my part.

[george1421]

Whelp, I can’t seem to duplicate your error. That doesn’t mean anything other than I can’t duplicate your errors in my lab. I tested on both vSphere and also virtual box on my LM laptop.

I simply downloaded a fresh iso of LM 18.2 Mate (I needed that iso for a home project anyway) and installed on the source VM. My source VM I created a 50GB hard drive and installed Mint 18.2 on it. I used all default settings, just an easy and quick install without making any decisions other than password. Once installed I pxe booted the target, registered with my FOG-Pi server and captured the image.

I created a new VM with a 65GB hard drive (different size than source disk by design), pxe booted, registered and then deployed at the end of registration.

Here is what I have the image definition setup as

lm_source

Here is the output df and lsblk of the source virtual machine.

jondoe@jondoe-VirtualBox ~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            474M     0  474M   0% /dev
tmpfs           100M  3.6M   96M   4% /run
/dev/sda1        49G  5.3G   41G  12% /
tmpfs           496M   92K  496M   1% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           496M     0  496M   0% /sys/fs/cgroup
cgmfs           100K     0  100K   0% /run/cgmanager/fs
tmpfs           100M  4.0K  100M   1% /run/user/108
tmpfs           100M   20K  100M   1% /run/user/1000
jondoe@jondoe-VirtualBox ~ $ lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0     11:0    1 1024M  0 rom  
sda      8:0    0   50G  0 disk 
├─sda2   8:2    0    1K  0 part 
├─sda5   8:5    0 1021M  0 part [SWAP]
└─sda1   8:1    0   49G  0 part /
jondoe@jondoe-VirtualBox ~ $ 

lm_target

jondoe@jondoe-VirtualBox ~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            474M     0  474M   0% /dev
tmpfs           100M  3.6M   96M   4% /run
/dev/sda1        63G  5.3G   55G   9% /
tmpfs           496M   92K  496M   1% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           496M     0  496M   0% /sys/fs/cgroup
cgmfs           100K     0  100K   0% /run/cgmanager/fs
tmpfs           100M  4.0K  100M   1% /run/user/108
tmpfs           100M   20K  100M   1% /run/user/1000
jondoe@jondoe-VirtualBox ~ $ 

ndoe@jondoe-VirtualBox ~ $ lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0     11:0    1 1024M  0 rom  
sda      8:0    0   65G  0 disk 
├─sda2   8:2    0    1K  0 part 
├─sda5   8:5    0 1021M  0 part 
└─sda1   8:1    0 63.7G  0 part /

As you can see on the target computer, its hard drive did expand to fit the size of the physical disk, which happens to be larger than the source disk.

From looking at the output of lsblk you an see that LM didn’t use LVM when creating the disk.

[Tom Elliott]

So if I’m to gather things correctly, you’re attempting to go down in size from 50gb (capture system) disk to a 30gb (deploy system) disk?, What was the sizes when you said you deployed to a “larger” disk?

[george1421]

@tom-elliott Whoops I missed that bit about going smaller than the source disk. Let me queue up the test environment and see if I can duplicate that condition too.

During testing I also confirmed that shrinking the source image didn’t mess up the source computer. Everything appear to run ok on the source image.