Failure to expand shrunken resizeable image from Linux machines
-
@tom-elliott I think this is what you are looking for?
Here’s a little folder with more errors I captured during the process…
-
@xipher No, I want to see it as FOG is trying to apply it, before/after, inbetween.
-
@tom-elliott Not sure I follow, sorry!
I didn’t see the output you were looking for during the process, I did capture just about the whole thing in the rest of the pictures included in the link below the picture of my prior reply.
I’m not /super/ familiar with FOG yet…
-
For this host, can you add to the Host Kernel Args:
isdebug=yes ismajordebug=1
Then run a deploy, and capture pictures of the “sfdisk output” after runningfogon the machine?
You will have to step through a lot of this, but it will likely glean more information as to why it’s not expanding. -
@tom-elliott I captured every step this time, here is a small album of a screenshot for each step in the imaging process from before, to during, to after
-
Not sure what else to provide to assist with this, I’ve moved on from testing with VM images to mass physical image testing, however when I capture from a ‘live’ physical machine then cast the image to another machine with a larger drive, there is still no expansion performed…
Was thinking FOG would be my silver bullet, but I’m totally at a loss for what I’m doing wrong after reading everything I can on the subject, as far as I can tell it’s a bug? Unintended functionality failure…?
It’s very frustrating to image a few systems with renewed confidence that a change I’ve made might have resolved it, only to see a partition barley a few gb in size, almost 100% used, and 99% of the drive free and not partitioned
-
@xipher There has to be something unique with your configuration since image resizing on deployment does work very well. We just need to identify what is unique about your configuration that is keeping expansion from working.
While this isn’t a solution only a stop gap measure. Before FOG supported single disk resizable we would build our image on a 40GB vm. This size was picked because it was smaller than any disks we would deploy to. Then when we deployed to a 60GB drive (for example) the image would deploy at 40GB, and then we used the windows diskpart command to expand the
drive to the size of the physical disk. This ‘expand’ command would be called from the setupcomplete.cmd batch file. This worked very, very well with fog 1.2.0 (the same could be said with 1.4.4). -
@xipher Have you confirmed that both your reference image and the target hard drive are in good shape? If by chance your reference image was corrupted because of bad sectors on the master hard drive, or the target drive has errors the deployment could fail.
Do you have a virtualization environment available. The idea is just create a 30GB disk, dump a 1 or 2 gb file on it and then capture and deploy that image to another vm. You don’t need a huge disk for the capture.
FOG does work and it works great.
-
@george1421 your Windows centric response actually begs a question.
Since I’m deploying Linux images, is there less support for doing this than there is for Windows?
I’ve built images with and without using lvm on the captured host, thinking it was that, but now I’m starting to think that the support for this feature and people talking about it are deploying Windows, not Linux…?
-
@george1421 all drives are AOK, SMART checked, etc, I’ve tried it after building a dozen or so images now and probably pulled it down a hundred times across a plethora of devices, from SSD’s to good ol’ spinning rust.
I’ve tried to create custom partition schemes that are just big enough to build the image for capture, I’ve tried to leave it using 100% of the drive
I have a VM infrastructure at home when I’m working on this there to try and resolve the issue, but no access to one when working on the problem in the field.
The only commonality at the moment is that it’s Linux Mint 18.2 x64 being captured, and always with a swap at the start of the drive, then two logical partitions on a single extent after that.
-
@xipher said in Failure to expand shrunken resizeable image from Linux machines:
@george1421 your Windows centric response actually begs a question.
Since I’m deploying Linux images, is there less support for doing this than there is for Windows?
I’ve built images with and without using lvm on the captured host, thinking it was that, but now I’m starting to think that the support for this feature and people talking about it are deploying Windows, not Linux…?
Well my apologies, since 90% of the people deploy windows computers, I assumed without verifying.
-
@xipher said in Failure to expand shrunken resizeable image from Linux machines:
The only commonality at the moment is that it’s Linux Mint 18.2 x64 being captured, and always with a swap at the start of the drive, then two logical partitions on a single extent after that
I will spin this test up in my home VM lab. I’m actually writing this post on on a LM 18.2 OS. I should be able to confirm that this is an issue later tonight.
Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
-
@george1421 keen to hear what you find! I’ll take some pictures of what I run into myself, source material and what I get in the end.
Also, didn’t mean to sound off color with the Windows comment, just genuine curiosity if I had the wrong idea on things, bad tone on my part.
-
Whelp, I can’t seem to duplicate your error. That doesn’t mean anything other than I can’t duplicate your errors in my lab. I tested on both vSphere and also virtual box on my LM laptop.
I simply downloaded a fresh iso of LM 18.2 Mate (I needed that iso for a home project anyway) and installed on the source VM. My source VM I created a 50GB hard drive and installed Mint 18.2 on it. I used all default settings, just an easy and quick install without making any decisions other than password. Once installed I pxe booted the target, registered with my FOG-Pi server and captured the image.
I created a new VM with a 65GB hard drive (different size than source disk by design), pxe booted, registered and then deployed at the end of registration.
Here is what I have the image definition setup as
lm_source
Here is the output
dfandlsblkof the source virtual machine.jondoe@jondoe-VirtualBox ~ $ df -h Filesystem Size Used Avail Use% Mounted on udev 474M 0 474M 0% /dev tmpfs 100M 3.6M 96M 4% /run /dev/sda1 49G 5.3G 41G 12% / tmpfs 496M 92K 496M 1% /dev/shm tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 496M 0 496M 0% /sys/fs/cgroup cgmfs 100K 0 100K 0% /run/cgmanager/fs tmpfs 100M 4.0K 100M 1% /run/user/108 tmpfs 100M 20K 100M 1% /run/user/1000 jondoe@jondoe-VirtualBox ~ $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 1024M 0 rom sda 8:0 0 50G 0 disk ├─sda2 8:2 0 1K 0 part ├─sda5 8:5 0 1021M 0 part [SWAP] └─sda1 8:1 0 49G 0 part / jondoe@jondoe-VirtualBox ~ $lm_target
jondoe@jondoe-VirtualBox ~ $ df -h Filesystem Size Used Avail Use% Mounted on udev 474M 0 474M 0% /dev tmpfs 100M 3.6M 96M 4% /run /dev/sda1 63G 5.3G 55G 9% / tmpfs 496M 92K 496M 1% /dev/shm tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 496M 0 496M 0% /sys/fs/cgroup cgmfs 100K 0 100K 0% /run/cgmanager/fs tmpfs 100M 4.0K 100M 1% /run/user/108 tmpfs 100M 20K 100M 1% /run/user/1000 jondoe@jondoe-VirtualBox ~ $ ndoe@jondoe-VirtualBox ~ $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 1024M 0 rom sda 8:0 0 65G 0 disk ├─sda2 8:2 0 1K 0 part ├─sda5 8:5 0 1021M 0 part └─sda1 8:1 0 63.7G 0 part /As you can see on the target computer, its hard drive did expand to fit the size of the physical disk, which happens to be larger than the source disk.
From looking at the output of lsblk you an see that LM didn’t use LVM when creating the disk.
-
So if I’m to gather things correctly, you’re attempting to go down in size from 50gb (capture system) disk to a 30gb (deploy system) disk?, What was the sizes when you said you deployed to a “larger” disk?
Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.
Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)
Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG
-
@tom-elliott Whoops I missed that bit about going smaller than the source disk. Let me queue up the test environment and see if I can duplicate that condition too.
During testing I also confirmed that shrinking the source image didn’t mess up the source computer. Everything appear to run ok on the source image.
Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
-
I was using a 100gb disk in my VM environment when I was testing the 50gb captured drive.
I’m recreating my test environment now virtually, though I do have the following pictures from a more ‘meat-space’ environment I was working with today…
That was testing a 128gb drive to a 320gb drive, the picture on the bottom shows the settings for the image.
Sorry that they’re actual pictures, the system doing the images is not connected to the network, internet, etc and I didn’t have a flash drive to bring back with me at the time.
-
@george1421 Well setting the image to a smaller drive than the source DID successfully replicate the OP’s issue.
jondoe@jondoe-VirtualBox ~ $ df -h Filesystem Size Used Avail Use% Mounted on udev 474M 0 474M 0% /dev tmpfs 100M 3.6M 96M 4% /run /dev/sda1 5.9G 5.3G 296M 95% / tmpfs 496M 112K 496M 1% /dev/shm tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 496M 0 496M 0% /sys/fs/cgroup cgmfs 100K 0 100K 0% /run/cgmanager/fs tmpfs 100M 20K 100M 1% /run/user/1000 jondoe@jondoe-VirtualBox ~ $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 1024M 0 rom sda 8:0 0 25G 0 disk └─sda1 8:1 0 6G 0 part / jondoe@jondoe-VirtualBox ~ $Note there are missing partitions too.
-
@george1421 The whole ‘going to a smaller disk’ thing seems to be in relation to offsets being passed that exceed the ‘physical’ dimensions of the drive being restored to, see this prior gallery for the errors related.
-
OK, so, after a fair bit of testing and retesting, here is what I’ve found out.
First of all, the following image gallery I created of the issue explains things well:
The TL;DR version is this:
LVM breaks things something /fierce/ with Mint’s default layout. If you choose to use LVM and don’t set a custom partition layout, what you get from the 18.2 installer is whats in the screenshot and it will not expand properly.
Having the root partition anywhere but as the first and primary partition of the drive makes things break. In my case, the SWAP partition was always first in my images.
Having the root partition as the first and primary partition makes the expansion work, almost 100%, it left some space but its good enough for right now.
Important Note: In my testing tonight I ended up using FOG 1.5 RC5 since I /just/ rebuilt in tonight to do this testing. Prior testing was performed on FOG 1.4.4 which for me was producing stranger results than these.
Is this intended functionality?
