Hard drive resize is not expanding

[Doctrg]

@Tom-Elliott I have the files uploaded to my Google Drive. I did find out that I was given some bad information the other day. The resize is also affecting our “BIOS” image. The reason we thought it was working was due to the HDD size being the same as the original drive. I was able to image a 500GB HDD, but when I tried an 80GB or 160GB I got the error “Target partition size (525MB) is smaller than source (670MB). Use option -C to disable size checking.”

I also booted into debug on the computer from the other day that I imaged which is showing only a 11GB drive. I ran ntfsresize -v /dev/sda4 and the output is as follows:

[Tom Elliott]

@Doctrg For the “bios” images, please recapture the images. This can also be fixed if you add the partition number to the d1.fixed_size_partitions file so it would contain 1:2:3 (most likely). (Changing the partition numbers to be proper for your image.)

[Tom Elliott]

Good news, I know what’s wrong. I still have a hard to believing that it “worked” once before, though that’s just my crazy thoughts.

So I built my own GPT based system and captured the image of it.

This worked beautifully, What hasn’t worked as well is the resizing of GPT.

So here’s what’s happening, from what I can see (during deploy process):

1. sfdisk is applied.
2. maths are done to attempt filling the disk.
3. Because the partition start isn’t shifted to the right, the freshly built sfdisk isn’t allowed to be applied.

Why does this occur particularly in GPT, it seems? This I haven’t quite figured. I can tell you it’s because of the strangeness of GPT partitions as they’re typically defined. Rather than shifting the next partitions start partition (whether they’re of fixed size or not) to a point that it might be able to able use, it’s trying to use the original start partition as defined in the sfdisk file, and not shifting start point of the other partitions.

I don’t know if this is trying to use the “full disk” for one partition rather than trying to evenly distribute it though. It feels like this might be what’s happening, though again I’m not 100%.

The reason this may be happening, I suspect, is because GPT lay’s out the disk much different than a typical MBR layout would be.

Typical MBR Layout will, typically, have the “larger” of the partitions at the end of the table.

For example:
Partition 1 will be a smaller boot partition.
Partition 2 will be the “data” partition (the one we want to resize).

I don’t know, and suspect this could cause issues, how this affects multiple-boot systems currently. It might have a layout like:
partition1 smaller windows boot partition/recovery?
partition 2 windows data (wants to be resized)
partition 3 linux swap or data
partition 4 linux swap or data.

Ultimately where the partitions are at shouldn’t matter, I just don’t know of a good way to approach it. This is particularly true in the case of “fixed” partitions, as we try not to adjust the start partitions at all for fixed partitions.

[Tom Elliott]

Things I do know:

We need to capture the image based on the starting point of the partition, regardless of if the partition has been shrunk. This is because the “start” points is where the data begins for that partition.

On deploy, we should NOT need to be so cautious to the start point whether the partition is a fixed size or not because we get to place the data whereever it may be needed.

[Tom Elliott]

Please re-run the FOG installer.

It should download a new set of init’s that should re-enable expanding hdd’s. I will admit, however, the expanding element is not perfect right now (it’s barely even pretty) but it should re-enable your GPT disks to re-expand appropriately.

The issue, from what I can see, has existed since March 9, 2016 (just in case anybody needed to know a date).

[Doctrg]

@Tom-Elliott I’m glad it is not just me… I don’t have much hair on top of my head to pull out. I do thank you for your guidence on the bios image as recapturing it fixed that issue. I re-ran the FOG installer and my main server and storage nodes are now on 1.3.5 RC-10. I tried deploying the GPT image and it still did not expand the disk size. It did however it did not resize and it blew-up Windows. It is acting as if the Start-End of the partitions is not correct.

[Tom Elliott]

@Doctrg I’m not sure i understand here.

You’re saying it DID or DIDN’T resize the partition? I understand the “blow-up” part just want to understand the rest.

[Doctrg]

@Tom-Elliott Sorry, the GPT image did not resize correctly.

[Tom Elliott]

Did it not resize at all or did it not resize as you are “expecting” it to?

[Doctrg]

@Tom-Elliott On the 256GB drive it did not resize. It came up showing as a 40GB drive when running fdisk -l /dev/sda. The drive size showed correct on the 500GB drive due to it being the size it was created on originally. Neither could boot into Windows though.

[Tom Elliott]

@Doctrg I know I’m sounding rude here.

I’m not worried, right now, that it’s not booting, I believe I know why it’s not and am trying to figure that part out, but it can wait for the moment. (it should work, and I don’t know why it’s not, but for now I don’t care about it – I know it is important trust me).

I’m trying to work out what to do to fix the expanding in GPT layouts though, as this was severely broken and has been broken for a very long time.

[Doctrg]

@Tom-Elliott Not a problem. If you guys were here I’d buy y’all a round.

[sudburr]

Using sysprep and an answerfile you can add:

<ExtendOSPartition>
        <Extend>true</Extend>
</ExtendOSPartition>

To the specialize phase.

See ExtendOSPartition.

[Tom Elliott]

Please re-run the installer and try again?

I know it’s a lot of back and forth, but it should work (hopefully).

At least the original and new sizes appear to be expanding on the same originating disk, I have not yet had the chance to test orig < smaller drive and orig > larger drive yet. Just same size disk for now. All appears to be working atleast in that regard.

[Tom Elliott]

A lot of work has been done on this and so far I think we’re on the right path.

The latest init’s are up and ready for more testing, just needing @Doctrg to run the test.

He’s already tested simply deploying back and it’s giving him some issues with the first partition (saying the partition is smaller than the source). I think this is because of the “math” that was being done originally up until I corrected around it last night.

With any luck, however, a fresh upload will enable this image to be deployed to smaller/larger systems.

[Tom Elliott]

I’m excited to hear news (hopefully good, but anything really) when you get a chance. Thank you.

[andreas.paul]

Hi,

i’ve updated to 1.3.5-RC-10, and i have the same issue now. I have an image with 2 Partitions “System Reservated” + C:\ and seems that the partion is also not expand to the size of the harddisk. Yesterday with the older version it has worked.

R,

Andreas

[Tom Elliott]

@andreas.paul What do you mean “with the older version it as worked”?

What older version?

[andreas.paul]

Hi,

yesterday i deploey a windows 10 around 10 times and today after the update he toled me 700mb free. But for now it’s working again. Don`t know what was wrong today morning :-D.

Don’t take care, thanks, i will have an eye on it.

R,

Andreas

[Tom Elliott]

@Doctrg, @andreas.paul If you can try out RC-11 hopefully things will work better for you.

I’ve released RC-11 to try to get more people testing as well. I’m hearing some success and others not so much. Just trying to get things worked out. I know I must be close by now.

Thing’s I’ve personally tested in regards to resizable images.

1. MBR -> Win 7 no extended partition.
   • Originating disk 60GB
   • Smaller disk 30GB
   • Larger disk 80GB
2. MBR -> CentOS 7 no extended partition.
   • Originating disk 60GB
   • Smaller disk 30GB
   • Larger disk 80GB
3. MBR -> CentOS 7 with Extended partition.
   • Originating disk 60GB
   • Smaller disk 30GB
   • Larger disk 80GB
4. GPT/EFI Win 10
   • Originating disk 60GB
   • Smaller disk 30GB
   • Larger disk 80GB
5. GPT/EFI CentOS 7
   • Originating disk 60GB
   • Smaller disk 30GB
   • Larger disk 80GB