Hard drive resize is not expanding

[Jaymes Driver]

@Tom-Elliott I didn’t know that… Learn something new every day!

[Doctrg]

@Tom-Elliott I’ve also learned something new today as well. I installed Windows 10 on a PC with a 500GB hard drive and uploaded it to FOG. I then deployed it to another computer with a 256GB SSD and it came up as a 15GB drive. I’ll post the contents of that image.

[Tom Elliott]

@Doctrg Well the good news is data is “shrinking appropriately”, the bad news, I have no idea what’s wrong.

[Tom Elliott]

Can you download from a debug session again?

When the image completes, get us the output of fdisk -l /dev/sda

[Doctrg]

@Tom-Elliott I did some testing before your last post, but I’ve got some outputs for ya. What I did is installed Windows 10 barebone to both machines and created a totally new image entry in FOG.

This output is from the 500GB HDD to 256GB SSD:

And this output is from the 256GB SSD to 500GB HDD:

[Wayne Workman]

@Doctrg We need to know what version you were on before you started having this issue. Please answer this question directly.

This command may help you figure out what version you came from. Please post the output.
ls -la /home

[Doctrg]

@Wayne-Workman We were on 1.3.2 and we were having issues uploading so we updated to an RC of 1.3.4. As far as I know everything was working at that point, but once I saw there was a stable version of 1.3.4, I updated to that.

[Wayne Workman]

@Doctrg It looks like you came from 1.3.0 previously. The 1.3.2 installer would have made a directory with 1.3.2 in the name had the 1.3.2 installer been ran - this is fine.

What I’m thinking, you probably won’t agree with. What I would want to do is ask you to setup a 1.3.0 server and manually transfer this broke image to it and see if it works on that setup or not. This would involve DHCP changes or an offline environment which can be challenging to setup if you’ve not done it before. It’s a lot of work. OR… you could share this known-good image with us (not via forums but via private messages and Mega or Dropbox or Google Drive) and let us test the image with the various fog versions and figure out what’s going on. Many of the active Developers and Moderators have test environments at home (like myself) that are setup to test whatever fog related stuff we want without any real work involved.

The reason why your thread has gotten the attention it has is because of how you are responding (professionally, with logic, soundly). I think that you know that this image really was a good image prior - so I really want to look into this to see if there’s anything at all wrong with fog. We’re trying to determine when it worked, and when it broke. For that, we need someone to do the testing. Either you with your own environment just for this thing, or you can share the image with someone here for testing.

[Doctrg]

@Wayne-Workman It would be helpful if I can send the image to one of you guys. I do not have control over our network so making DHCP changes would be out of my hands and we are in the middle of moving our offices so I do not have an offline environment setup yet to do testing. I’m downloading the image for the storage node now and will upload it to my Google drive once it is done.

[Doctrg]

@Wayne-Workman I’m still working on getting the files uploaded. Who do I need to PM to send the link to the shared folder? Also, what program is being used for the resizing? My co-worker that has been the FOG administrator since the beginning is wondering if we can try to do a debug deploy and do a verbose on the resize.

[Tom Elliott]

@Doctrg You could enable the “majorDebug” flag, but I don’t know what it’s going to, or not, display.

[Doctrg]

@Tom-Elliott I have the files uploaded to my Google Drive. I did find out that I was given some bad information the other day. The resize is also affecting our “BIOS” image. The reason we thought it was working was due to the HDD size being the same as the original drive. I was able to image a 500GB HDD, but when I tried an 80GB or 160GB I got the error “Target partition size (525MB) is smaller than source (670MB). Use option -C to disable size checking.”

I also booted into debug on the computer from the other day that I imaged which is showing only a 11GB drive. I ran ntfsresize -v /dev/sda4 and the output is as follows:

[Tom Elliott]

@Doctrg For the “bios” images, please recapture the images. This can also be fixed if you add the partition number to the d1.fixed_size_partitions file so it would contain 1:2:3 (most likely). (Changing the partition numbers to be proper for your image.)

[Tom Elliott]

Good news, I know what’s wrong. I still have a hard to believing that it “worked” once before, though that’s just my crazy thoughts.

So I built my own GPT based system and captured the image of it.

This worked beautifully, What hasn’t worked as well is the resizing of GPT.

So here’s what’s happening, from what I can see (during deploy process):

1. sfdisk is applied.
2. maths are done to attempt filling the disk.
3. Because the partition start isn’t shifted to the right, the freshly built sfdisk isn’t allowed to be applied.

Why does this occur particularly in GPT, it seems? This I haven’t quite figured. I can tell you it’s because of the strangeness of GPT partitions as they’re typically defined. Rather than shifting the next partitions start partition (whether they’re of fixed size or not) to a point that it might be able to able use, it’s trying to use the original start partition as defined in the sfdisk file, and not shifting start point of the other partitions.

I don’t know if this is trying to use the “full disk” for one partition rather than trying to evenly distribute it though. It feels like this might be what’s happening, though again I’m not 100%.

The reason this may be happening, I suspect, is because GPT lay’s out the disk much different than a typical MBR layout would be.

Typical MBR Layout will, typically, have the “larger” of the partitions at the end of the table.

For example:
Partition 1 will be a smaller boot partition.
Partition 2 will be the “data” partition (the one we want to resize).

I don’t know, and suspect this could cause issues, how this affects multiple-boot systems currently. It might have a layout like:
partition1 smaller windows boot partition/recovery?
partition 2 windows data (wants to be resized)
partition 3 linux swap or data
partition 4 linux swap or data.

Ultimately where the partitions are at shouldn’t matter, I just don’t know of a good way to approach it. This is particularly true in the case of “fixed” partitions, as we try not to adjust the start partitions at all for fixed partitions.

[Tom Elliott]

Things I do know:

We need to capture the image based on the starting point of the partition, regardless of if the partition has been shrunk. This is because the “start” points is where the data begins for that partition.

On deploy, we should NOT need to be so cautious to the start point whether the partition is a fixed size or not because we get to place the data whereever it may be needed.

[Tom Elliott]

Please re-run the FOG installer.

It should download a new set of init’s that should re-enable expanding hdd’s. I will admit, however, the expanding element is not perfect right now (it’s barely even pretty) but it should re-enable your GPT disks to re-expand appropriately.

The issue, from what I can see, has existed since March 9, 2016 (just in case anybody needed to know a date).

[Doctrg]

@Tom-Elliott I’m glad it is not just me… I don’t have much hair on top of my head to pull out. I do thank you for your guidence on the bios image as recapturing it fixed that issue. I re-ran the FOG installer and my main server and storage nodes are now on 1.3.5 RC-10. I tried deploying the GPT image and it still did not expand the disk size. It did however it did not resize and it blew-up Windows. It is acting as if the Start-End of the partitions is not correct.

[Tom Elliott]

@Doctrg I’m not sure i understand here.

You’re saying it DID or DIDN’T resize the partition? I understand the “blow-up” part just want to understand the rest.

[Doctrg]

@Tom-Elliott Sorry, the GPT image did not resize correctly.

[Tom Elliott]

Did it not resize at all or did it not resize as you are “expecting” it to?