Hard drive resize is not expanding



  • Description
    Server
    • FOG Version: 1.3.5 RC 7
    • OS: RedHat
    Client
    • Service Version:
    • OS: Windows 10 Enterprise
    Description

    We just noticed today that since updating to version 1.3.4 the hard drives are not expanding to use the rest of the HDD. I saw someone else had a similar issue, so I updated to 1.3.5 RC 7 and that did not fix the issue. Once the update did not fix the issue I thought maybe it was the image since I had uploaded it after updating to 1.3.4, but I used an older image and it had the same issue. As of this evening I’ve only been able to test this out with our UEFI image. I’ll try to see if we have the same issue with our BIOS image in the morning.


  • Senior Developer

    A lot of work has been done on this and so far I think we’re on the right path.

    The latest init’s are up and ready for more testing, just needing @Doctrg to run the test.

    He’s already tested simply deploying back and it’s giving him some issues with the first partition (saying the partition is smaller than the source). I think this is because of the “math” that was being done originally up until I corrected around it last night.

    With any luck, however, a fresh upload will enable this image to be deployed to smaller/larger systems.


  • Senior Developer

    Please re-run the installer and try again?

    I know it’s a lot of back and forth, but it should work (hopefully).

    At least the original and new sizes appear to be expanding on the same originating disk, I have not yet had the chance to test orig < smaller drive and orig > larger drive yet. Just same size disk for now. All appears to be working atleast in that regard.



  • Using sysprep and an answerfile you can add:

    <ExtendOSPartition>
            <Extend>true</Extend>
    </ExtendOSPartition>
    

    To the specialize phase.

    See ExtendOSPartition.



  • @Tom-Elliott Not a problem. If you guys were here I’d buy y’all a round.


  • Senior Developer

    @Doctrg I know I’m sounding rude here.

    I’m not worried, right now, that it’s not booting, I believe I know why it’s not and am trying to figure that part out, but it can wait for the moment. (it should work, and I don’t know why it’s not, but for now I don’t care about it – I know it is important trust me).

    I’m trying to work out what to do to fix the expanding in GPT layouts though, as this was severely broken and has been broken for a very long time.



  • @Tom-Elliott On the 256GB drive it did not resize. It came up showing as a 40GB drive when running fdisk -l /dev/sda. The drive size showed correct on the 500GB drive due to it being the size it was created on originally. Neither could boot into Windows though.


  • Senior Developer

    Did it not resize at all or did it not resize as you are “expecting” it to?



  • @Tom-Elliott Sorry, the GPT image did not resize correctly.


  • Senior Developer

    @Doctrg I’m not sure i understand here.

    You’re saying it DID or DIDN’T resize the partition? I understand the “blow-up” part just want to understand the rest.



  • @Tom-Elliott I’m glad it is not just me… I don’t have much hair on top of my head to pull out. I do thank you for your guidence on the bios image as recapturing it fixed that issue. I re-ran the FOG installer and my main server and storage nodes are now on 1.3.5 RC-10. I tried deploying the GPT image and it still did not expand the disk size. It did however it did not resize and it blew-up Windows. It is acting as if the Start-End of the partitions is not correct.


  • Senior Developer

    Please re-run the FOG installer.

    It should download a new set of init’s that should re-enable expanding hdd’s. I will admit, however, the expanding element is not perfect right now (it’s barely even pretty) but it should re-enable your GPT disks to re-expand appropriately.

    The issue, from what I can see, has existed since March 9, 2016 (just in case anybody needed to know a date).


  • Senior Developer

    Things I do know:

    We need to capture the image based on the starting point of the partition, regardless of if the partition has been shrunk. This is because the “start” points is where the data begins for that partition.

    On deploy, we should NOT need to be so cautious to the start point whether the partition is a fixed size or not because we get to place the data whereever it may be needed.


  • Senior Developer

    Good news, I know what’s wrong. I still have a hard to believing that it “worked” once before, though that’s just my crazy thoughts.

    So I built my own GPT based system and captured the image of it.

    This worked beautifully, What hasn’t worked as well is the resizing of GPT.

    So here’s what’s happening, from what I can see (during deploy process):

    1. sfdisk is applied.
    2. maths are done to attempt filling the disk.
    3. Because the partition start isn’t shifted to the right, the freshly built sfdisk isn’t allowed to be applied.

    Why does this occur particularly in GPT, it seems? This I haven’t quite figured. I can tell you it’s because of the strangeness of GPT partitions as they’re typically defined. Rather than shifting the next partitions start partition (whether they’re of fixed size or not) to a point that it might be able to able use, it’s trying to use the original start partition as defined in the sfdisk file, and not shifting start point of the other partitions.

    I don’t know if this is trying to use the “full disk” for one partition rather than trying to evenly distribute it though. It feels like this might be what’s happening, though again I’m not 100%.

    The reason this may be happening, I suspect, is because GPT lay’s out the disk much different than a typical MBR layout would be.

    Typical MBR Layout will, typically, have the “larger” of the partitions at the end of the table.

    For example:
    Partition 1 will be a smaller boot partition.
    Partition 2 will be the “data” partition (the one we want to resize).

    I don’t know, and suspect this could cause issues, how this affects multiple-boot systems currently. It might have a layout like:
    partition1 smaller windows boot partition/recovery?
    partition 2 windows data (wants to be resized)
    partition 3 linux swap or data
    partition 4 linux swap or data.

    Ultimately where the partitions are at shouldn’t matter, I just don’t know of a good way to approach it. This is particularly true in the case of “fixed” partitions, as we try not to adjust the start partitions at all for fixed partitions.


  • Senior Developer

    @Doctrg For the “bios” images, please recapture the images. This can also be fixed if you add the partition number to the d1.fixed_size_partitions file so it would contain 1:2:3 (most likely). (Changing the partition numbers to be proper for your image.)



  • @Tom-Elliott I have the files uploaded to my Google Drive. I did find out that I was given some bad information the other day. The resize is also affecting our “BIOS” image. The reason we thought it was working was due to the HDD size being the same as the original drive. I was able to image a 500GB HDD, but when I tried an 80GB or 160GB I got the error “Target partition size (525MB) is smaller than source (670MB). Use option -C to disable size checking.”

    I also booted into debug on the computer from the other day that I imaged which is showing only a 11GB drive. I ran ntfsresize -v /dev/sda4 and the output is as follows:

    0_1487625095458_20170220_150955.png


  • Senior Developer

    @Doctrg You could enable the “majorDebug” flag, but I don’t know what it’s going to, or not, display.



  • @Wayne-Workman I’m still working on getting the files uploaded. Who do I need to PM to send the link to the shared folder? Also, what program is being used for the resizing? My co-worker that has been the FOG administrator since the beginning is wondering if we can try to do a debug deploy and do a verbose on the resize.



  • @Wayne-Workman It would be helpful if I can send the image to one of you guys. I do not have control over our network so making DHCP changes would be out of my hands and we are in the middle of moving our offices so I do not have an offline environment setup yet to do testing. I’m downloading the image for the storage node now and will upload it to my Google drive once it is done.


  • Moderator

    @Doctrg It looks like you came from 1.3.0 previously. The 1.3.2 installer would have made a directory with 1.3.2 in the name had the 1.3.2 installer been ran - this is fine.

    What I’m thinking, you probably won’t agree with. What I would want to do is ask you to setup a 1.3.0 server and manually transfer this broke image to it and see if it works on that setup or not. This would involve DHCP changes or an offline environment which can be challenging to setup if you’ve not done it before. It’s a lot of work. OR… you could share this known-good image with us (not via forums but via private messages and Mega or Dropbox or Google Drive) and let us test the image with the various fog versions and figure out what’s going on. Many of the active Developers and Moderators have test environments at home (like myself) that are setup to test whatever fog related stuff we want without any real work involved.

    The reason why your thread has gotten the attention it has is because of how you are responding (professionally, with logic, soundly). I think that you know that this image really was a good image prior - so I really want to look into this to see if there’s anything at all wrong with fog. We’re trying to determine when it worked, and when it broke. For that, we need someone to do the testing. Either you with your own environment just for this thing, or you can share the image with someone here for testing.


Log in to reply
 

Looks like your connection to FOG Project was lost, please wait while we try to reconnect.