sda3 and sda4 of resizable windows 7 image not restoring

[JJ Fullmer]

I was trying to use FOG to make an image of a computer so I could then clone that image to an ssd for said computer.
This process typically works for me without issue. But most computers have a image put there by fog anyway. This one I believe was the default hp windows 7. But it now has proprietary software on it, so I want to not mess with a new OS type of thing. I just wanted to upgrade the HDD to a SSD.

So I uploaded the image from the HDD and it uploads 4 partitions.
Then I download the image to the ssd and partclone opens for /dev/sda1 and /dev/sda2 but then it finishes.
I ran a debug download and saw that after the download a fdisk -l does show 4 partitions on the ssd.

fdisk -l /dev/sda
Disk /dev/sda: 111.8 GiB, 120034123776 bytes, 234441648 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x4322fd0a

Device     Boot     Start       End  Sectors  Size Id Type
/dev/sda1  *         2048    206847   204800  100M  7 HPFS/NTFS/exFAT
/dev/sda2          206848  98137915 97931068 46.7G  7 HPFS/NTFS/exFAT
/dev/sda3       955262976 976558079 21295104 10.2G  7 HPFS/NTFS/exFAT
/dev/sda4       976558080 976762879   204800  100M  c W95 FAT32 (LBA)

But Windows 7 will not boot and if I run fixparts /dev/sda it says partitions 3 and 4 are oversized.

fixparts /dev/sda
FixParts 1.0.0

Loading MBR data from /dev/sda
Warning: Deleting oversized partition #3! Start = 955262976, length = 21295104
Warning: Deleting oversized partition #4! Start = 976558080, length = 204800

MBR command (? for help):

I tried writing those changes, but it still doesn’t boot. When I boot into a windows 7 install disk and look at diskpart via command prompt it says that there is just partition 1 and it is a RAW format.

Here is the ls and cat of the image files on the server in case it helps…

ls of images folder

ls -l /images/UPSpvc/
total 52529608
-rwxrwxrwx 1 fog root           4 Jun 29 11:54 d1.fixed_size_partitions
-rwxrwxrwx 1 fog root     1048576 Jun 29 11:54 d1.mbr
-rwxrwxrwx 1 fog root         306 Jun 29 11:54 d1.minimum.partitions
-rwxrwxrwx 1 fog root          30 Jun 29 11:54 d1.original.fstypes
-rwxrwxrwx 1 fog root           0 Jun 29 11:54 d1.original.swapuuids
-rwxrwxrwx 1 fog root    28012455 Jun 29 11:55 d1p1.img
-rwxrwxrwx 1 fog root 44053554839 Jun 29 12:06 d1p2.img
-rwxrwxrwx 1 fog root  9684646204 Jun 29 12:09 d1p3.img
-rwxrwxrwx 1 fog root    23019696 Jun 29 12:09 d1p4.img
-rwxrwxrwx 1 fog root         306 Jun 29 11:54 d1.partitions

d1.partitions contents

cat d1.partitions
label: dos
label-id: 0x4322fd0a
device: /dev/sda
unit: sectors

/dev/sda1 : start=        2048, size=      204800, type=7, bootable
/dev/sda2 : start=      206848, size=   955056128, type=7
/dev/sda3 : start=   955262976, size=    21295104, type=7
/dev/sda4 : start=   976558080, size=      204800, type=c

I need to get this computer back into production asap and I’d prefer to not have to put the old slow hdd back in to accomplish that. Is there anything else I’m not thinking of that I can do other than remake the image as a raw sector by sector image?

Thanks,
-JJ

[Wayne Workman]

Pinging our partitioning hot shots @Tom-Elliott @Sebastian-Roth @george1421

Also, what’s the output of: fdisk -l /images/UPSpvc/d1.mbr ?

And the output of this: cat /images/UPSpvc/d1.fixed_size_partitions

[JJ Fullmer]

@Wayne-Workman

cat /images/UPSpvc/d1.fixed_size_partitions
4:1

I am opening a new debug session to try the fdisk -l on that…

Also, I realized I can’t actually do a raw image. The HDD is 500 GB and SSD is 120 GB so I need the resizable to work

[Wayne Workman]

@JJ-Fullmer said in sda3 and sda4 of resizable windows 7 image not restoring:

I am opening a new debug session to try the fdisk -l on that…

It’ll read it from the mbr file on the actual fog server itself.

[JJ Fullmer]

@Wayne-Workman Oh duh…

 fdisk -l /images/UPSpvc/d1.mbr
You must set cylinders.
You can do this from the extra functions menu.

Disk /images/UPSpvc/d1.mbr: 1 MB, 1048576 bytes, 2048 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x4322fd0a

                Device Boot      Start         End      Blocks   Id  System
/images/UPSpvc/d1.mbr1   *        2048      206847      102400    7  HPFS/NTFS/exFAT
/images/UPSpvc/d1.mbr2          206848    98137915    48965534    7  HPFS/NTFS/exFAT
/images/UPSpvc/d1.mbr3       955262976   976558079    10647552    7  HPFS/NTFS/exFAT
/images/UPSpvc/d1.mbr4       976558080   976762879      102400    c  W95 FAT32 (LBA)

[Wayne Workman]

@JJ-Fullmer and cat /images/UPSpvc/d1.minimum.partitions please.

[JJ Fullmer]

@Wayne-Workman

cat /images/UPSpvc/d1.minimum.partitions
label: dos
label-id: 0x4322fd0a
device: /dev/sda
unit: sectors

/dev/sda1 : start=        2048, size=      204800, type=7, bootable
/dev/sda2 : start=      206848, size=    97931068, type=7
/dev/sda3 : start=   955262976, size=    21295104, type=7
/dev/sda4 : start=   976558080, size=      204800, type=c

[JJ Fullmer]

So I put the original HDD back in the machine and started a debug upload.
I ran fdisk -l /dev/sda and I am seeing some discrepancies

fdisk -l /dev/sda
Disk /dev/sda: 465.8 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x4322fd0a

Device     Boot     Start       End   Sectors   Size Id Type
/dev/sda1  *         2048    206847    204800   100M  7 HPFS/NTFS/exFAT
/dev/sda2          206848 955262975 955056128 455.4G  7 HPFS/NTFS/exFAT
/dev/sda3       955262976 976558079  21295104  10.2G  7 HPFS/NTFS/exFAT
/dev/sda4       976558080 976762879    204800   100M  c W95 FAT32 (LBA)

No wait, the size is just bigger on sda2 because the harddrive was bigger.
I guess it’s not properly placing the 3rd and 4th partitions because their sector starts and ends are out of the range of a disk with 120 GB vs 500 GB.
I guess I just need to edit the partition table to line up with the smaller size properly?
That Or I could shrink the hard drive and move the partitions manually on the HDD and then reupload the image.

[Wayne Workman]

@JJ-Fullmer Well if the drive is a true 120GB SSD, then you have about 128849018880 sectors to work with. Here’s how I figured that:

the way the start and size columns work for the d1.minimum.partitions file works is, the start is where the partition starts, and start+size is the last sector of the partition. The next partition should use the next sector. You have to fit all your partitions and their data into 128849018880 sectors.

You can take time to get the d1.minimum.partitions right, or, more easier as you said, resize the partitions as small as possible on the original disk and upload that, then manually expand them out on the new disk.

[JJ Fullmer]

@Wayne-Workman So in theory if I edit d1.minimum.partitions to reflect where the partitions should each start, it will work as it is?

[Wayne Workman]

@JJ-Fullmer Should. But keep in mind how much space is in use on those partitions, you have to allow for the space used or it won’t work.

At any rate, make a copy of the d1.minimum.partitions file as a backup.
cp /images/UPSpvc/d1.minimum.partitions /images/UPSpvc/d1.minimum.partitions.old

[JJ Fullmer]

@Wayne-Workman http://thestarman.pcministry.com/asm/mbr/DriveOffsets.htm
You may want to check your sector math…
Anyway I took a look at what those extra partitions were and saw that they were the HP recovery tools partitions that are useless to me.
So I tried the fixparts delete the partition solution again and then did a bootrec /rebuildbcd and it works now without the stupid partitions.

I’m pretty sure though that editing the minimum partitions so that the maximum sectors minus the size of that last partitions was the start and so on would work.

Thanks,
-JJ

[Wayne Workman]

@JJ-Fullmer Well, if there isn’t 1,024 MB in a GB, and there isn’t 1,024 KB in a MB, and there isn’t 1,024 B in a KB, and there isn’t 512 B in a sector, then I’m lost…

The formula for going directly from GB to B is Bytes = Gigabytes x 1,073,741,824 I found on a site.

[JJ Fullmer]

@Wayne-Workman
120 * 1073741824 = 128849018880
128849018880 / 512 = 251,658,240
Wait that is what you have you just kept typing the 12 digit number. And I was seeing a higher number of sectors for the 120 GB then the last sector of the 500 GB drive (976,762,879) And I was like, that can’t be right. But I looked again and you just hit copy on the wrong number I think.

[Wayne Workman]

@JJ-Fullmer Yup. I copied the wrong number lol.

[Sebastian Roth]

@JJ-Fullmer Well this is an interesting one as we obviously never had anyone wanting to resize this kind of partition layout yet.

As we can see from your posts the old disk is a lot larger than the new one:

Disk /dev/sda: 465.8 GiB, 500107862016 bytes, 976773168 sectors

vs.

Disk /dev/sda: 111.8 GiB, 120034123776 bytes, 234441648 sectors

While this does not have to be a problem per se it is causing an issue in your case because sda3 and sda4 start at a position way beyond the SSD capacity. If my understanding of the magic resize algorithm is correct we don’t move any start sectors of any of the partitions as this might cause an issue. We keep all the start sectors and only shrink/expand the partitions. This is usually fine because most installations have the big system partition last and therefore this is being shrunk/expanded.

A couple of times Tom and I have discussed rewriting that magic resize algorithm and improving it. Your case would definitely be on the list for improvements. But on the other hand we are a bit worried about making things worse because we would need to do assumptions on which partition’s start sector can be moved and which can’t (windows boot partitions are very picky with being moved!). A lot of things can go wrong if we need to do those kind of guess work.

[JJ Fullmer]

@Sebastian-Roth After I realized the issue I thought of another instance where something like this happened that would be another case that would benefit from the redoing the algorithm. I wanted to convert a mbr image/windows install to an efi style without redoing the whole thing and I actually found a tool that can do that. The problem is my original image is only 1 partition and efi requires having a seperate efi partition. I discovered I was able to shrink my partition and just add a small efi partition to the end and I was able to successfully convert the image. However when I tried to deploy the image I found that the efi partition would always start at the same spot regardless of the size of the hard drive. So I stopped trying to do that. Granted with it being the boot partition it could get picky with being moved and make the whole idea moot. And granted if I had a standard 2 partition windows mbr installation, I could have deleted the first partition to create the efi partition and then fog would deploy the image like any other. So it’s a special case scenario within a special case scenario.

But perhaps the algorithm would only change when there are more than 2 partitions. Like if partitions is greater than 2 do a bunch of math, maybe either require specifying which partition is the one you want to be resized/is the main data partition, or check volume names for mount points or some sort of universal indication that a partition is the main partition. Or just assume the largest of the original partitions wants to resize and do the math based on the current algorithm to find the max size, but just call the sector count used to do the math of it’s max size the sector size of the new hard drive minus the sum of sectors of sda3 and up. I don’t know though, maybe that’s helpful, maybe it’s just ranting. Overall it’s only useful in very special cases and probably not worth the effort. Unless something about partition formatting changes in the near future or something.

[Wayne Workman]

Maybe checkmark boxes in the image definition to tell the inits to move the start sectors? Could be I guess 5 checkboxes, for 5 partitions. Some may have more partitions than that but I think 5 would cover most.