Fog Unable to upload multiple disk

[dougthermal]

I am trying to deploy a fog image with multiple hard drives. We have uploaded with the multiple disk option and it has created files for each disk, however when it tries to download it is only applying the first disk. I have went into debug mode and the same happens during the fog.download process. It appears to image the first disk successfully, however never continues to the second disk.

What can we do to progress this on or find out additional information about what it is doing?

[Tom Elliott]

What version of fog?

[dougthermal]

1.2.0

I Just realized my subject was misleading. It is actually the download process I’m having trouble with. The upload appears to have been successful.

[Sebastian Roth]

What files do you see in the image directory? Please run (on the server):
[CODE]ls -al /images/*[/CODE]

FOG comes with a small utility to find out about the disks your have. You can run this tool in a debug session to see the disks being detected before actually starting the imaging process Run a debug download session for this host. When you get to the prompt type:
[CODE]fogpartinfo --list-devices[/CODE]

[dougthermal]

I have reuploaded the image, and receive the same result. I have tried manually deleting all partitions on the original disks before imaging, however it still has this issue.

Commands requested below:

Server:
[EMAIL=‘root@fog-image:/images/lenovothinkcentrev2’][U][COLOR=#0066cc]root@fog-image:/images/lenovothinkcentrev2[/COLOR][/U][/EMAIL]# ls -alh
total 46G
drwxrwxrwx 2 root root 4.0K Jan 22 15:14 .
drwxrwxrwx 17 fog fog 4.0K Jan 26 11:39 …
-rwxrwxrwx 1 root root 512 Jan 22 14:15 d1.mbr
-rwxrwxrwx 1 root root 25G Jan 22 14:32 d1p1.img
-rwxrwxrwx 1 root root 512 Jan 22 14:32 d2.mbr
-rwxrwxrwx 1 root root 8.2M Jan 22 14:32 d2p1.img
-rwxrwxrwx 1 root root 21G Jan 22 15:13 d2p2.img
-rwxrwxrwx 1 root root 126 Jan 22 15:13 d2p3.img
-rwxrwxrwx 1 root root 189M Jan 22 15:14 d2p5.img

Client (where image was previously attempted):

[root@10 / ] # fogpartinfo --list-devices
Error: Invalid partition table on /dev/sdb – wrong signature 0.
/dev/sda
[root@10 / ] # fogpartinfo --list-parts /dev/sda
/dev/sda1
[root@10 / ] # fogpartinfo --list-parts /dev/sdb
Error: Invalid partition table on /dev/sdb – wrong signature 0.

[Sebastian Roth]

Oooops! Not the error you’d want to see on your master client.

I’ve dug through a lot of partition tables and related source code lately and we might be able to fix this. Can you please provide some more information: Output of ‘sfdisk -d /dev/sdb’, operating systems installed on the disks, OSID for this image in FOG. And it would be great if you could upload the files ‘d1.mbr’, ‘d2.mbr’ and ‘d2p3.img’ (which should all be quite small files) here in the forum. Maybe you have to tar/zip the files first.

[QUOTE]I have tried manually deleting all partitions on the original disks before imaging, however it still has this issue.[/QUOTE]
FOG actually deletes all partitions before applying the partition table and imaging the partitions… No need to do that yourself.

I hope you are not in a hurry to get this going. I am pretty sure we can find a solution but it’ll take a few days I suppose…

[dougthermal]

I’ve attached the requested files.

Command output below:

[root@10 /]#: sfdisk -d /dev/sdb
sfdisk: Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
sfdisk: ERROR: sector 232814043 does not have an msdos signature

partition table of /dev/sdb

unit: sectors

/dev/sdb1 : start =2048, size=204800, Id= 7, bootable
/dev/sdb2 : start = 206848, size= 232593408, Id= 7
/dev/sdb3 : start =323814043, size= 1622501, Id= f
/dev/sdb4 : start = 0, size= 0, Id= 0
[root@10 /]#

[url=“/_imported_xf_attachments/1/1646_fog-image.zip?:”]fog-image.zip[/url]

[dougthermal]

Fog Server OS Info:

root@fog-image:/images/LenovoThinkCentrev3# uname -a
Linux fog-image 3.13.0-32-generic #57~precise1-Ubuntu SMP Tue Jul 15 03:50:54 UTC 2014 i686 i686 i386 GNU/Linux

root@fog-image:/images/LenovoThinkCentrev3# cat /etc/issue
Ubuntu 12.04.5 LTS \n \l

root@fog-image:/images/LenovoThinkCentrev3# cat /etc/debian_version
wheezy/sid

[Tom Elliott]

Upgrade to Trunk?

This was/is a known issue and i’m pretty sure it’s been fixed.

[dougthermal]

Running release 2961 now.

Do I need to re-upload the image or should a new download work? Tried redownloading and have had the same result.

[dougthermal]

Reuploaded image and downloaded with same result in this version.

[Sebastian Roth]

[QUOTE]This was/is a known issue and i’m pretty sure it’s been fixed.[/QUOTE]
Unfortunately this seams to be kind of a different issue to the one I had on my system…
But Tom is right, there were some major changes with extended partitions lately so you really needed upgrade to current SVN and upload a complete new image!

After upgrading to the newest SVN version your files in the image directory should look a little different. Let me guess:
[CODE]ls -alh /images/lenovothinkcentrev2
total 46G
…
-rwxrwxrwx 1 root root 512 Feb 2 14:32 d2.mbr
-rwxrwxrwx 1 root root 0 Feb 2 14:32 d2.original.swapuuids
-rwxrwxrwx 1 root root 310 Feb 2 14:32 d2.partitions
-rwxrwxrwx 1 root root 8.2M Feb 2 14:32 d2p1.img
-rwxrwxrwx 1 root root 21G Feb 2 15:13 d2p2.img
-rwxrwxrwx 1 root root 189M Feb 2 15:14 d2p5.img[/CODE]

Especially ‘d2.partitions’ should exist now! My second guess is that your ‘d2.partitions’ looks like this:
[CODE]cat /images/lenovothinkcentrev2/d2.partitions

partition table of /dev/sdb

unit: sectors

/dev/sdb1 : start= 2048, size= 204800, Id= 7, bootable
/dev/sdb2 : start= 206848, size=232593408, Id= 7
/dev/sdb3 : start=232814043, size= 1622501, Id= f
/dev/sdb4 : start= 0, size= 0, Id= 0[/CODE]
Could you please confirm (or post the results if different!) this!? And may I ask you to upload another sample. Please run this commands (be very carefull to not get the parameters wrong!!) on your upload client machine. Simply start an upload debug session, run the command and shutdown the machine without imaging after this.
[CODE]fog
…

Hit Enter till you see ‘Mounting File System . . .’ then Ctrl+C

…
dd if=/dev/sdb of=/images/sdb3.img bs=512 count=2 skip=232814043[/CODE]
[B]Again: Really make sure that you get all the parameters and numbers right here!!![/B]
You’ll find the file ‘sdb3.img’ on your FOG Server in ‘/images/dev/sdb3.img’.

[Sebastian Roth]

I must’ve been blind or deaf (maybe both)… sorry! The extended partition related change we made only fixed things for ‘mps’ (multi partition single disk) but I totally forgot about ‘mpa’. Should be fixed in the newest SVN by now.

As a quick fix you can use the following partition table and put it into ‘/images/…/d2.partitions’:
[CODE]# partition table of /dev/sdb
unit: sectors

/dev/sdb1 : start= 2048, size= 204800, Id= 7, bootable
/dev/sdb2 : start= 206848, size=232593408, Id= 7
/dev/sdb3 : start=232814043, size= 1622501, Id= f
/dev/sdb4 : start= 0, size= 0, Id= 0
/dev/sdb5 : start=232814106, size= 1622438, Id= 7[/CODE]

Just put that in place and try deploying the image to your computers. No new upload or SVN_Upgrade needed for testing this! Would be great to hear if this is working for you.

But I am still kind of concerned about the error you posted when running fogpartinfo and sfdisk. Hopefully I can shed a light on this too.

[dougthermal]

I’ll just download newest svn and reupload. I don’t mind testing the feature fully to make sure it works.

Is there any developer documentation beyond the code comments? I was considering adding some features for my own use or public use (if they prove useful for others) and made my way through the download script, but any documentation that is available would speed up the effort considerably.

[dougthermal]

Receive message “Missing Operating System”

still receive: Error: Invalid partition table on /dev/sdb – wrong signature 0.

sfdisk /dev/sdb

sfdisk: Checking that no-one is using this disk right now …
sfdisk: OK

Disk /dev/sdb: 14593 cylinders, 255 heads, 63 sectors/track
sfdisk: Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
sfdisk: ERROR: sector 232814043 does not have an msdos signature
Old situation:
Units: cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End #cyls #blocks Id System
/dev/sdb1 * 0+ 12- 13- 102400 7 HPFS/NTFS/exFAT
/dev/sdb2 12+ 14491- 14479- 116296704 7 HPFS/NTFS/exFAT
/dev/sdb3 14492+ 14592- 101- 811250+ f W95 Ext’d (LBA)
/dev/sdb4 0 - 0 0 0 Empty
sfdisk: Input in the following format; absent fields get a default value.
<start> <size> <type [E,S,L,X,hex]> <bootable [-,*]> <c,h,s> <c,h,s>
Usually you only need to specify <start> and <size> (and perhaps <type>).

Do you still want me to dd sdb3 to a file skipping this block?

I have uploaded this pc as a single disk multiple partition clone (unplugged larger disk) and it works without issue. Not sure if that helps isolate the issue.

[Tom Elliott]

Based on the “funnyness” I’m seeing, is the /dev/sdb disk supposed to be GPT or MBR?

[Sebastian Roth]

[quote=“dougthermal, post: 41718, member: 28335”]…

Do you still want me to dd sdb3 to a file skipping this block?

I have uploaded this pc as a single disk multiple partition clone (unplugged larger disk) and it works without issue. Not sure if that helps isolate the issue.[/quote]

Yes please upload sdb3.img!

Very interesting. Do I get that right?? If you unplug sda (sdb will be sda then) and run FOG with ‘mps’ there is no error??

[Sebastian Roth]

[quote=“Tom Elliott, post: 41719, member: 7271”]Based on the “funnyness” I’m seeing, is the /dev/sdb disk supposed to be GPT or MBR?[/quote]
From what I could see in the uploaded files it is definitely a normal MBR not GPT! Is that right?

[dougthermal]

Should be MBR, I get this on gdisk:

[I]# gdisk -l /dev/sdb[/I]
[I]GPT fdisk (gdisk) version 0.8.6[/I]

[I]Partition table scan:[/I]
[I] MBR: MBR only[/I]
[I] BSD: not present[/I]
[I] APM: not present[/I]
[I] GPT: not present[/I]

[I][/I]
[I]Found invalid GPT and valid MBR; converting MBR to GPT format.[/I]
[I][/I]

[I]Disk /dev/sdb: 234441648 sectors, 111.8 GiB[/I]
[I]Logical sector size: 512 bytes[/I]
[I]Disk identifier (GUID): FF0A8C36-404A-4876-8E98-DEF355411B86[/I]
[I]Partition table holds up to 128 entries[/I]
[I]First usable sector is 34, last usable sector is 234441614[/I]
[I]Partitions will be aligned on 2-sector boundaries[/I]
[I]Total free space is 20935 sectors (10.2 MiB)[/I]

[I]Number Start (sector) End (sector) Size Code Name[/I]
[I] 1 2048 206847 100.0 MiB 0700 Microsoft basic data[/I]
[I] 2 206848 232800255 110.9 GiB 0700 Microsoft basic data[/I]
[I] 5 232814106 234436543 792.2 MiB 0700 Microsoft basic data[/I]

Ran DD. Attached result. Command output below:
[I]# dd if=/dev/sdb of=/images/sdb3.img bs=512 count=2 skip=232814043[/I]
[I]2+0 records in[/I]
[I]2+0 records out[/I]
[I]1024 bytes (1.0 kB) copied, 0.0288977 s, 35.4 kB/s[/I]

[url=“/_imported_xf_attachments/1/1649_sdb3.zip?:”]sdb3.zip[/url]

[dougthermal]

[quote=“Uncle Frank, post: 41729, member: 28116”]Yes please upload sdb3.img!

Very interesting. Do I get that right?? If you unplug sda (sdb will be sda then) and run FOG with ‘mps’ there is no error??[/quote]

Yes that is correct. We were able to run it with mps, and it uploaded and deployed without error. System booted normally.