Fog Unable to upload multiple disk



  • I am trying to deploy a fog image with multiple hard drives. We have uploaded with the multiple disk option and it has created files for each disk, however when it tries to download it is only applying the first disk. I have went into debug mode and the same happens during the fog.download process. It appears to image the first disk successfully, however never continues to the second disk.

    What can we do to progress this on or find out additional information about what it is doing?


  • Developer

    Thanks for the update. That’s really weird and I still can’t get my head around what’s going on there. As I said, I’ve tried to reproduce the issue but I just can’t do it. Very special. Would be great if you keep us posted on new findings.



  • I haven’t had a chance to run the walkparts or other commands, but changing the order of the disks seems to have resolved the issue completely. We plugged the primary boot disk into the first slot (to be sda) and secondary disk in second slot (sdb) and it uploads and deploys without issue.


  • Developer

    Too bad that I just can’t reproduce the issue on my test system (QEMU) although I use two (virtual) disks and exactly your partition tables…
    As you experience no problems uploading and downloading when there is only one disk installed I started wondering about it being a problem in util-linux itself. But I can’t find any pointer to that on the net. Any ideas on how we can narrow this down?

    Although it might not be very wise to do for the operating systems you have on your disks, would you mind trying to swap the disks (sda <-> sdb) just for one upload/download test. Only if you got some spare time… Please try walkparts, testdisk and the fake d2.partitions first!


  • Developer

    I’ve looked through all your dumps, samples and posts several times but it still does not ring a bell for me. From what I can see in those files it’s all just perfectly normal and should not report any errors (hopefully I haven’t missed anything!). Two more things you might try out. First is this script I found on the net: [url]https://raw.githubusercontent.com/arvidjaar/scriptlets/master/walkparts.sh[/url]
    Download it to the FOG server and put it in ‘/images/dev’. The only thing I had to change to make it work within FOG is change ‘/bin/sh’ to ‘/bin/bash’ in the first line of the script! Then start a debug upload([S]download)[/S] session and run:
    [CODE]fog

    Hit Enter till you see ‘Mounting File System . . .’ then Ctrl+C


    /images/walkparts.sh /dev/sdb
    …[/CODE]

    What do you get from that script?? An while you have that session running you could give ‘testdisk’ a go too. Would be interesting to see if ‘testdisk’ is complaining about the missing msdos signature too?!?!!

    [B]EDIT: It was quite late when I wrote this and I got carried away! Do the first two tests on your master-client in a debug-upload-session but please do not continue with the third one on that same client!! Start one or more of your other client(s) in a normal/debug multicast/download session for the following test…[/B]

    One more thing you can try out while you’ve got that debug download session running. As kind of a workaround you could try faking the partition table. You probably have a complete uploaded image (d1p1.img, d2p1.img, d2p2.img d2p5.img) on your FOG server ready. Possibly the only problem keeping you from deploying those image files proberly is the incorrect ‘d2.partitions’ file. Edit ‘/images/lenovothinkcentrev2/d2.partitions’:
    [CODE]# partition table of /dev/sdb
    unit: sectors

    /dev/sdb1 : start= 2048, size= 204800, Id= 7, bootable
    /dev/sdb2 : start= 206848, size=232593408, Id= 7
    /dev/sdb3 : start=232814043, size= 1622501, Id= f
    /dev/sdb4 : start= 0, size= 0, Id= 0
    /dev/sdb5 : start=232814106, size= 1622438, Id= 7[/CODE]
    Then try downloading the image to your client. Does that work??



  • [quote=“Uncle Frank, post: 41729, member: 28116”]Yes please upload sdb3.img!

    Very interesting. Do I get that right?? If you unplug sda (sdb will be sda then) and run FOG with ‘mps’ there is no error??[/quote]

    Yes that is correct. We were able to run it with mps, and it uploaded and deployed without error. System booted normally.



  • Should be MBR, I get this on gdisk:

    [I]# gdisk -l /dev/sdb[/I]
    [I]GPT fdisk (gdisk) version 0.8.6[/I]

    [I]Partition table scan:[/I]
    [I] MBR: MBR only[/I]
    [I] BSD: not present[/I]
    [I] APM: not present[/I]
    [I] GPT: not present[/I]

    [I][/I]
    [I]Found invalid GPT and valid MBR; converting MBR to GPT format.[/I]
    [I]
    [/I]

    [I]Disk /dev/sdb: 234441648 sectors, 111.8 GiB[/I]
    [I]Logical sector size: 512 bytes[/I]
    [I]Disk identifier (GUID): FF0A8C36-404A-4876-8E98-DEF355411B86[/I]
    [I]Partition table holds up to 128 entries[/I]
    [I]First usable sector is 34, last usable sector is 234441614[/I]
    [I]Partitions will be aligned on 2-sector boundaries[/I]
    [I]Total free space is 20935 sectors (10.2 MiB)[/I]

    [I]Number Start (sector) End (sector) Size Code Name[/I]
    [I] 1 2048 206847 100.0 MiB 0700 Microsoft basic data[/I]
    [I] 2 206848 232800255 110.9 GiB 0700 Microsoft basic data[/I]
    [I] 5 232814106 234436543 792.2 MiB 0700 Microsoft basic data[/I]

    Ran DD. Attached result. Command output below:
    [I]# dd if=/dev/sdb of=/images/sdb3.img bs=512 count=2 skip=232814043[/I]
    [I]2+0 records in[/I]
    [I]2+0 records out[/I]
    [I]1024 bytes (1.0 kB) copied, 0.0288977 s, 35.4 kB/s[/I]

    [url="/_imported_xf_attachments/1/1649_sdb3.zip?:"]sdb3.zip[/url]


  • Developer

    [quote=“Tom Elliott, post: 41719, member: 7271”]Based on the “funnyness” I’m seeing, is the /dev/sdb disk supposed to be GPT or MBR?[/quote]
    From what I could see in the uploaded files it is definitely a normal MBR not GPT! Is that right?


  • Developer

    [quote=“dougthermal, post: 41718, member: 28335”]…

    Do you still want me to dd sdb3 to a file skipping this block?

    I have uploaded this pc as a single disk multiple partition clone (unplugged larger disk) and it works without issue. Not sure if that helps isolate the issue.[/quote]

    Yes please upload sdb3.img!

    Very interesting. Do I get that right?? If you unplug sda (sdb will be sda then) and run FOG with ‘mps’ there is no error??


  • Senior Developer

    Based on the “funnyness” I’m seeing, is the /dev/sdb disk supposed to be GPT or MBR?



  • Receive message “Missing Operating System”

    still receive: Error: Invalid partition table on /dev/sdb – wrong signature 0.

    sfdisk /dev/sdb

    sfdisk: Checking that no-one is using this disk right now …
    sfdisk: OK

    Disk /dev/sdb: 14593 cylinders, 255 heads, 63 sectors/track
    sfdisk: Warning: extended partition does not start at a cylinder boundary.
    DOS and Linux will interpret the contents differently.
    sfdisk: ERROR: sector 232814043 does not have an msdos signature
    Old situation:
    Units: cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

    Device Boot Start End #cyls #blocks Id System
    /dev/sdb1 * 0+ 12- 13- 102400 7 HPFS/NTFS/exFAT
    /dev/sdb2 12+ 14491- 14479- 116296704 7 HPFS/NTFS/exFAT
    /dev/sdb3 14492+ 14592- 101- 811250+ f W95 Ext’d (LBA)
    /dev/sdb4 0 - 0 0 0 Empty
    sfdisk: Input in the following format; absent fields get a default value.
    <start> <size> <type [E,S,L,X,hex]> <bootable [-,*]> <c,h,s> <c,h,s>
    Usually you only need to specify <start> and <size> (and perhaps <type>).

    Do you still want me to dd sdb3 to a file skipping this block?

    I have uploaded this pc as a single disk multiple partition clone (unplugged larger disk) and it works without issue. Not sure if that helps isolate the issue.



  • I’ll just download newest svn and reupload. I don’t mind testing the feature fully to make sure it works.

    Is there any developer documentation beyond the code comments? I was considering adding some features for my own use or public use (if they prove useful for others) and made my way through the download script, but any documentation that is available would speed up the effort considerably.


  • Developer

    I must’ve been blind or deaf (maybe both)… sorry! The extended partition related change we made only fixed things for ‘mps’ (multi partition single disk) but I totally forgot about ‘mpa’. Should be fixed in the newest SVN by now.

    As a quick fix you can use the following partition table and put it into ‘/images/…/d2.partitions’:
    [CODE]# partition table of /dev/sdb
    unit: sectors

    /dev/sdb1 : start= 2048, size= 204800, Id= 7, bootable
    /dev/sdb2 : start= 206848, size=232593408, Id= 7
    /dev/sdb3 : start=232814043, size= 1622501, Id= f
    /dev/sdb4 : start= 0, size= 0, Id= 0
    /dev/sdb5 : start=232814106, size= 1622438, Id= 7[/CODE]

    Just put that in place and try deploying the image to your computers. No new upload or SVN_Upgrade needed for testing this! Would be great to hear if this is working for you.

    But I am still kind of concerned about the error you posted when running fogpartinfo and sfdisk. Hopefully I can shed a light on this too.


  • Developer

    [QUOTE]This was/is a known issue and i’m pretty sure it’s been fixed.[/QUOTE]
    Unfortunately this seams to be kind of a different issue to the one I had on my system…
    But Tom is right, there were some major changes with extended partitions lately so you really needed upgrade to current SVN and upload a complete new image!

    After upgrading to the newest SVN version your files in the image directory should look a little different. Let me guess:
    [CODE]ls -alh /images/lenovothinkcentrev2
    total 46G

    -rwxrwxrwx 1 root root 512 Feb 2 14:32 d2.mbr
    -rwxrwxrwx 1 root root 0 Feb 2 14:32 d2.original.swapuuids
    -rwxrwxrwx 1 root root 310 Feb 2 14:32 d2.partitions
    -rwxrwxrwx 1 root root 8.2M Feb 2 14:32 d2p1.img
    -rwxrwxrwx 1 root root 21G Feb 2 15:13 d2p2.img
    -rwxrwxrwx 1 root root 189M Feb 2 15:14 d2p5.img[/CODE]

    Especially ‘d2.partitions’ should exist now! My second guess is that your ‘d2.partitions’ looks like this:
    [CODE]cat /images/lenovothinkcentrev2/d2.partitions

    partition table of /dev/sdb

    unit: sectors

    /dev/sdb1 : start= 2048, size= 204800, Id= 7, bootable
    /dev/sdb2 : start= 206848, size=232593408, Id= 7
    /dev/sdb3 : start=232814043, size= 1622501, Id= f
    /dev/sdb4 : start= 0, size= 0, Id= 0[/CODE]
    Could you please confirm (or post the results if different!) this!? And may I ask you to upload another sample. Please run this commands (be very carefull to not get the parameters wrong!!) on your upload client machine. Simply start an upload debug session, run the command and shutdown the machine without imaging after this.
    [CODE]fog

    Hit Enter till you see ‘Mounting File System . . .’ then Ctrl+C


    dd if=/dev/sdb of=/images/sdb3.img bs=512 count=2 skip=232814043[/CODE]
    [B]Again: Really make sure that you get all the parameters and numbers right here!!![/B]
    You’ll find the file ‘sdb3.img’ on your FOG Server in ‘/images/dev/sdb3.img’.



  • Reuploaded image and downloaded with same result in this version.



  • Running release 2961 now.

    Do I need to re-upload the image or should a new download work? Tried redownloading and have had the same result.


  • Senior Developer

    Upgrade to Trunk?

    This was/is a known issue and i’m pretty sure it’s been fixed.



  • Fog Server OS Info:

    root@fog-image:/images/LenovoThinkCentrev3# uname -a
    Linux fog-image 3.13.0-32-generic #57~precise1-Ubuntu SMP Tue Jul 15 03:50:54 UTC 2014 i686 i686 i386 GNU/Linux

    root@fog-image:/images/LenovoThinkCentrev3# cat /etc/issue
    Ubuntu 12.04.5 LTS \n \l

    root@fog-image:/images/LenovoThinkCentrev3# cat /etc/debian_version
    wheezy/sid



  • I’ve attached the requested files.

    Command output below:

    [root@10 /]#: sfdisk -d /dev/sdb
    sfdisk: Warning: extended partition does not start at a cylinder boundary.
    DOS and Linux will interpret the contents differently.
    sfdisk: ERROR: sector 232814043 does not have an msdos signature

    partition table of /dev/sdb

    unit: sectors

    /dev/sdb1 : start =2048, size=204800, Id= 7, bootable
    /dev/sdb2 : start = 206848, size= 232593408, Id= 7
    /dev/sdb3 : start =323814043, size= 1622501, Id= f
    /dev/sdb4 : start = 0, size= 0, Id= 0
    [root@10 /]#

    [url="/_imported_xf_attachments/1/1646_fog-image.zip?:"]fog-image.zip[/url]


  • Developer

    Oooops! Not the error you’d want to see on your master client.

    I’ve dug through a lot of partition tables and related source code lately and we might be able to fix this. Can you please provide some more information: Output of ‘sfdisk -d /dev/sdb’, operating systems installed on the disks, OSID for this image in FOG. And it would be great if you could upload the files ‘d1.mbr’, ‘d2.mbr’ and ‘d2p3.img’ (which should all be quite small files) here in the forum. Maybe you have to tar/zip the files first.

    [QUOTE]I have tried manually deleting all partitions on the original disks before imaging, however it still has this issue.[/QUOTE]
    FOG actually deletes all partitions before applying the partition table and imaging the partitions… No need to do that yourself.

    I hope you are not in a hurry to get this going. I am pretty sure we can find a solution but it’ll take a few days I suppose…


Log in to reply
 

410
Online

39184
Users

10832
Topics

103133
Posts

Looks like your connection to FOG Project was lost, please wait while we try to reconnect.