• Register
    • Login
    • Search
    • Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search

    Solved Can't image from storage node

    FOG Problems
    3
    9
    1066
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • N
      nrg last edited by nrg

      master node/server: fog 1.5.0-RC-10
      os: ubuntu 14.04 LTS

      storage node: fog 1.5.0-RC-10
      os: ubuntu 14.04 LTS

      both server hardware and software are identical.

      master server images perfectly fine and have been using it all year long. Just recently added the storage node and it replicated all the images fine according to the logs. Then when trying to image two computer, the 2nd computer one fail with an error. It connects to the storage node, pulls the image and does the calculating bitmap and right when it gets to d1p3.img file it says “read Error: no such file or directory”

      I check the folder and the file is there. I believe it’s a file integrity issue.

      pigz: skipping: <stdin>: corrupt -- invalid deflate data (invalid code lengths set)  pigz: abort: internal threads error```
      

      I get that error in the partsclone window on computer 2.
      then a warning message shows
      read ERROR:No such file or directory
      args Passed: /images/LABGEN/d1p3.img* /dev/sda3

      then I get the ntfs volume failed error but that’s because it didn’t image sda3 correctly.

      i reboot computer two and set it to image by itself so it can talk to the master node and it images perfectly on the same selected image.

      I saw this thread: https://forums.fogproject.org/topic/6318/image-file-integrity/6 but cannot understand its solution. maybe its a tool to replicate and do a checksum after?

      thanks.

      1 Reply Last reply Reply Quote 0
      • N
        nrg last edited by

        Re: Can’t image from storage node
        Not sure what happened but it seems to be working now. Well imaging from the storage node works when I tried it last week in another group of computers from a separate image that’s stored on the server. I saw the storage node IP on the client computer when imaging.
        I have not touched or updated neither the main or node servers.
        I have not tried to image from LABGEN from storage node. so 50% working from the main issue. Will follow up or capture a new iamge for LABGEN image soon.

        thank you everyone!

        1 Reply Last reply Reply Quote 0
        • S
          Sebastian Roth Moderator last edited by

          @nrg Well then it might be the storage node having an issue - disk or RAM!?!

          Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

          Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

          1 Reply Last reply Reply Quote 1
          • N
            nrg @Sebastian Roth last edited by nrg

            @sebastian-roth said in Can’t image from storage node:

            memtest

            did a badblock test on all three sda partition on the storage node. no errors.

            administrator@31-FOGNODE1:~$ sudo badblocks -v /dev/sda1
            Checking blocks 0 to 524287
            Checking for bad blocks (read-only test): done                                                       
            Pass completed, 0 bad blocks found. (0/0/0 errors)
            administrator@31-FOGNODE1:~$ sudo badblocks -v /dev/sda2
            Checking blocks 0 to 483710975
            Checking for bad blocks (read-only test): done
            Pass completed, 0 bad blocks found. (0/0/0 errors)
            administrator@31-FOGNODE1:~$ sudo badblocks -v /dev/sda3
            Checking blocks 0 to 4149247
            Checking for bad blocks (read-only test): done
            Pass completed, 0 bad blocks found. (0/0/0 errors)
            

            btw, i have my fog storage node running Ubuntu EFI installed. could that be an issue?
            also, i couldn’t find memtest in the grub menu. i also tried putting it back and installing it without success. guess im SOL.

            don’t think there’s anything wrong with the client I tried to image from the storage node. Reason, I was able to image from the master node just fine.

            everyone, thank you for all your efforts 😃

            edit: I ran memtest86 and there is no errors. pass perfectly fine. so no hard drive issue or memory issue… dont know whats wrong =\

            1 Reply Last reply Reply Quote 0
            • S
              Sebastian Roth Moderator last edited by

              @nrg So if it’s clearly not the image files (checksums identical, log ok) I can only think of the client machine being the issue.

              Looking at the pictures again I find the message “No such file or directory” to have caused some confusion. It kind of sounds as if the file d1p3.img is missing. But this is not the case. What FOG does is pipe d1pX.img files through a temporary FIFO device that will extract the content on the fly so partclone can use the data straight away. When this temporary FIFO extraction fails partclone would crash with “No such file or directory” as the FIFO isn’t there anymore.

              So the question is, why does the temporary FIFO process pigz abort with “corrupted – invalid deflate data (invalid code lengths set)”? I’d run badblock test and memtest on both the FOG storage node and the client.

              Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

              Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

              N 1 Reply Last reply Reply Quote 0
              • N
                nrg @Sebastian Roth last edited by

                @sebastian-roth

                master node:

                822a5ed907cb41c30631dc6c160f243f  /images/LABGEN/d1.fixed_size_partitions
                dc9ea3b81f67be37d7d63b297dec1941  /images/LABGEN/d1.mbr
                69d95b8f7a25b7cb62095f3cd358e55c  /images/LABGEN/d1.minimum.partitions
                b293627989626a35e2d6631747b45faf  /images/LABGEN/d1.original.fstypes
                d41d8cd98f00b204e9800998ecf8427e  /images/LABGEN/d1.original.swapuuids
                72c1e188a1f4c3594d7712c41657d227  /images/LABGEN/d1.original.uuids
                2c72b745cb040e72a8f0de0cabc18120  /images/LABGEN/d1p1.img
                9fa1331ab13e0b4f0798289b26752109  /images/LABGEN/d1p2.img
                2021c4a6186077ef7e52320ff7718ef5  /images/LABGEN/d1p3.img
                1fd435efeeb55203f8277a883ff1c17c  /images/LABGEN/d1.partitions
                

                Storage Node:

                822a5ed907cb41c30631dc6c160f243f  /images/LABGEN/d1.fixed_size_partitions
                dc9ea3b81f67be37d7d63b297dec1941  /images/LABGEN/d1.mbr
                69d95b8f7a25b7cb62095f3cd358e55c  /images/LABGEN/d1.minimum.partitions
                b293627989626a35e2d6631747b45faf  /images/LABGEN/d1.original.fstypes
                d41d8cd98f00b204e9800998ecf8427e  /images/LABGEN/d1.original.swapuuids
                72c1e188a1f4c3594d7712c41657d227  /images/LABGEN/d1.original.uuids
                2c72b745cb040e72a8f0de0cabc18120  /images/LABGEN/d1p1.img
                9fa1331ab13e0b4f0798289b26752109  /images/LABGEN/d1p2.img
                2021c4a6186077ef7e52320ff7718ef5  /images/LABGEN/d1p3.img
                1fd435efeeb55203f8277a883ff1c17c  /images/LABGEN/d1.partitions
                

                The logs look fine.
                [12-01-17 8:10:21 pm] | LABGEN: No need to sync d1p3.img file to 31-FOGNODE1
                [12-01-17 8:10:21 pm] | LABGEN: No need to sync d1p3.img file to 31-FOGNODE1
                [12-01-17 8:10:21 pm] | LABGEN: No need to sync d1p2.img file to 31-FOGNODE1
                [12-01-17 8:10:21 pm] | LABGEN: No need to sync d1p2.img file to 31-FOGNODE1
                [12-01-17 8:10:20 pm] | LABGEN: No need to sync d1p1.img file to 31-FOGNODE1
                [12-01-17 8:10:20 pm] | LABGEN: No need to sync d1p1.img file to 31-FOGNODE1
                [12-01-17 8:10:20 pm] | LABGEN: No need to sync d1.partitions file to 31-FOGNODE1
                [12-01-17 8:10:20 pm] | LABGEN: No need to sync d1.partitions file to 31-FOGNODE1
                [12-01-17 8:10:19 pm] | LABGEN: No need to sync d1.original.uuids file to 31-FOGNODE1
                [12-01-17 8:10:19 pm] | LABGEN: No need to sync d1.original.uuids file to 31-FOGNODE1
                [12-01-17 8:10:19 pm] | LABGEN: No need to sync d1.original.swapuuids file to 31-FOGNODE1
                [12-01-17 8:10:19 pm] | LABGEN: No need to sync d1.original.swapuuids file to 31-FOGNODE1
                [12-01-17 8:10:19 pm] | LABGEN: No need to sync d1.original.fstypes file to 31-FOGNODE1
                [12-01-17 8:10:19 pm] | LABGEN: No need to sync d1.original.fstypes file to 31-FOGNODE1
                [12-01-17 8:10:19 pm] | LABGEN: No need to sync d1.minimum.partitions file to 31-FOGNODE1
                [12-01-17 8:10:18 pm] | LABGEN: No need to sync d1.minimum.partitions file to 31-FOGNODE1
                [12-01-17 8:10:18 pm] | LABGEN: No need to sync d1.mbr file to 31-FOGNODE1
                [12-01-17 8:10:18 pm] | LABGEN: No need to sync d1.mbr file to 31-FOGNODE1
                [12-01-17 8:10:18 pm] | LABGEN: No need to sync d1.fixed_size_partitions file to 31-FOGNODE1
                [12-01-17 8:10:18 pm] | LABGEN: No need to sync d1.fixed_size_partitions file to 31-FOGNODE1
                [12-01-17 8:10:17 pm] | Image Name: LABGEN

                for the LABGEN image, it was first captured Jun 15th 2016 on the master node and it works fine.
                the LABGEN image was replicated on the storage node on Nov 9th 2017. The reason why you see Dec 1st on the two files on the storage node is because I manually deleted d1p3.img on the storage node and had the masternode replicate the file again. I knew that it runs the replicator service every 30mins? so if the server did not see the file, it would replicate itself again. that explains the date.

                NOTE: WTF? why does the logs show duplicated lines? bug?

                1 Reply Last reply Reply Quote 0
                • S
                  Sebastian Roth Moderator last edited by Sebastian Roth

                  @nrg Just do make sure… What do you get when running md5sum /images/LABGEN/* on both system (this will take a while to calculate, couple of minutes…).

                  As well please check the replication log in the web UI and post here. I am wondering about the timestamps of the files on the storage node. Most are Nov 9th but d1p3.img and d1.fixed_size_partitions are Dez 1st. Maybe those are being replicated over and over again and you seem to run into that when trying to image.

                  Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                  Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                  N 1 Reply Last reply Reply Quote 0
                  • N
                    nrg last edited by nrg

                    This is from the storage node 10.31.1.16:

                    total 18148960
                    drwxrwxrwx  2 fog fog         4096 Dec  1 09:07 .
                    drwxrwxrwx 15 fog root        4096 Nov  9 15:14 ..
                    -rwxrwxrwx  1 fog fog            4 Dec  1 09:07 d1.fixed_size_partitions
                    -rwxrwxrwx  1 fog fog      1048576 Nov  9 15:14 d1.mbr
                    -rwxrwxrwx  1 fog fog          629 Nov  9 15:14 d1.minimum.partitions
                    -rwxrwxrwx  1 fog fog           15 Nov  9 15:14 d1.original.fstypes
                    -rwxrwxrwx  1 fog fog            0 Nov  9 15:14 d1.original.swapuuids
                    -rwxrwxrwx  1 fog fog          215 Nov  9 15:14 d1.original.uuids
                    -rwxrwxrwx  1 fog fog     11614954 Nov  9 15:14 d1p1.img
                    -rwxrwxrwx  1 fog fog      2171231 Nov  9 15:14 d1p2.img
                    -rwxrwxrwx  1 fog fog  18569660631 Dec  1 09:10 d1p3.img
                    -rwxrwxrwx  1 fog fog          629 Nov  9 15:14 d1.partitions
                    

                    this is from the master node 10.31.1.15:

                    total 18148960
                    drwxrwxrwx  2 fog root        4096 Jun 15  2016 .
                    drwxrwxrwx 15 fog root        4096 Aug 17 12:41 ..
                    -rwxrwxrwx  1 fog root           4 Jun 15  2016 d1.fixed_size_partitions
                    -rwxrwxrwx  1 fog root     1048576 Jun 15  2016 d1.mbr
                    -rwxrwxrwx  1 fog root         629 Jun 15  2016 d1.minimum.partitions
                    -rwxrwxrwx  1 fog root          15 Jun 15  2016 d1.original.fstypes
                    -rwxrwxrwx  1 fog root           0 Jun 15  2016 d1.original.swapuuids
                    -rwxrwxrwx  1 fog root         215 Jun 15  2016 d1.original.uuids
                    -rwxrwxrwx  1 fog root    11614954 Jun 15  2016 d1p1.img
                    -rwxrwxrwx  1 fog root     2171231 Jun 15  2016 d1p2.img
                    -rwxrwxrwx  1 fog root 18569660631 Jun 15  2016 d1p3.img
                    -rwxrwxrwx  1 fog root         629 Jun 15  2016 d1.partitions
                    
                    

                    You can see the date change of d1p3 on the storage node because I deleted the file and had fog replicate it again today. replicating works, mysql and ftp works.

                    I believe fog is designed to jump to the storage node on the 2nd computer when imaging.
                    reading through the wiki, it says that computer 1 = MasterNode, computer 2 = Storage node.
                    I can see the IP of the storage node on the 2nd computer in the partsclone screen.
                    0_1512152787122_error_1.png
                    0_1512153744637_error_2.png

                    kernal parameter
                    0_1512153651684_error_3.jpg

                    1 Reply Last reply Reply Quote 0
                    • george1421
                      george1421 Moderator last edited by

                      On the storage node can you run the following command from the linux console.

                      ls -la /images/LABGEN and post the results here?

                      One additional question…
                      How are you directing the clients to use the storage node vs the master node? When imaging fails it should print out the kernel parameters, confirm that you only see the ip address of the storage node in there for storageip or just use a mobile phone and grab a screen shot of the kernel parameters and post the image here and we can take a look at the results.

                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                      1 Reply Last reply Reply Quote 0
                      • 1 / 1
                      • First post
                        Last post

                      105
                      Online

                      10.4k
                      Users

                      16.4k
                      Topics

                      150.5k
                      Posts

                      Copyright © 2012-2023 FOG Project