• Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Register
    • Login

    FOG UEFI image sync with storage node seems to be looping\failing on UUID info.

    Scheduled Pinned Locked Moved Unsolved
    FOG Problems
    4
    10
    1.2k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • B
      BardWood
      last edited by Wayne Workman

      Greetings all,

      I’ve checked the SN and it does have data. I’m not really sure if this is specifically related to EFI images but I’ve had many issues with UUID and the efi images. This seems to be another instance of UUID weirdness. I’ve let this sync run for ~5 days but it never seems to finish. We are talking about roughly 70GB over a 10mb link (VPLS) so it’s possible that it’s still not complete. Anyway, here’s a ton of replication log output:

      [11-27-17 10:53:21 pm] | X1CG2-UEFI-FC-V1: No need to sync d1.partitions file to dub-imgstore-s
[11-27-17 10:53:13 pm] | X1CG2-UEFI-FC-V1: No need to sync d1.original.uuids file to dub-imgstore-s
[11-27-17 10:53:03 pm] * Deleting remote file: /images/X1CG2-UEFI-FC-V1/
[11-27-17 10:53:03 pm] | Files do not match.
[11-27-17 10:53:03 pm] | 0 0 /images/X1CG2-UEFI-FC-V1/d1.original.swapuuids ftp://fog:Ec9m%2FQHVbCJkFwUGMkqCEFvxxpldVFGfZnW49xVuZWU%3D@172.18.40.2/images/X1CG2-UEFI-FC-V1/
[11-27-17 10:52:56 pm] | X1CG2-UEFI-FC-V1: No need to sync d1.original.fstypes file to dub-imgstore-s
[11-27-17 10:52:47 pm] | X1CG2-UEFI-FC-V1: No need to sync d1.minimum.partitions file to dub-imgstore-s
[11-27-17 10:52:38 pm] | X1CG2-UEFI-FC-V1: No need to sync d1.mbr file to dub-imgstore-s
[11-27-17 10:52:25 pm] | X1CG2-UEFI-FC-V1: No need to sync d1.fixed_size_partitions file to dub-imgstore-s
[11-27-17 10:52:14 pm] | Image Name: X1CG2-UEFI-FC-V1
[11-27-17 10:52:14 pm] * Found Image to transfer to 1 node
[11-27-17 10:52:14 pm] * Started sync for Image O7010-WIN10-LEGACY
			lftp -e 'set xfer:log 1; set xfer:log-file "/opt/fog/log/fogreplicator.O7010-WIN10-LEGACY.transfer.dub-imgstore-s.log";set ftp:list-options -a;set net:max-retries 10;set net:timeout 30; mirror -c -r -R --ignore-time -vvv --exclude ".srvprivate" "/images/O7010-WIN10-LEGACY" "/images/O7010-WIN10-LEGACY"; exit' -u fog,[Protected] 172.18.40.2
[11-27-17 10:52:14 pm] | CMD:
[11-27-17 10:52:14 pm] * Starting Sync Actions
[11-27-17 10:52:14 pm] * Deleting remote file: /images/O7010-WIN10-LEGACY/d1p2.img
[11-27-17 10:52:14 pm] | Files do not match.
[11-27-17 10:52:14 pm] | 11381549962 0 /images/O7010-WIN10-LEGACY/d1p2.img ftp://fog:Ec9m%2FQHVbCJkFwUGMkqCEFvxxpldVFGfZnW49xVuZWU%3D@172.18.40.2/images/O7010-WIN10-LEGACY/d1p2.img
[11-27-17 10:52:08 pm] | O7010-WIN10-LEGACY: No need to sync d1p1.img file to dub-imgstore-s
[11-27-17 10:46:20 pm] | O7010-WIN10-LEGACY: No need to sync d1.partitions file to dub-imgstore-s
[11-27-17 10:46:10 pm] * Deleting remote file: /images/O7010-WIN10-LEGACY/
[11-27-17 10:46:10 pm] | Files do not match.
[11-27-17 10:46:10 pm] | 0 0 /images/O7010-WIN10-LEGACY/d1.original.swapuuids ftp://fog:Ec9m%2FQHVbCJkFwUGMkqCEFvxxpldVFGfZnW49xVuZWU%3D@172.18.40.2/images/O7010-WIN10-LEGACY/
[11-27-17 10:46:03 pm] | O7010-WIN10-LEGACY: No need to sync d1.original.fstypes file to dub-imgstore-s
[11-27-17 10:45:54 pm] | O7010-WIN10-LEGACY: No need to sync d1.minimum.partitions file to dub-imgstore-s
[11-27-17 10:45:45 pm] | O7010-WIN10-LEGACY: No need to sync d1.mbr file to dub-imgstore-s
[11-27-17 10:45:32 pm] | O7010-WIN10-LEGACY: No need to sync d1.fixed_size_partitions file to dub-imgstore-s
[11-27-17 10:45:21 pm] | Image Name: O7010-WIN10-LEGACY
[11-27-17 10:45:21 pm] * Found Image to transfer to 1 node
[11-27-17 10:45:21 pm] * Attempting to perform Group -> Nodes image replication.
[11-27-17 10:45:21 pm] * Started sync for Image X1CG5-UEFI-FC-V1
			lftp -e 'set xfer:log 1; set xfer:log-file "/opt/fog/log/fogreplicator.X1CG5-UEFI-FC-V1.transfer.SF-MASTER.log";set ftp:list-options -a;set net:max-retries 10;set net:timeout 30; mirror -c -r -R --ignore-time -vvv --exclude ".srvprivate" "/images/X1CG5-UEFI-FC" "/images/X1CG5-UEFI-FC"; exit' -u fog,[Protected] 172.16.40.33
[11-27-17 10:45:21 pm] | CMD:
[11-27-17 10:45:21 pm] * Starting Sync Actions
[11-27-17 10:45:21 pm] * Deleting remote file: /images/X1CG5-UEFI-FC/
[11-27-17 10:45:21 pm] | Files do not match.
[11-27-17 10:45:21 pm] | 14512943391 0 /images/X1CG5-UEFI-FC/d1p4.img cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:21 pm] * Deleting remote file: /images/X1CG5-UEFI-FC/
[11-27-17 10:45:21 pm] | Files do not match.
[11-27-17 10:45:21 pm] | 6259444 0 /images/X1CG5-UEFI-FC/d1p3.img cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:20 pm] * Deleting remote file: /images/X1CG5-UEFI-FC/
[11-27-17 10:45:20 pm] | Files do not match.
[11-27-17 10:45:20 pm] | 12902901 0 /images/X1CG5-UEFI-FC/d1p2.img cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:20 pm] * Deleting remote file: /images/X1CG5-UEFI-FC/
[11-27-17 10:45:20 pm] | Files do not match.
[11-27-17 10:45:20 pm] | 357253418 0 /images/X1CG5-UEFI-FC/d1p1.img cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:20 pm] * Deleting remote file: /images/X1CG5-UEFI-FC/
[11-27-17 10:45:20 pm] | Files do not match.
[11-27-17 10:45:20 pm] | 886 0 /images/X1CG5-UEFI-FC/d1.partitions cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:20 pm] | X1CG5-UEFI-FC-V1: No need to sync d1.original.swapuuids file to SF-MASTER
[11-27-17 10:45:19 pm] * Deleting remote file: /images/X1CG5-UEFI-FC/
[11-27-17 10:45:19 pm] | Files do not match.
[11-27-17 10:45:19 pm] | 20 0 /images/X1CG5-UEFI-FC/d1.original.fstypes cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:19 pm] * Deleting remote file: /images/X1CG5-UEFI-FC/
[11-27-17 10:45:19 pm] | Files do not match.
[11-27-17 10:45:19 pm] | 886 0 /images/X1CG5-UEFI-FC/d1.minimum.partitions cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:19 pm] * Deleting remote file: /images/X1CG5-UEFI-FC/
[11-27-17 10:45:19 pm] | Files do not match.
[11-27-17 10:45:19 pm] | 1048576 0 /images/X1CG5-UEFI-FC/d1.mbr cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:18 pm] * Deleting remote file: /images/X1CG5-UEFI-FC/
[11-27-17 10:45:18 pm] | Files do not match.
[11-27-17 10:45:18 pm] | 7 0 /images/X1CG5-UEFI-FC/d1.fixed_size_partitions cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:17 pm] | Image Name: X1CG5-UEFI-FC-V1
[11-27-17 10:45:17 pm] * Found Image to transfer to 2 groups
[11-27-17 10:45:17 pm] * Started sync for Image X1CG4-UEFI-FC-V1
			lftp -e 'set xfer:log 1; set xfer:log-file "/opt/fog/log/fogreplicator.X1CG4-UEFI-FC-V1.transfer.SF-MASTER.log";set ftp:list-options -a;set net:max-retries 10;set net:timeout 30; mirror -c -r -R --ignore-time -vvv --exclude ".srvprivate" "/images/X1CG4-UEFI-FC-V1" "/images/X1CG4-UEFI-FC-V1"; exit' -u fog,[Protected] 172.16.40.33
[11-27-17 10:45:17 pm] | CMD:
[11-27-17 10:45:17 pm] * Starting Sync Actions
[11-27-17 10:45:17 pm] * Deleting remote file: /images/X1CG4-UEFI-FC-V1/
[11-27-17 10:45:17 pm] | Files do not match.
[11-27-17 10:45:17 pm] | 14832263898 0 /images/X1CG4-UEFI-FC-V1/d1p4.img cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:17 pm] * Deleting remote file: /images/X1CG4-UEFI-FC-V1/
[11-27-17 10:45:17 pm] | Files do not match.
[11-27-17 10:45:17 pm] | 6259444 0 /images/X1CG4-UEFI-FC-V1/d1p3.img cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:17 pm] * Deleting remote file: /images/X1CG4-UEFI-FC-V1/
[11-27-17 10:45:17 pm] | Files do not match.
[11-27-17 10:45:17 pm] | 12903254 0 /images/X1CG4-UEFI-FC-V1/d1p2.img cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:16 pm] * Deleting remote file: /images/X1CG4-UEFI-FC-V1/
[11-27-17 10:45:16 pm] | Files do not match.
[11-27-17 10:45:16 pm] | 357256473 0 /images/X1CG4-UEFI-FC-V1/d1p1.img cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:16 pm] * Deleting remote file: /images/X1CG4-UEFI-FC-V1/
[11-27-17 10:45:16 pm] | Files do not match.
[11-27-17 10:45:16 pm] | 861 0 /images/X1CG4-UEFI-FC-V1/d1.partitions cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:16 pm] * Deleting remote file: /images/X1CG4-UEFI-FC-V1/
[11-27-17 10:45:16 pm] | Files do not match.
[11-27-17 10:45:16 pm] | 283 0 /images/X1CG4-UEFI-FC-V1/d1.original.uuids cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:16 pm] | X1CG4-UEFI-FC-V1: No need to sync d1.original.swapuuids file to SF-MASTER
[11-27-17 10:45:15 pm] * Deleting remote file: /images/X1CG4-UEFI-FC-V1/
[11-27-17 10:45:15 pm] | Files do not match.
[11-27-17 10:45:15 pm] | 15 0 /images/X1CG4-UEFI-FC-V1/d1.original.fstypes cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:15 pm] * Deleting remote file: /images/X1CG4-UEFI-FC-V1/
[11-27-17 10:45:15 pm] | Files do not match.
[11-27-17 10:45:15 pm] | 861 0 /images/X1CG4-UEFI-FC-V1/d1.minimum.partitions cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:15 pm] * Deleting remote file: /images/X1CG4-UEFI-FC-V1/
[11-27-17 10:45:15 pm] | Files do not match.
[11-27-17 10:45:15 pm] | 1048576 0 /images/X1CG4-UEFI-FC-V1/d1.mbr cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
[11-27-17 10:45:14 pm] * Deleting remote file: /images/X1CG4-UEFI-FC-V1/
[11-27-17 10:45:14 pm] | Files do not match.
[11-27-17 10:45:14 pm] | 7 0 /images/X1CG4-UEFI-FC-V1/d1.fixed_size_partitions cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e

      Mod edited to use codebox

      J Wayne WorkmanW 2 Replies Last reply Reply Quote 0
      • J
        JGallo @BardWood
        last edited by

        @bardwood

        Is it possible that the replication hasn’t finished prior to the next image replication check occurs? That happened to me and I had to increase the IMAGEREPSLEEPTIME for storage nodes on networks with slow connections.

        Wayne WorkmanW 1 Reply Last reply Reply Quote 0
        • Wayne WorkmanW
          Wayne Workman
          last edited by Wayne Workman

          Just wanted to drop a line and say that the image type doesn’t have any bearing on how FOG Image Replication works.

          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
          Daily Clean Installation Results:
          https://fogtesting.fogproject.us/
          FOG Reporting:
          https://fog-external-reporting-results.fogproject.us/

          1 Reply Last reply Reply Quote 0
          • Wayne WorkmanW
            Wayne Workman @BardWood
            last edited by

            @bardwood said in FOG UEFI image sync with storage node seems to be looping\failing on UUID info.:

            70GB over a 10mb link

            And that’s a large image over a tiny link… it could be possible that the sync isn’t finished… but it may also be having problems.

            Basics first:

            • Please check free space on the destination node with the command df -h Look for partitions that are full. Do not assume your huge disks are not full, please go run the command and verify there are no full partitions.
            • Next, is the destination node’s hard disk failing? Maybe run a disk diagnostic tool to doublecheck.
            • Third, is the source node’s hard disk failing? Run a test and find out.

            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
            Daily Clean Installation Results:
            https://fogtesting.fogproject.us/
            FOG Reporting:
            https://fog-external-reporting-results.fogproject.us/

            B 1 Reply Last reply Reply Quote 0
            • Wayne WorkmanW
              Wayne Workman @JGallo
              last edited by Wayne Workman

              @jgallo said in FOG UEFI image sync with storage node seems to be looping\failing on UUID info.:

              Is it possible that the replication hasn’t finished prior to the next image replication check occurs?

              No. I found that bug years ago, I worked with @tom-elliott to resolve it. He changed the replication so that it is aware of lftp processes that it started in the past, and will skip a file that is still in progress from a previous iteration.

              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
              Daily Clean Installation Results:
              https://fogtesting.fogproject.us/
              FOG Reporting:
              https://fog-external-reporting-results.fogproject.us/

              1 Reply Last reply Reply Quote 0
              • B
                BardWood @Wayne Workman
                last edited by Wayne Workman

                @wayne-workman

                No disk full or disk error issues:

                [root@dub-imgstore-s images]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_sfimgstores-LogVol01
                      198G   66G  122G  35% /
tmpfs                 7.8G     0  7.8G   0% /dev/shm
/dev/sda1             190M  164M   17M  91% /boot

                The disk on MASTER/SN are fine but there is more to the story. When I was capturing these images, all the disk info would get captured but it would error on ‘d1.original.swapuuids’. I found an old forum post saying to delete these if this error happened (paraphrasing). I deleted these, the image capture completed, did a few imaging tests using this image, and I thought I was good to go. What I suspect is happening, is that somewhere there’s a file manifest which lists the file(s) I deleted. The file isn’t there, so when it does the remote check it seems there is a mismatch. It tries to sync but can’t sync that file (because it doesn’t exist) so it starts over again. What I suspect I need to do, is somehow edit whatever constitutes a manifest.

                1 Reply Last reply Reply Quote 0
                • S
                  Sebastian Roth Moderator
                  last edited by

                  @BardWood Can you post a link to the thread that mentioned you should delete that d1.original.swapuuids file?

                  Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                  Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                  B 1 Reply Last reply Reply Quote 0
                  • B
                    BardWood @Sebastian Roth
                    last edited by

                    @sebastian-roth Searching… It was something Tom posted a while back (but I’ve seen the post within the last 30 days).

                    B 1 Reply Last reply Reply Quote 0
                    • B
                      BardWood @BardWood
                      last edited by

                      @sebastian-roth I’m not seeing it (but I haven’t given up!). In the meantime, is there a graceful way to stop the sync? Or edit something which tells it to ignore any missing files? The issue is that I know the images are ‘good’ (meaning deployable in the current state) because I’ve deployed them to relevant hardware.

                      1 Reply Last reply Reply Quote 0
                      • S
                        Sebastian Roth Moderator
                        last edited by Sebastian Roth

                        @bardwood said in FOG UEFI image sync with storage node seems to be looping\failing on UUID info.:

                        When I was capturing these images, all the disk info would get captured but it would error on ‘d1.original.swapuuids’.

                        We need to know the exact error - otherwise it’s just guess work going the wrong way.

                        Though I am not an expert on the replication logic yet I think the issue could be that you’ve deleted that file on the master. Please move that d1.original.swapuuids on your storage node out of the way (so you still have a backup copy of it just in case): mv /images/X1CG2-UEFI-FC-V1/d1.original.swapuuids /root

                        Then see if replication stops looping.

                        Replication is a very tricky thing to get right and so messing with it (deleting files by hand) makes it even harder.

                        Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                        Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                        1 Reply Last reply Reply Quote 0
                        • 1 / 1
                        • First post
                          Last post

                        211

                        Online

                        11.6k

                        Users

                        17.1k

                        Topics

                        154.6k

                        Posts
                        Copyright © 2012-2024 FOG Project