Image Replication Deletes and starts over

[Raymond Bell]

Images Replicate then delete and start over again.
I check the size when they are done replicating and the images is same size on all storage nodes
Server and Storage nodes 1.5.9

[05-25-21 1:01:28 pm] | Replication already running with PID: 5047
[05-25-21 1:01:28 pm] | Replication already running with PID: 4131
[05-25-21 1:01:28 pm] | Image Name: Windows 8 32Bit 9-5-19
[05-25-21 1:01:28 pm] * Found Image to transfer to 2 nodes
[05-25-21 1:01:28 pm] | Started sync for Image Win 10 64Bit 4-15-21 - Resource id #29962
[05-25-21 1:01:28 pm] | CMD: lftp -e 'set xfer:log 1; set xfer:log-file /opt/fog/log/fogreplicator.Win 10 64Bit 4-15-21.transfer.fog-ThinkCentre-M58p.log;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30; mirror -c --parallel=20 -R --ignore-time -vvv --exclude ".srvprivate" "/images/Win1064Bit4-15-21" "/images/Win1064Bit4-15-21"; exit' -u fogproject,[Protected] 10.24.28.17
[05-25-21 1:01:27 pm] # Win 10 64Bit 4-15-21: Deleting remote file d1p2.img
[05-25-21 1:01:27 pm] # Win 10 64Bit 4-15-21: File size mismatch - d1p2.img: -1469040948 != 50070566604
[05-25-21 1:01:27 pm] # Win 10 64Bit 4-15-21: No need to sync d1p1.img (fog-ThinkCentre-M58p)
[05-25-21 1:01:26 pm] # Win 10 64Bit 4-15-21: No need to sync d1.partitions (fog-ThinkCentre-M58p)
[05-25-21 1:01:26 pm] # Win 10 64Bit 4-15-21: No need to sync d1.original.swapuuids (fog-ThinkCentre-M58p)
[05-25-21 1:01:26 pm] # Win 10 64Bit 4-15-21: No need to sync d1.original.fstypes (fog-ThinkCentre-M58p)
[05-25-21 1:01:26 pm] # Win 10 64Bit 4-15-21: No need to sync d1.minimum.partitions (fog-ThinkCentre-M58p)
[05-25-21 1:01:26 pm] # Win 10 64Bit 4-15-21: No need to sync d1.mbr (fog-ThinkCentre-M58p)
[05-25-21 1:01:25 pm] # Win 10 64Bit 4-15-21: No need to sync d1.fixed_size_partitions (fog-ThinkCentre-M58p)
[05-25-21 1:01:25 pm] | Started sync for Image Win 10 64Bit 4-15-21 - Resource id #29897
[05-25-21 1:01:25 pm] | CMD: lftp -e 'set xfer:log 1; set xfer:log-file /opt/fog/log/fogreplicator.Win 10 64Bit 4-15-21.transfer.Fog-Main-Storage.log;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30; mirror -c --parallel=20 -R --ignore-time -vvv --exclude ".srvprivate" "/images/Win1064Bit4-15-21" "/images/Win1064Bit4-15-21"; exit' -u fogproject,[Protected] 10.24.32.232
[05-25-21 1:01:24 pm] # Win 10 64Bit 4-15-21: Deleting remote file d1p2.img
[05-25-21 1:01:24 pm] # Win 10 64Bit 4-15-21: File size mismatch - d1p2.img: -1469040948 != 50070566604
[05-25-21 1:01:24 pm] # Win 10 64Bit 4-15-21: No need to sync d1p1.img (Fog-Main-Storage)
[05-25-21 1:01:10 pm] # Win 10 64Bit 4-15-21: No need to sync d1.partitions (Fog-Main-Storage)
[05-25-21 1:01:10 pm] # Win 10 64Bit 4-15-21: No need to sync d1.original.swapuuids (Fog-Main-Storage)
[05-25-21 1:01:10 pm] # Win 10 64Bit 4-15-21: No need to sync d1.original.fstypes (Fog-Main-Storage)
[05-25-21 1:01:10 pm] # Win 10 64Bit 4-15-21: No need to sync d1.minimum.partitions (Fog-Main-Storage)
[05-25-21 1:01:10 pm] # Win 10 64Bit 4-15-21: No need to sync d1.mbr (Fog-Main-Storage)
[05-25-21 1:01:08 pm] # Win 10 64Bit 4-15-21: No need to sync d1.fixed_size_partitions (Fog-Main-Storage)
[05-25-21 1:01:08 pm] | Image Name: Win 10 64Bit 4-15-21
[05-25-21 1:01:08 pm] * Found Image to transfer to 2 nodes
[05-25-21 1:01:08 pm] | Replication already running with PID: 5014
[05-25-21 1:01:08 pm] | Replication already running with PID: 5009
[05-25-21 1:01:08 pm] | Image Name: Title1 6-29-16
[05-25-21 1:01:08 pm] * Found Image to transfer to 2 nodes
[05-25-21 1:01:08 pm] | Replication already running with PID: 18321
[05-25-21 1:01:08 pm] | Replication already running with PID: 18304
[05-25-21 1:01:08 pm] | Image Name: Lenovo-E15-UEFI-Lap-5-24-21
[05-25-21 1:01:08 pm] * Found Image to transfer to 2 nodes
[05-25-21 1:01:07 pm] | Started sync for Image Lenovo-E15-Lap-64Bit - Resource id #29807
[05-25-21 1:01:07 pm] | CMD: lftp -e 'set xfer:log 1; set xfer:log-file /opt/fog/log/fogreplicator.Lenovo-E15-Lap-64Bit.transfer.fog-ThinkCentre-M58p.log;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30; mirror -c --parallel=20 -R --ignore-time -vvv --exclude ".srvprivate" "/images/Lenovo-E15-Lap-64Bit" "/images/Lenovo-E15-Lap-64Bit"; exit' -u fogproject,[Protected] 10.24.28.17
[05-25-21 1:01:07 pm] # Lenovo-E15-Lap-64Bit: No need to sync d1p3.img (fog-ThinkCentre-M58p)
[05-25-21 1:01:03 pm] # Lenovo-E15-Lap-64Bit: Deleting remote file d1p2.img
[05-25-21 1:01:03 pm] # Lenovo-E15-Lap-64Bit: File size mismatch - d1p2.img: -2109178658 != 49430428894
[05-25-21 1:01:03 pm] # Lenovo-E15-Lap-64Bit: No need to sync d1p1.img (fog-ThinkCentre-M58p)
[05-25-21 1:00:50 pm] # Lenovo-E15-Lap-64Bit: No need to sync d1.partitions (fog-ThinkCentre-M58p)
[05-25-21 1:00:49 pm] # Lenovo-E15-Lap-64Bit: No need to sync d1.mbr (fog-ThinkCentre-M58p)
[05-25-21 1:00:48 pm] | Started sync for Image Lenovo-E15-Lap-64Bit - Resource id #29763
[05-25-21 1:00:47 pm] | CMD: lftp -e 'set xfer:log 1; set xfer:log-file /opt/fog/log/fogreplicator.Lenovo-E15-Lap-64Bit.transfer.Fog-Main-Storage.log;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30; mirror -c --parallel=20 -R --ignore-time -vvv --exclude ".srvprivate" "/images/Lenovo-E15-Lap-64Bit" "/images/Lenovo-E15-Lap-64Bit"; exit' -u fogproject,[Protected] 10.24.32.232
[05-25-21 1:00:47 pm] # Lenovo-E15-Lap-64Bit: No need to sync d1p3.img (Fog-Main-Storage)
[05-25-21 1:00:35 pm] # Lenovo-E15-Lap-64Bit: Deleting remote file d1p2.img
[05-25-21 1:00:35 pm] # Lenovo-E15-Lap-64Bit: File size mismatch - d1p2.img: -2109178658 != 49430428894
[05-25-21 1:00:34 pm] # Lenovo-E15-Lap-64Bit: No need to sync d1p1.img (Fog-Main-Storage)
[05-25-21 1:00:21 pm] # Lenovo-E15-Lap-64Bit: No need to sync d1.partitions (Fog-Main-Storage)
[05-25-21 1:00:21 pm] # Lenovo-E15-Lap-64Bit: No need to sync d1.mbr (Fog-Main-Storage)
[05-25-21 1:00:20 pm] | Image Name: Lenovo-E15-Lap-64Bit
[05-25-21 1:00:20 pm] * Found Image to transfer to 2 nodes
[05-25-21 1:00:20 pm] | Replication already running with PID: 16166
[05-25-21 1:00:20 pm] | Replication already running with PID: 16164
[05-25-21 1:00:20 pm] | Image Name: HS-173-64Bit
[05-25-21 1:00:20 pm] * Found Image to transfer to 2 nodes
[05-25-21 1:00:20 pm] | Replication already running with PID: 22682
[05-25-21 1:00:20 pm] | Replication already running with PID: 22665
[05-25-21 1:00:20 pm] | Image Name: HS-125-M700-64Bit-Win10
[05-25-21 1:00:20 pm] * Found Image to transfer to 2 nodes
[05-25-21 1:00:20 pm] * Attempting to perform Group -> Nodes image replication.
[05-25-21 1:00:20 pm] | There are no other members to sync to.
[05-25-21 1:00:20 pm] | Image Name: Windows 8 32Bit 9-5-19
[05-25-21 1:00:20 pm] * Not syncing Image between groups
[05-25-21 1:00:20 pm] | There are no other members to sync to.
[05-25-21 1:00:20 pm] | Image Name: Win 10 64Bit 4-15-21
[05-25-21 1:00:20 pm] * Not syncing Image between groups
[05-25-21 1:00:20 pm] | This is not the primary group.
[05-25-21 1:00:20 pm] | Not syncing Image: Title1 6-29-16
[05-25-21 1:00:20 pm] | There are no other members to sync to.
[05-25-21 1:00:20 pm] | Image Name: Lenovo-E15-UEFI-Lap-5-24-21
[05-25-21 1:00:20 pm] * Not syncing Image between groups
[05-25-21 1:00:20 pm] | There are no other members to sync to.
[05-25-21 1:00:20 pm] | Image Name: Lenovo-E15-Lap-64Bit
[05-25-21 1:00:20 pm] * Not syncing Image between groups
[05-25-21 1:00:20 pm] | There are no other members to sync to.
[05-25-21 1:00:20 pm] | Image Name: HS-173-64Bit
[05-25-21 1:00:20 pm] * Not syncing Image between groups
[05-25-21 1:00:20 pm] | There are no other members to sync to.
[05-25-21 1:00:20 pm] | Image Name: HS-125-M700-64Bit-Win10
[05-25-21 1:00:20 pm] * Not syncing Image between groups
[05-25-21 1:00:20 pm] * All files synced for this item.
[05-25-21 1:00:20 pm] # dev/postinitscripts: No need to sync fog.postinit (fog-ThinkCentre-M58p)
[05-25-21 1:00:19 pm] * All files synced for this item.
[05-25-21 1:00:19 pm] # dev/postinitscripts: No need to sync fog.postinit (Fog-Main-Storage)
[05-25-21 1:00:19 pm] | File Name: dev/postinitscripts
[05-25-21 1:00:19 pm] * Found Image to transfer to 2 nodes
[05-25-21 1:00:19 pm] | Replicating postinitscripts
[05-25-21 1:00:19 pm] * All files synced for this item.
[05-25-21 1:00:19 pm] # postdownloadscripts: No need to sync fog.postdownload (fog-ThinkCentre-M58p)
[05-25-21 1:00:18 pm] * All files synced for this item.
[05-25-21 1:00:18 pm] # postdownloadscripts: No need to sync fog.postdownload (Fog-Main-Storage)
[05-25-21 1:00:12 pm] | File Name: postdownloadscripts
[05-25-21 1:00:12 pm] * Found Image to transfer to 2 nodes
[05-25-21 1:00:12 pm] | Replicating postdownloadscripts
[05-25-21 1:00:12 pm] * Attempting to perform Group -> Group image replication.
[05-25-21 1:00:12 pm] * We are node ID: 18. We are node name: fog-GX620-K2
[05-25-21 1:00:12 pm] * We are group ID: 1. We are group name: gsdfog
[05-25-21 1:00:12 pm] * Starting Image Replication.
[05-25-21 12:58:28 pm] | Sync finished - Resource id #20567
[05-25-21 12:58:27 pm] | Sync finished - Resource id #20632
[05-25-21 12:51:30 pm] | Sync finished - Resource id #20187
[05-25-21 12:51:29 pm] | Sync finished - Resource id #20231
[05-25-21 12:50:28 pm] | Replication already running with PID: 5047
[05-25-21 12:50:28 pm] | Replication already running with PID: 4131
[05-25-21 12:50:28 pm] | Image Name: Windows 8 32Bit 9-5-19
[05-25-21 12:50:28 pm] * Found Image to transfer to 2 nodes
[05-25-21 12:50:28 pm] | Replication already running with PID: 31525
[05-25-21 12:50:28 pm] | Replication already running with PID: 31522

[Raymond Bell]

@sebastian-roth
Yes one of the storage nodes is the master
But i am working on changing that as we speak and creating a new master 64bit then the others will be changed to 64 bit also

as of right now i have
server 64 bit 16.04 xubuntu not storage node
1 Ubuntu 16.04 32bit storage node master
1 Ubuntu 16.04 32bit storage node
1 ubuntu 20.04 64 bit storage node

I am going to just use the 64bit as master after i get the images all copied over, then disable the other 2 32 bit nodes, upgrade them to 64 bit and re-enable them to see if that fixes some of the issues i am having since the 1.5.9 upgrade

[Sebastian Roth]

@Raymond-Bell Found the older post in the forums: https://forums.fogproject.org/topic/14100/file-size-mismatch-fogreplicator

Possibly we never tried to fix this in the code as 32 bit OS installs are not very common anymore…

Is your system 32 bit?

[Raymond Bell]

@sebastian-roth Thanks

[Sebastian Roth]

@raymond-bell said in Image Replication Deletes and starts over:

File size mismatch - d1p2.img: -2109178658 != 49430428894

Here we got why… One of your servers seems to be using a 32 bit PHP install causing an Integer overflow and therefore the size mismatch. I am fairly sure this is fixed in the dev-branch version. Let me check the repo to see if I can find that quickly.