Fog 1.6.14 Driver Replication Issue
-
I know that this isn’t intended but we’ve been using it for years. On my Fog production server 1.5 this is not a problem and everything replicates correctly, but on 1.6, my driver folders are continually replicating. I have an “Image” that just enables replicate the folder. So the image name would be “Driver_OptiPlex7900” and the image path would be “Drivers/OptiPlex7900” The Replication Log reports that the File does not exist and starts the replication process every single time. I have well over 20 Driver models and its nice being able to assign them to locations to replicate since not all sites have each model. otherwise its just wasting disk space.
Fog 1.6.14
Ubuntu 26.04.1 -
@FlareImp Thanks for the details. Each “File does not exist” line in the Image Replicator log names the file the replicator could not find on the storage node. Can you post the following:
- One full replication cycle from the Image Replicator log (FOG Configuration > Log Viewer), including the “File does not exist” lines.
- The FOG version on each storage node that receives the driver folders. Do any of them still run 1.5?
- For one file that the log names, the
ls -laoutput for that file on the master node and on the storage node.
-
@FlareImp All Nodes are running same version, 1.6.0.14
File Output: I did use rsync to get the files moved to the Node and disabled the rep service.
Master Fog Server
total 1125304
drwxrwxrwx 2 fogproject administrator 4096 May 22 2024 .
drwxrwxrwx 3 fogproject administrator 4096 Oct 21 2025 …
-rwxrwxrwx 1 fogproject administrator 718000218 Nov 19 2021 ‘7450 AIO-WIN11-A10-YHP57.CAB’
-rwxrwxrwx 1 fogproject administrator 434290695 Nov 30 2022 Video.zipSlave Fog Node Master
total 1125304
drwxrwxrwx 2 fogproject administrator 4096 Nov 10 20:02 .
drwxrwxrwx 3 fogproject administrator 4096 Nov 10 18:31 …
-rwxrwxrwx 1 fogproject administrator 718000218 Nov 10 20:04 ‘7450 AIO-WIN11-A10-YHP57.CAB’
-rwxrwxrwx 1 fogproject administrator 434290695 Nov 30 2022 Video.zip[11-11-26 8:56:50 am] ===== FOG 1.6.0.14 – ImageReplicator starting =====
[11-11-26 8:56:50 am] Interface Ready with IP Address: 10.229.9.58
[11-11-26 8:56:50 am] Interface Ready with IP Address: 127.0.0.1
[11-11-26 8:56:50 am] Interface Ready with IP Address: 127.0.1.1
[11-11-26 8:56:50 am] Interface Ready with IP Address: tv-fog-main
[11-11-26 8:56:50 am] * Starting ImageReplicator Service
[11-11-26 8:56:50 am] * Checking for new items every 3600 seconds
[11-11-26 8:56:50 am] * Starting service loop
[11-11-26 8:56:50 am] * Starting Image Replication.
[11-11-26 8:56:50 am] * We are group ID: 1. We are group name: A(Default)
[11-11-26 8:56:50 am] * We are node ID: 1. We are node name: TV-Fog-Main
[11-11-26 8:56:50 am] * Attempting to perform Group -> Group image replication.
[11-11-26 8:56:50 am] | Replicating postdownloadscripts
[11-11-26 8:56:50 am] * Not syncing Image between nodes
[11-11-26 8:56:50 am] | Image Name:
[11-11-26 8:56:50 am] | There are no members to sync to
[11-11-26 8:56:50 am] | Replicating postinitscripts
[11-11-26 8:56:50 am] * Not syncing Image between nodes
[11-11-26 8:56:50 am] | Image Name:
[11-11-26 8:56:50 am] | There are no members to sync to
[11-11-26 8:56:50 am] * Found Image to transfer to 3 groups
[11-11-26 8:56:50 am] | Image Name: Apps
[11-11-26 8:56:50 am] # Apps: File does not exist Programs(TVAE-Fog)
[11-11-26 8:56:50 am] # Apps: File does not exist Programs/ArcticWolf(TVAE-Fog)
[11-11-26 8:56:51 am] | Apps: No need to sync Programs/ArcticWolf/arctic-wolf-monitoring-475419-1414859e86a9.json file to TVAE-Fog
[11-11-26 8:56:52 am] | Apps: No need to sync Programs/ArcticWolf/arcticwolfagent_2025.03.13-1_amd64.deb file to TVAE-Fog
[11-11-26 8:56:53 am] # Apps: File size mismatch - Programs/ArcticWolf/arcticwolf linux install instructions.docx: 17504 != 0
[11-11-26 8:56:53 am] | Deleting remote file: Programs/ArcticWolf/arcticwolf linux install instructions.docx
[11-11-26 8:56:54 am] | Apps: No need to sync Programs/ArcticWolf/customer.json file to TVAE-Fog
[11-11-26 8:56:54 am] # Apps: File does not exist Programs/ArcticWolf/managed security awarenes (MA)(TVAE-Fog)
[11-11-26 8:56:55 am] | Apps: No need to sync Programs/ArcticWolf/managed security awarenes (MA)/crested-tracer-510019-s0-d5e09422a6e2.json file to TVAE-Fog
[11-11-26 8:56:56 am] # Apps: File size mismatch - Programs/MicrosoftOffice2019.zip: 2107665867 != 1695455288
[11-11-26 8:56:56 am] | Deleting remote file: Programs/MicrosoftOffice2019.zip
[11-11-26 8:56:57 am] # Apps: File size mismatch - Programs/MicrosoftOfficePP2024.zip: 3173410323 != 1789805904
[11-11-26 8:56:57 am] | Deleting remote file: Programs/MicrosoftOfficePP2024.zip
[11-11-26 8:56:58 am] | Apps: No need to sync Programs/Notebook.zip file to TVAE-Fog
[11-11-26 8:56:58 am] # Apps: File does not exist Scripts(TVAE-Fog)
[11-11-26 8:56:59 am] | Apps: No need to sync Scripts/ClearBootManager.ps1 file to TVAE-Fog
[11-11-26 8:57:01 am] | Apps: No need to sync Scripts/fog-agent-0.1.21-x64.msi file to TVAE-Fog
[11-11-26 8:57:02 am] | Apps: No need to sync Scripts/fog-agent-linux-amd64 file to TVAE-Fog
[11-11-26 8:57:03 am] | Apps: No need to sync Scripts/fog-root.pem file to TVAE-Fog
[11-11-26 8:57:04 am] | Apps: No need to sync Scripts/SetupComplete.cmd file to TVAE-Fog
[11-11-26 8:57:05 am] | Apps: No need to sync Scripts/SmartInstaller.exe file to TVAE-Fog
[11-11-26 8:57:06 am] | Apps: No need to sync Scripts/unattendwin10.xml file to TVAE-Fog
[11-11-26 8:57:07 am] | Apps: No need to sync Scripts/unattendwin11_21H2.xml file to TVAE-Fog
[11-11-26 8:57:07 am] # Apps: File does not exist on master node, deleting Programs/chromeosflex/boot.sdi on TVAE-Fog
[11-11-26 8:57:09 am] | Apps: No need to sync Programs/chromeosflex/boot.sdi file to TVAE-Fog
[11-11-26 8:57:09 am] # Apps: File does not exist on master node, deleting Programs/chromeosflex/chromeosflex.wim on TVAE-Fog
[11-11-26 8:57:10 am] | Apps: No need to sync Programs/chromeosflex/chromeosflex.wim file to TVAE-Fog
[11-11-26 8:57:10 am] # Apps: File does not exist on master node, deleting Programs/chromeosflex/dd.exe on TVAE-Fog
[11-11-26 8:57:12 am] | Apps: No need to sync Programs/chromeosflex/dd.exe file to TVAE-Fog
[11-11-26 8:57:12 am] # Apps: File does not exist on master node, deleting Programs/chromeosflex/disk_clean.txt on TVAE-Fog
[11-11-26 8:57:13 am] | Apps: No need to sync Programs/chromeosflex/disk_clean.txt file to TVAE-Fog
[11-11-26 8:57:13 am] # Apps: File does not exist on master node, deleting Programs/chromeosflex/imagex.exe on TVAE-Fog
[11-11-26 8:57:15 am] | Apps: No need to sync Programs/chromeosflex/imagex.exe file to TVAE-Fog
[11-11-26 8:57:15 am] # Apps: File does not exist on master node, deleting Programs/chromeosflex/startnet.cmd on TVAE-Fog
[11-11-26 8:57:16 am] | Apps: No need to sync Programs/chromeosflex/startnet.cmd file to TVAE-Fog
[11-11-26 8:57:16 am] # Apps: File does not exist on master node, deleting Programs/chromeosflex/wow32.dll on TVAE-Fog
[11-11-26 8:57:18 am] | Apps: No need to sync Programs/chromeosflex/wow32.dll file to TVAE-Fog
[11-11-26 8:57:18 am] # Apps: File does not exist on master node, deleting Programs/chromeosflex/wpeinit.log on TVAE-Fog
[11-11-26 8:57:19 am] | Apps: No need to sync Programs/chromeosflex/wpeinit.log file to TVAE-Fog
[11-11-26 8:57:19 am] | CMD: lftp -e ‘set xfer:log 1; set xfer:log-file “/opt/fog/log/fogreplicator.Apps.transfer.TVAE-Fog.log”;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30;set net:limit-total-rate 0:64000000; mirror -c --parallel=20 -R --ignore-time -vvv --exclude “.srvprivate” “/images/Apps” “/images/Apps”;exit’ -u fogproject,[redacted] 10.228.9.50
[11-11-26 8:57:19 am] * Started sync for Image Apps - Resource id #549
[11-11-26 8:57:20 am] # Apps: File does not exist Programs(TVNE-Fog)
[11-11-26 8:57:20 am] # Apps: File does not exist Programs/ArcticWolf(TVNE-Fog)
[11-11-26 8:57:21 am] | Apps: No need to sync Programs/ArcticWolf/arctic-wolf-monitoring-475419-1414859e86a9.json file to TVNE-Fog
[11-11-26 8:57:22 am] | Apps: No need to sync Programs/ArcticWolf/arcticwolfagent_2025.03.13-1_amd64.deb file to TVNE-Fog
[11-11-26 8:57:23 am] # Apps: File size mismatch - Programs/ArcticWolf/arcticwolf linux install instructions.docx: 17504 != 0
[11-11-26 8:57:23 am] | Deleting remote file: Programs/ArcticWolf/arcticwolf linux install instructions.docx
[11-11-26 8:57:24 am] | Apps: No need to sync Programs/ArcticWolf/customer.json file to TVNE-Fog
[11-11-26 8:57:24 am] # Apps: File does not exist Programs/ArcticWolf/managed security awarenes (MA)(TVNE-Fog)
[11-11-26 8:57:25 am] | Apps: No need to sync Programs/ArcticWolf/managed security awarenes (MA)/crested-tracer-510019-s0-d5e09422a6e2.json file to TVNE-Fog
[11-11-26 8:57:25 am] # Apps: File size mismatch - Programs/MicrosoftOffice2019.zip: 2107665867 != 1695455288
[11-11-26 8:57:25 am] | Deleting remote file: Programs/MicrosoftOffice2019.zip
[11-11-26 8:57:26 am] # Apps: File size mismatch - Programs/MicrosoftOfficePP2024.zip: 3173410323 != 1789805904
[11-11-26 8:57:26 am] | Deleting remote file: Programs/MicrosoftOfficePP2024.zip
[11-11-26 8:57:27 am] | Apps: No need to sync Programs/Notebook.zip file to TVNE-Fog
[11-11-26 8:57:27 am] # Apps: File does not exist Scripts(TVNE-Fog)
[11-11-26 8:57:28 am] | Apps: No need to sync Scripts/ClearBootManager.ps1 file to TVNE-Fog
[11-11-26 8:57:29 am] | Apps: No need to sync Scripts/fog-agent-0.1.21-x64.msi file to TVNE-Fog
[11-11-26 8:57:30 am] | Apps: No need to sync Scripts/fog-agent-linux-amd64 file to TVNE-Fog
[11-11-26 8:57:31 am] | Apps: No need to sync Scripts/fog-root.pem file to TVNE-Fog
[11-11-26 8:57:32 am] | Apps: No need to sync Scripts/SetupComplete.cmd file to TVNE-Fog
[11-11-26 8:57:33 am] | Apps: No need to sync Scripts/SmartInstaller.exe file to TVNE-Fog
[11-11-26 8:57:34 am] | Apps: No need to sync Scripts/unattendwin10.xml file to TVNE-Fog
[11-11-26 8:57:35 am] | Apps: No need to sync Scripts/unattendwin11_21H2.xml file to TVNE-Fog
[11-11-26 8:57:35 am] # Apps: File does not exist on master node, deleting Programs/chromeosflex/boot.sdi on TVNE-Fog
[11-11-26 8:57:37 am] | Apps: No need to sync Programs/chromeosflex/boot.sdi file to TVNE-Fog
[11-11-26 8:57:37 am] # Apps: File does not exist on master node, deleting Programs/chromeosflex/chromeosflex.wim on TVNE-Fog
[11-11-26 8:57:38 am] | Apps: No need to sync Programs/chromeosflex/chromeosflex.wim file to TVNE-Fog
[11-11-26 8:57:38 am] # Apps: File does not exist on master node, deleting Programs/chromeosflex/dd.exe on TVNE-Fog
[11-11-26 8:57:39 am] | Apps: No need to sync Programs/chromeosflex/dd.exe file to TVNE-Fog
[11-11-26 8:57:39 am] # Apps: File does not exist on master node, deleting Programs/chromeosflex/disk_clean.txt on TVNE-Fog
[11-11-26 8:57:41 am] | Apps: No need to sync Programs/chromeosflex/disk_clean.txt file to TVNE-Fog
[11-11-26 8:57:41 am] # Apps: File does not exist on master node, deleting Programs/chromeosflex/imagex.exe on TVNE-Fog
[11-11-26 8:57:42 am] | Apps: No need to sync Programs/chromeosflex/imagex.exe file to TVNE-Fog
[11-11-26 8:57:42 am] # Apps: File does not exist on master node, deleting Programs/chromeosflex/startnet.cmd on TVNE-Fog
[11-11-26 8:57:44 am] | Apps: No need to sync Programs/chromeosflex/startnet.cmd file to TVNE-Fog
[11-11-26 8:57:44 am] # Apps: File does not exist on master node, deleting Programs/chromeosflex/wow32.dll on TVNE-Fog
[11-11-26 8:57:45 am] | Apps: No need to sync Programs/chromeosflex/wow32.dll file to TVNE-Fog
[11-11-26 8:57:45 am] # Apps: File does not exist on master node, deleting Programs/chromeosflex/wpeinit.log on TVNE-Fog
[11-11-26 8:57:47 am] | Apps: No need to sync Programs/chromeosflex/wpeinit.log file to TVNE-Fog
[11-11-26 8:57:47 am] | CMD: lftp -e ‘set xfer:log 1; set xfer:log-file “/opt/fog/log/fogreplicator.Apps.transfer.TVNE-Fog.log”;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30;set net:limit-total-rate 0:64000000; mirror -c --parallel=20 -R --ignore-time -vvv --exclude “.srvprivate” “/images/Apps” “/images/Apps”;exit’ -u fogproject,[redacted] 10.225.9.50
[11-11-26 8:57:47 am] * Started sync for Image Apps - Resource id #858
[11-11-26 8:57:47 am] * Found Image to transfer to 2 groups
[11-11-26 8:57:47 am] | Image Name: DellPro16PC16255
[11-11-26 8:57:47 am] # DellPro16PC16255: File does not exist win11(TVNE-Fog)
[11-11-26 8:57:49 am] | DellPro16PC16255: No need to sync win11/DellPro16PC16255.zip file to TVNE-Fog
[11-11-26 8:57:50 am] | DellPro16PC16255: No need to sync WLAN-Tri-Valley.xml file to TVNE-Fog
[11-11-26 8:57:50 am] | CMD: lftp -e ‘set xfer:log 1; set xfer:log-file “/opt/fog/log/fogreplicator.DellPro16PC16255.transfer.TVNE-Fog.log”;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30;set net:limit-total-rate 0:64000000; mirror -c --parallel=20 -R --ignore-time -vvv --exclude “.srvprivate” “/images/Drivers/DellPro16PC16255” “/images/Drivers/DellPro16PC16255”;exit’ -u fogproject,[redacted] 10.225.9.50
[11-11-26 8:57:50 am] * Started sync for Image DellPro16PC16255 - Resource id #904
[11-11-26 8:57:50 am] * Found Image to transfer to 3 groups
[11-11-26 8:57:50 am] | Image Name: Drivers_DellPro24All-in-OneQC24250
[11-11-26 8:57:51 am] # Drivers_DellPro24All-in-OneQC24250: File does not exist win11(TVAE-Fog)
[11-11-26 8:57:51 am] # Drivers_DellPro24All-in-OneQC24250: File size mismatch - win11/DellPro24All-in-OneQC24250.zip: 2334435418 != 1760239600
[11-11-26 8:57:51 am] | Deleting remote file: win11/DellPro24All-in-OneQC24250.zip
[11-11-26 8:57:53 am] | Drivers_DellPro24All-in-OneQC24250: No need to sync WLAN-Tri-Valley.xml file to TVAE-Fog
[11-11-26 8:57:53 am] | CMD: lftp -e ‘set xfer:log 1; set xfer:log-file “/opt/fog/log/fogreplicator.Drivers_DellPro24All-in-OneQC24250.transfer.TVAE-Fog.log”;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30;set net:limit-total-rate 0:64000000; mirror -c --parallel=20 -R --ignore-time -vvv --exclude “.srvprivate” “/images/Drivers/DellPro24All-in-OneQC24250” “/images/Drivers/DellPro24All-in-OneQC24250”;exit’ -u fogproject,[redacted] 10.228.9.50
[11-11-26 8:57:53 am] * Started sync for Image Drivers_DellPro24All-in-OneQC24250 - Resource id #956
[11-11-26 8:57:54 am] # Drivers_DellPro24All-in-OneQC24250: File does not exist win11(TVNE-Fog)
[11-11-26 8:57:54 am] # Drivers_DellPro24All-in-OneQC24250: File size mismatch - win11/DellPro24All-in-OneQC24250.zip: 2334435418 != 1760239600
[11-11-26 8:57:54 am] | Deleting remote file: win11/DellPro24All-in-OneQC24250.zip
[11-11-26 8:57:55 am] | Drivers_DellPro24All-in-OneQC24250: No need to sync WLAN-Tri-Valley.xml file to TVNE-Fog
[11-11-26 8:57:55 am] | CMD: lftp -e ‘set xfer:log 1; set xfer:log-file “/opt/fog/log/fogreplicator.Drivers_DellPro24All-in-OneQC24250.transfer.TVNE-Fog.log”;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30;set net:limit-total-rate 0:64000000; mirror -c --parallel=20 -R --ignore-time -vvv --exclude “.srvprivate” “/images/Drivers/DellPro24All-in-OneQC24250” “/images/Drivers/DellPro24All-in-OneQC24250”;exit’ -u fogproject,[redacted] 10.225.9.50
[11-11-26 8:57:55 am] * Started sync for Image Drivers_DellPro24All-in-OneQC24250 - Resource id #990
[11-11-26 8:57:56 am] * Not syncing Image between groups
[11-11-26 8:57:56 am] | Image Name: Drivers_DellPro7142-in-1P704260
[11-11-26 8:57:56 am] | There are no members to sync to
[11-11-26 8:57:56 am] * Not syncing Image between groups
[11-11-26 8:57:56 am] | Image Name: Drivers_DellProMax16MC16250
[11-11-26 8:57:56 am] | There are no members to sync to
[11-11-26 8:57:56 am] * Not syncing Image between groups
[11-11-26 8:57:56 am] | Image Name: Drivers_DellProRugged14RB14250
[11-11-26 8:57:56 am] | There are no members to sync to
[11-11-26 8:57:56 am] * Found Image to transfer to 3 groups
[11-11-26 8:57:56 am] | Image Name: Drivers_Latitude3500
[11-11-26 8:57:56 am] # Drivers_Latitude3500: File does not exist win11(TVAE-Fog)
[11-11-26 8:57:57 am] # Drivers_Latitude3500: File size mismatch - win11/3500-win11-A00-YH77C.CAB: 1692345974 != 1558966032
[11-11-26 8:57:57 am] | Deleting remote file: win11/3500-win11-A00-YH77C.CAB
[11-11-26 8:57:58 am] | Drivers_Latitude3500: No need to sync WLAN-Tri-Valley.xml file to TVAE-Fog
[11-11-26 8:57:58 am] | CMD: lftp -e ‘set xfer:log 1; set xfer:log-file “/opt/fog/log/fogreplicator.Drivers_Latitude3500.transfer.TVAE-Fog.log”;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30;set net:limit-total-rate 0:64000000; mirror -c --parallel=20 -R --ignore-time -vvv --exclude “.srvprivate” “/images/Drivers/Latitude3500” “/images/Drivers/Latitude3500”;exit’ -u fogproject,[redacted] 10.228.9.50
[11-11-26 8:57:58 am] * Started sync for Image Drivers_Latitude3500 - Resource id #1075
[11-11-26 8:57:59 am] # Drivers_Latitude3500: File does not exist win11(TVNE-Fog)
[11-11-26 8:58:00 am] # Drivers_Latitude3500: File size mismatch - win11/3500-win11-A00-YH77C.CAB: 1692345974 != 1558966032
[11-11-26 8:58:00 am] | Deleting remote file: win11/3500-win11-A00-YH77C.CAB
[11-11-26 8:58:01 am] | Drivers_Latitude3500: No need to sync WLAN-Tri-Valley.xml file to TVNE-Fog
[11-11-26 8:58:01 am] | CMD: lftp -e ‘set xfer:log 1; set xfer:log-file “/opt/fog/log/fogreplicator.Drivers_Latitude3500.transfer.TVNE-Fog.log”;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30;set net:limit-total-rate 0:64000000; mirror -c --parallel=20 -R --ignore-time -vvv --exclude “.srvprivate” “/images/Drivers/Latitude3500” “/images/Drivers/Latitude3500”;exit’ -u fogproject,[redacted] 10.225.9.50
[11-11-26 8:58:01 am] * Started sync for Image Drivers_Latitude3500 - Resource id #1109
[11-11-26 8:58:01 am] * Found Image to transfer to 3 groups
[11-11-26 8:58:01 am] | Image Name: Drivers_Latitude3510
[11-11-26 8:58:02 am] # Drivers_Latitude3510: File does not exist win11(TVAE-Fog)
[11-11-26 8:58:03 am] # Drivers_Latitude3510: File size mismatch - win11/3510win11.zip: 1736413172 != 1568858768
[11-11-26 8:58:03 am] | Deleting remote file: win11/3510win11.zip
[11-11-26 8:58:04 am] | Drivers_Latitude3510: No need to sync WLAN-Tri-Valley.xml file to TVAE-Fog
[11-11-26 8:58:04 am] | CMD: lftp -e ‘set xfer:log 1; set xfer:log-file “/opt/fog/log/fogreplicator.Drivers_Latitude3510.transfer.TVAE-Fog.log”;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30;set net:limit-total-rate 0:64000000; mirror -c --parallel=20 -R --ignore-time -vvv --exclude “.srvprivate” “/images/Drivers/Latitude3510” “/images/Drivers/Latitude3510”;exit’ -u fogproject,[redacted] 10.228.9.50
[11-11-26 8:58:04 am] * Started sync for Image Drivers_Latitude3510 - Resource id #1161
[11-11-26 8:58:05 am] # Drivers_Latitude3510: File does not exist win11(TVNE-Fog)
[11-11-26 8:58:05 am] # Drivers_Latitude3510: File size mismatch - win11/3510win11.zip: 1736413172 != 1568858768
[11-11-26 8:58:05 am] | Deleting remote file: win11/3510win11.zip
[11-11-26 8:58:10 am] | Drivers_Latitude3510: No need to sync WLAN-Tri-Valley.xml file to TVNE-Fog
[11-11-26 8:58:10 am] | CMD: lftp -e ‘set xfer:log 1; set xfer:log-file “/opt/fog/log/fogreplicator.Drivers_Latitude3510.transfer.TVNE-Fog.log”;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30;set net:limit-total-rate 0:64000000; mirror -c --parallel=20 -R --ignore-time -vvv --exclude “.srvprivate” “/images/Drivers/Latitude3510” “/images/Drivers/Latitude3510”;exit’ -u fogproject,[redacted] 10.225.9.50
[11-11-26 8:58:10 am] * Started sync for Image Drivers_Latitude3510 - Resource id #1195
[11-11-26 8:58:10 am] * Found Image to transfer to 3 groups
[11-11-26 8:58:10 am] | Image Name: Drivers_Latitude3520
[11-11-26 8:58:10 am] # Drivers_Latitude3520: File does not exist win11(TVAE-Fog)
[11-11-26 8:58:11 am] # Drivers_Latitude3520: File size mismatch - win11/3520win11.zip: 3035149676 != 1517288456
[11-11-26 8:58:11 am] | Deleting remote file: win11/3520win11.zip
[11-11-26 8:58:12 am] | Drivers_Latitude3520: No need to sync WLAN-Tri-Valley.xml file to TVAE-Fog
[11-11-26 8:58:12 am] | CMD: lftp -e ‘set xfer:log 1; set xfer:log-file “/opt/fog/log/fogreplicator.Drivers_Latitude3520.transfer.TVAE-Fog.log”;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30;set net:limit-total-rate 0:64000000; mirror -c --parallel=20 -R --ignore-time -vvv --exclude “.srvprivate” “/images/Drivers/Latitude3520” “/images/Drivers/Latitude3520”;exit’ -u fogproject,[redacted] 10.228.9.50
[11-11-26 8:58:12 am] * Started sync for Image Drivers_Latitude3520 - Resource id #1247
[11-11-26 8:58:13 am] # Drivers_Latitude3520: File does not exist win11(TVNE-Fog)
[11-11-26 8:58:14 am] # Drivers_Latitude3520: File size mismatch - win11/3520win11.zip: 3035149676 != 1517288456
[11-11-26 8:58:14 am] | Deleting remote file: win11/3520win11.zip
[11-11-26 8:58:15 am] | Drivers_Latitude3520: No need to sync WLAN-Tri-Valley.xml file to TVNE-Fog
[11-11-26 8:58:15 am] | CMD: lftp -e ‘set xfer:log 1; set xfer:log-file “/opt/fog/log/fogreplicator.Drivers_Latitude3520.transfer.TVNE-Fog.log”;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30;set net:limit-total-rate 0:64000000; mirror -c --parallel=20 -R --ignore-time -vvv --exclude “.srvprivate” “/images/Drivers/Latitude3520” “/images/Drivers/Latitude3520”;exit’ -u fogproject,[redacted] 10.225.9.50
[11-11-26 8:58:15 am] * Started sync for Image Drivers_Latitude3520 - Resource id #1281
[11-11-26 8:58:16 am] * Found Image to transfer to 2 groups
[11-11-26 8:58:16 am] | Image Name: Drivers_Latitude3540
[11-11-26 8:58:16 am] # Drivers_Latitude3540: File does not exist win11(TVNE-Fog)
[11-11-26 8:58:18 am] # Drivers_Latitude3540: File size mismatch - win11/3540_win11.zip: 1959506485 != 1530737272
[11-11-26 8:58:18 am] | Deleting remote file: win11/3540_win11.zip
[11-11-26 8:58:20 am] | Drivers_Latitude3540: No need to sync WLAN-Tri-Valley.xml file to TVNE-Fog
[11-11-26 8:58:20 am] | CMD: lftp -e ‘set xfer:log 1; set xfer:log-file “/opt/fog/log/fogreplicator.Drivers_Latitude3540.transfer.TVNE-Fog.log”;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30;set net:limit-total-rate 0:64000000; mirror -c --parallel=20 -R --ignore-time -vvv --exclude “.srvprivate” “/images/Drivers/Latitude3540” “/images/Drivers/Latitude3540”;exit’ -u fogproject,[redacted] 10.225.9.50
[11-11-26 8:58:20 am] * Started sync for Image Drivers_Latitude3540 - Resource id #1329
[11-11-26 8:58:20 am] * Not syncing Image between groups
[11-11-26 8:58:20 am] | Image Name: Drivers_Latitude3550
[11-11-26 8:58:20 am] | There are no members to sync to
[11-11-26 8:58:20 am] * Found Image to transfer to 2 groups
[11-11-26 8:58:20 am] | Image Name: Drivers_Latitude3590
[11-11-26 8:58:21 am] # Drivers_Latitude3590: File does not exist win11(TVNE-Fog)
[11-11-26 8:58:23 am] | Drivers_Latitude3590: No need to sync win11/3590-WIN10-A07-0N1F5.CAB file to TVNE-Fog
[11-11-26 8:58:24 am] | Drivers_Latitude3590: No need to sync WLAN-Tri-Valley.xml file to TVNE-Fog
[11-11-26 8:58:24 am] | CMD: lftp -e ‘set xfer:log 1; set xfer:log-file “/opt/fog/log/fogreplicator.Drivers_Latitude3590.transfer.TVNE-Fog.log”;set ftp:list-options -a;set net:max-retries 10;set net:timeout 30;set net:limit-total-rate 0:64000000; mirror -c --parallel=20 -R --ignore-time -vvv --exclude “.srvprivate” “/images/Drivers/Latitude3590” “/images/Drivers/Latitude3590”;exit’ -u fogproject,[redacted] 10.225.9.50
[11-11-26 8:58:24 am] * Started sync for Image Drivers_Latitude3590 - Resource id #1386
[11-11-26 8:58:25 am] * Not syncing Image between groups
[11-11-26 8:58:25 am] | Image Name: Drivers_Latitude5430Rugged
[11-11-26 8:58:25 am] | There are no members to sync to
[11-11-26 8:58:25 am] * Not syncing Image between groups
[11-11-26 8:58:25 am] | Image Name: Drivers_OptiPlex5260AIO
[11-11-26 8:58:25 am] | There are no members to sync to
[11-11-26 8:58:25 am] * Not syncing Image between groups
[11-11-26 8:58:25 am] | Image Name: Drivers_OptiPlex7060
[11-11-26 8:58:25 am] | There are no members to sync to
[11-11-26 8:58:25 am] * Found Image to transfer to 3 groups
[11-11-26 8:58:25 am] | Image Name: Drivers_OptiPlex7400AIO -
@FlareImp Thanks, the log shows the cause. 1.6.0.14 has a replication bug: the check for a transfer that is still running never matched. So each cycle compared the files while lftp was still copying them. It saw the half-copied file on the node, logged “File size mismatch”, deleted it, and started the copy again. A large file over your rate-limited link never finished. That is why rsync worked and the replicator did not.
1.6.0.158 fixes this. Please update the master and every storage node to 1.6.0.158, then turn the replicator back on.
The “File does not exist win11” lines are the subfolder name, not a missing file. 1.5 logs the same lines. They make lftp run once per cycle, but it copies nothing when the files already match.
-
@Tom-Elliott The issue is still happening. I have attached the replication log.
fogreplicator.log -
@Tom-Elliott I could be wrong but after stopping the Replication service on the Main and Master Slave. There was still a copy going on from the Slave node to the Master slave until I stopped the server on the slave node. backwords replication?
-
@FlareImp Thanks, that copy in the other direction is a good clue. The log shows two separate problems.
-
A file name with a space is never seen on the storage node. For example, “7450 AIO-WIN11-A10-YHP57.CAB” shows a size of 0 on the node every cycle, so the replicator deletes it and copies it again. This is a bug in how FOG reads the node’s FTP listing, and a fix is on the way. Until then, renaming those files without spaces stops the loop for them.
-
Files without a space, such as win11/3500-win11-A00-YH77C.CAB, show the same smaller size on both nodes every hour. A copy from the other node back toward your main server would explain that. Each storage group’s master sends an image to the other groups only when its own group is that image’s primary group. If another group’s master also sends the image, the two copies overwrite each other.
Can you check these for one affected image, such as Drivers_Latitude3500:
- Image Management > Drivers_Latitude3500 > Storage Groups tab: which group is marked primary?
- Storage Management: for each storage group, which node is the master? Is the node you saw copying marked as a master?
ls -l /images/Drivers/Latitude3500/win11/on the main server and on that node.
-
-
- I removed the spaces in the names of the driver files.
- The Image Primary Storage Group are set to “A(Default) - (1)”
Master Slave Node
administrator@tvne-fog:~$ ls -l /images/Drivers/Latitude3500/win11/ total 1522432 -rwxr-xr-x 1 fogproject fogproject 1558966032 Nov 13 14:20 3500-win11-A00-YH77C.CABMain
administrator@tv-fog-main:~$ ls -l /images/Drivers/Latitude3500/win11/ total 1652688 -rwxrwxrwx 1 fogproject administrator 1692345974 Dec 13 2023 3500-win11-A00-YH77C.CAB``` -
@FlareImp Thanks. The main server has the full file (1692345974 bytes). The node tvne-fog has a short copy (1558966032 bytes), and the log shows that same short size every hour. A copy that is still running would show a different size each time. So something writes the same short file to the node again and again.
The copy you saw from the other node is the likely source. Only a node marked as master runs replication. A master sends to every other enabled node in its own storage group, even to another master. If that node is also marked master and has a short copy of the file, it overwrites the copy on tvne-fog each cycle.
Can you check these:
- Storage Management: list each storage node, its storage group, and whether “Is Master Node” is checked.
- On the node you saw copying:
ls -l /images/Drivers/Latitude3500/win11/ - On the same node: the top of
/opt/fog/log/fogreplicator.logfor one cycle. It shows which nodes that node sends to.
If that node is a master in the same group as tvne-fog, uncheck “Is Master Node” on it. A storage group must have only one master.
-
@Tom-Elliott
There is only one master in each storage group. I have N01 and N02 disabled for right now until I can get replication from Main to tvne-fog working. -
This is the replication log from TVNE-Fog-N01 that is not a master node in the group.
fogreplicator.log -
@Tom-Elliott Was in the process of doing an rsync from TVNE-Fog to TVNE-Fog-N01 and I got this
Win11_25H2_FabLab/d1.shrunken.partitions 849 100% 1.46kB/s 0:00:00 (xfr#63, to-chk=21/141) Win11_25H2_FabLab/d1p1.img 0 0% 0.00kB/s 0:00:00 file has vanished: "/images/Win11_25H2_Smart26/d1.fixed_size_partitions" file has vanished: "/images/Win11_25H2_Smart26/d1.mbr" file has vanished: "/images/Win11_25H2_Smart26/d1.minimum.partitions" file has vanished: "/images/Win11_25H2_Smart26/d1.original.fstypes" file has vanished: "/images/Win11_25H2_Smart26/d1.original.swapuuids" file has vanished: "/images/Win11_25H2_Smart26/d1.partitions" file has vanished: "/images/Win11_25H2_Smart26/d1.shrunken.partitions" file has vanished: "/images/Win11_25H2_Smart26/d1p1.img" file has vanished: "/images/Win11_25H2_Smart26/d1p2.img" file has vanished: "/images/Win11_25H2_Smart26/d1p3.img" file has vanished: "/images/Win11_25H2_Smart26/d1p4.img" 12,705,589 100% 17.54MB/s 0:00:00 (xfr#64, to-chk=20/141)I had just finished copying the files from TV-Fog-Main to TVNE-Fog and I checked them and everything was there and matched up. I am unsure what just deleted the files.
