Posts made by mp12

@Tom-Elliott after I noticed that you can’t distribute splitfiles via multicast, I discarded this possibility.

@Tom-Elliott after replicating 492 files and waiting one night, I can say that nothing goes wrong with small files. The check of the 200M files takes 2-3 seconds. Large files are still not checked correctly. I don’t know exactly at what size the test fails. A file with 11G is checked correctly. Files from 85G lead to errors.

@tom-elliott
just wanted to see if things improved over the weekend. But the new node has the same problem as discribed before. I will recheck the master and try a Zstd split.

@tom-elliott I will set up a new node and check if something goes wrong in replication. If not I will remove the old node.

@tom-elliott there is no lftp running. I am having one default image that passes the check normally. The filed1p3.img is around 11G.

[05-18-18 7:29:19 am] | Image Name: xxx_Default_Win10
[05-18-18 7:29:19 am] | xxx_Default_Win10: No need to sync d1.fixed_size_partitions file to node
[05-18-18 7:29:19 am] | xxx_Default_Win10: No need to sync d1.mbr file to node
[05-18-18 7:29:20 am] | xxx_Default_Win10: No need to sync d1.minimum.partitions file to node
[05-18-18 7:29:20 am] | xxx_Default_Win10: No need to sync d1.original.fstypes file to node
[05-18-18 7:29:20 am] | xxx_Default_Win10: No need to sync d1.original.swapuuids file to node
[05-18-18 7:29:20 am] | xxx_Default_Win10: No need to sync d1.original.uuids file to node
[05-18-18 7:29:20 am] | xxx_Default_Win10: No need to sync d1.partitions file to node
[05-18-18 7:29:21 am] | xxx_Default_Win10: No need to sync d1p1.img file to node
[05-18-18 7:29:21 am] | xxx_Default_Win10: No need to sync d1p2.img file to node
[05-18-18 7:30:49 am] | xxx_Default_Win10: No need to sync d1p3.img file to node
[05-18-18 7:30:52 am] | xxx_Default_Win10: No need to sync d1p4.img file to node
[05-18-18 7:30:52 am] * All files synced for this item.

Free disk space around 70G. I will move one image to free up some space and reboot both maschines.

The load from sha512sum process has decreased to 80% on the node. But now the master is also involved checking hash with high load at around 80%. Smaller files seem to be synced/checked properly. Bigger files obvisly do not match?

Inside the fogreplication.log I am getting messages like this:

[05-17-18 3:38:59 pm]  | Image Name: xxx_win7_26042018
[05-17-18 3:39:00 pm]  | xxx_win7_26042018: No need to sync d1.fixed_size_partitions file to node
[05-17-18 3:39:00 pm]  | xxx_win7_26042018: No need to sync d1.mbr file to node
[05-17-18 3:39:00 pm]  | xxx_win7_26042018: No need to sync d1.minimum.partitions file to node
[05-17-18 3:39:00 pm]  | xxx_win7_26042018: No need to sync d1.original.fstypes file to node
[05-17-18 3:39:01 pm]  | xxx_win7_26042018: No need to sync d1.original.swapuuids file to node
[05-17-18 3:39:01 pm]  | xxx_win7_26042018: No need to sync d1.partitions file to node
[05-17-18 3:39:01 pm]  | xxx_win7_26042018: No need to sync d1p1.img file to node
[05-17-18 3:51:16 pm]  | Files do not match on server: node
[05-17-18 3:51:16 pm]  | Deleting remote file: /images/xxx_win7_26042018/d1p2.img
[05-17-18 3:51:16 pm]  * Starting Sync Actions
[05-17-18 3:51:16 pm]  | CMD: lftp -e 'set xfer:log 1; set xfer:log-file "/opt/fog/log/fogreplicator.xxx_win7_26042018.transfer.node.log";set ftp:list-options -a;set net:max-retries 10;set net:timeout 30;  mirror -c --parallel=20 -R --ignore-time -vvv --exclude ".srvprivate" "/images/xxx_win7_26042018" "/images/xxx_win7_26042018"; exit' -u fog,[Protected] x.x.x.x

Last entry /opt/fog/log/fogreplicator.xxx_win7_26042018.transfer.node.log is in the past. I don’t think that the d1p2.img file was copied to the node again. I checked the Bandwidthmonitor for 30 minutes. Nothing higher than 3Mbps.

2018-05-17 10:12:37 /images/xxx_win7_26042018/d1p2.img -> ftp://xxx@x.x.x.x/%2Fimages/xxx_win7_26042018/d1p2.img 0-101176680804 27.78 MiB/s

So I think that the sync/check process now only has problems with larger files.
Size of d1p2.img is around 95G.

Thanks @Tom-Elliott. I switched to working. Deleted all images on fog node to see if replication is working properly. I will report back tomorrow.

@kagashe check this thread. https://forums.fogproject.org/topic/5300/cant-upload-snapins-anymore

maybe it has something to do with the configuration of your php.ini file. post_max_size upload_max_filesize

what does apache error.log say?

@tom-elliott just normal installed. One master server and one storage node. Maybe this is the mistake? Some time ago we had the location plugin enabled. But the problem existed before, I guess. We had a secound storage node some time ago, which I removed from the group. Both (all) running as VM.

Here is a copy from the fogreplicator.log. Maybe this helps

[04-30-18 1:25:42 pm] 
==================================
=== ==== ===== ====
=== ========= == === == ===
=== ======== ==== == ==== ===
=== ======== ==== == =========
=== ==== ==== == =========
=== ======== ==== == === ===
=== ======== ==== == ==== ===
=== ========= == === == ===
=== ========== ===== ====
==================================
===== Free Opensource Ghost ======
==================================
============ Credits =============
= https://fogproject.org/Credits =
==================================
== Released under GPL Version 3 ==
==================================


[04-30-18 1:25:42 pm] Interface Ready with IP Address: 127.0.0.1
[04-30-18 1:25:42 pm] Interface Ready with IP Address: 127.0.1.1
[04-30-18 1:25:42 pm] Interface Ready with IP Address: IPADRESS
[04-30-18 1:25:42 pm] Interface Ready with IP Address: HOSTNAME
[04-30-18 1:25:42 pm] * Starting ImageReplicator Service
[04-30-18 1:25:42 pm] * Checking for new items every 600 seconds
[04-30-18 1:25:42 pm] * Starting service loop
[04-30-18 1:25:42 pm] * Starting Image Replication.
[04-30-18 1:25:42 pm] * We are group ID: 1. We are group name: default
[04-30-18 1:25:42 pm] * We are node ID: 1. We are node name: fog
[04-30-18 1:25:42 pm] * Attempting to perform Group -> Group image replication.
[04-30-18 1:25:42 pm] | Replicating postdownloadscripts
[04-30-18 1:25:42 pm] * Found Image to transfer to 1 node
[04-30-18 1:25:42 pm] | File Name: postdownloadscripts
[04-30-18 1:25:43 pm] | postdownloadscripts: No need to sync fog.postdownload file to fognode
[04-30-18 1:25:43 pm] | 233 0 /images/postdownloadscripts/fog.postdownload ftp://XXX@IPADRESS/images/postdownloadscripts/fog.postdownload
[04-30-18 1:25:43 pm] | Files do not match.
[04-30-18 1:25:43 pm] * Deleting remote file: /images/postdownloadscripts/fog.postdownload
[04-30-18 1:25:43 pm] * Starting Sync Actions
[04-30-18 1:25:43 pm] | CMD:
			lftp -e 'set xfer:log 1; set xfer:log-file "/opt/fog/log/fogreplicator..transfer.fognode.log";set ftp:list-options -a;set net:max-retries 10;set net:timeout 30; mirror -c --parallel=20 -R --ignore-time -vvv --exclude ".srvprivate" "/images/postdownloadscripts" "/images/postdownloadscripts"; exit' -u fog,[Protected] IPADRESS
[04-30-18 1:25:43 pm] * Started sync for Image postdownloadscripts
[04-30-18 1:25:43 pm] | Replicating postinitscripts
[04-30-18 1:25:43 pm] * Found Image to transfer to 1 node
[04-30-18 1:25:43 pm] | File Name: dev/postinitscripts
[04-30-18 1:25:43 pm] | dev/postinitscripts: No need to sync fog.postinit file to fognode
[04-30-18 1:25:44 pm] | 249 0 /images/dev/postinitscripts/fog.postinit ftp://XXX@IPADRESS/images/dev/postinitscripts/fog.postinit
[04-30-18 1:25:44 pm] | Files do not match.
[04-30-18 1:25:44 pm] * Deleting remote file: /images/dev/postinitscripts/fog.postinit
[04-30-18 1:25:44 pm] * Starting Sync Actions
[04-30-18 1:25:44 pm] | CMD:
			lftp -e 'set xfer:log 1; set xfer:log-file "/opt/fog/log/fogreplicator..transfer.fognode.log";set ftp:list-options -a;set net:max-retries 10;set net:timeout 30; mirror -c --parallel=20 -R --ignore-time -vvv --exclude ".srvprivate" "/images/dev/postinitscripts" "/images/dev/postinitscripts"; exit' -u fog,[Protected] IPADRESS
[04-30-18 1:25:44 pm] * Started sync for Image dev/postinitscripts
[04-30-18 1:25:44 pm] * Not syncing Image between groups
[04-30-18 1:25:44 pm] | Image Name: Dell_Default_Win10
[04-30-18 1:25:44 pm] | There are no other members to sync to.
[04-30-18 1:25:44 pm] * Not syncing Image between groups
[04-30-18 1:25:44 pm] | Image Name: pbu_win7_19042018
[04-30-18 1:25:44 pm] | There are no other members to sync to.
[04-30-18 1:25:44 pm] * Not syncing Image between groups
[04-30-18 1:25:44 pm] | Image Name: sojus_win10_09012018
[04-30-18 1:25:44 pm] | There are no other members to sync to.
[04-30-18 1:25:44 pm] * Not syncing Image between groups
[04-30-18 1:25:44 pm] | Image Name: sojus_win7_20042018
[04-30-18 1:25:44 pm] | There are no other members to sync to.
[04-30-18 1:25:44 pm] * Not syncing Image between groups
[04-30-18 1:25:44 pm] | Image Name: sojus_win7_26042018
[04-30-18 1:25:44 pm] | There are no other members to sync to.
[04-30-18 1:25:44 pm] * Attempting to perform Group -> Nodes image replication.
[04-30-18 1:25:44 pm] * Found Image to transfer to 1 node
[04-30-18 1:25:44 pm] | Image Name: Dell_Default_Win10
[04-30-18 1:25:44 pm] | Dell_Default_Win10: No need to sync d1.fixed_size_partitions file to fognode
[04-30-18 1:25:44 pm] | 5 0 /images/Dell_Win10/d1.fixed_size_partitions ftp://XXX@IPADRESS/images/Dell_Win10/d1.fixed_size_partitions
[04-30-18 1:25:44 pm] | Files do not match.
[04-30-18 1:25:44 pm] * Deleting remote file: /images/Dell_Win10/d1.fixed_size_partitions
[04-30-18 1:25:44 pm] | 1048576 0 /images/Dell_Win10/d1.mbr ftp://XXX@IPADRESS/images/Dell_Win10/d1.mbr
[04-30-18 1:25:44 pm] | Files do not match.
[04-30-18 1:25:44 pm] * Deleting remote file: /images/Dell_Win10/d1.mbr
[04-30-18 1:25:44 pm] | 1048576 0 /images/Dell_Win10/d1.mbr ftp://XXX@IPADRESS/images/Dell_Win10/d1.mbr
[04-30-18 1:25:44 pm] | Files do not match.
[04-30-18 1:25:44 pm] * Deleting remote file: /images/Dell_Win10/d1.mbr
[04-30-18 1:25:44 pm] | 839 0 /images/Dell_Win10/d1.minimum.partitions ftp://XXX@IPADRESS/images/Dell_Win10/d1.minimum.partitions
[04-30-18 1:25:44 pm] | Files do not match.
[04-30-18 1:25:44 pm] * Deleting remote file: /images/Dell_Win10/d1.minimum.partitions
[04-30-18 1:25:45 pm] | 839 0 /images/Dell_Win10/d1.minimum.partitions ftp://XXX@IPADRESS/images/Dell_Win10/d1.minimum.partitions
[04-30-18 1:25:45 pm] | Files do not match.
[04-30-18 1:25:45 pm] * Deleting remote file: /images/Dell_Win10/d1.minimum.partitions
[04-30-18 1:25:45 pm] | 30 0 /images/Dell_Win10/d1.original.fstypes ftp://XXX@IPADRESS/images/Dell_Win10/d1.original.fstypes
[04-30-18 1:25:45 pm] | Files do not match.
[04-30-18 1:25:45 pm] * Deleting remote file: /images/Dell_Win10/d1.original.fstypes
[04-30-18 1:25:45 pm] | 30 0 /images/Dell_Win10/d1.original.fstypes ftp://XXX@IPADRESS/images/Dell_Win10/d1.original.fstypes
[04-30-18 1:25:45 pm] | Files do not match.
[04-30-18 1:25:45 pm] * Deleting remote file: /images/Dell_Win10/d1.original.fstypes
[04-30-18 1:25:45 pm] | Dell_Default_Win10: No need to sync d1.original.swapuuids file to fognode
[04-30-18 1:25:45 pm] | Dell_Default_Win10: No need to sync d1.original.swapuuids file to fognode
[04-30-18 1:25:45 pm] | 283 0 /images/Dell_Win10/d1.original.uuids ftp://XXX@IPADRESS/images/Dell_Win10/d1.original.uuids
[04-30-18 1:25:45 pm] | Files do not match.
[04-30-18 1:25:45 pm] * Deleting remote file: /images/Dell_Win10/d1.original.uuids
[04-30-18 1:25:45 pm] | 283 0 /images/Dell_Win10/d1.original.uuids ftp://XXX@IPADRESS/images/Dell_Win10/d1.original.uuids
[04-30-18 1:25:45 pm] | Files do not match.
[04-30-18 1:25:45 pm] * Deleting remote file: /images/Dell_Win10/d1.original.uuids
[04-30-18 1:25:45 pm] | 833 0 /images/Dell_Win10/d1.partitions ftp://XXX@IPADRESS/images/Dell_Win10/d1.partitions
[04-30-18 1:25:45 pm] | Files do not match.
[04-30-18 1:25:45 pm] * Deleting remote file: /images/Dell_Win10/d1.partitions
[04-30-18 1:25:46 pm] | 833 0 /images/Dell_Win10/d1.partitions ftp://XXX@IPADRESS/images/Dell_Win10/d1.partitions
[04-30-18 1:25:46 pm] | Files do not match.
[04-30-18 1:25:46 pm] * Deleting remote file: /images/Dell_Win10/d1.partitions
[04-30-18 1:25:46 pm] | 10829637 0 /images/Dell_Win10/d1p1.img ftp://XXX@IPADRESS/images/Dell_Win10/d1p1.img
[04-30-18 1:25:46 pm] | Files do not match.
[04-30-18 1:25:46 pm] * Deleting remote file: /images/Dell_Win10/d1p1.img
[04-30-18 1:25:46 pm] | 10829637 0 /images/Dell_Win10/d1p1.img ftp://XXX@IPADRESS/images/Dell_Win10/d1p1.img
[04-30-18 1:25:46 pm] | Files do not match.
[04-30-18 1:25:46 pm] * Deleting remote file: /images/Dell_Win10/d1p1.img
[04-30-18 1:25:46 pm] | 1503016 0 /images/Dell_Win10/d1p2.img ftp://XXX@IPADRESS/images/Dell_Win10/d1p2.img
[04-30-18 1:25:46 pm] | Files do not match.
[04-30-18 1:25:46 pm] * Deleting remote file: /images/Dell_Win10/d1p2.img
[04-30-18 1:25:46 pm] | 1503016 0 /images/Dell_Win10/d1p2.img ftp://XXX@IPADRESS/images/Dell_Win10/d1p2.img
[04-30-18 1:25:46 pm] | Files do not match.
[04-30-18 1:25:46 pm] * Deleting remote file: /images/Dell_Win10/d1p2.img
[04-30-18 1:33:11 pm] | 11452366679 0 /images/Dell_Win10/d1p3.img ftp://XXX@IPADRESS/images/Dell_Win10/d1p3.img
[04-30-18 1:33:11 pm] | Files do not match.
[04-30-18 1:33:11 pm] * Deleting remote file: /images/Dell_Win10/d1p3.img
[04-30-18 1:33:44 pm] | 11452366679 0 /images/Dell_Win10/d1p3.img ftp://XXX@IPADRESS/images/Dell_Win10/d1p3.img
[04-30-18 1:33:44 pm] | Files do not match.
[04-30-18 1:33:44 pm] * Deleting remote file: /images/Dell_Win10/d1p3.img
[04-30-18 1:33:47 pm] | 400428853 0 /images/Dell_Win10/d1p4.img ftp://XXX@IPADRESS/images/Dell_Win10/d1p4.img
[04-30-18 1:33:47 pm] | Files do not match.
[04-30-18 1:33:47 pm] * Deleting remote file: /images/Dell_Win10/d1p4.img
[04-30-18 1:33:48 pm] | 400428853 0 /images/Dell_Win10/d1p4.img ftp://XXX@IPADRESS/images/Dell_Win10/d1p4.img
[04-30-18 1:33:48 pm] | Files do not match.
[04-30-18 1:33:48 pm] * Deleting remote file: /images/Dell_Win10/d1p4.img
[04-30-18 1:33:48 pm] * Starting Sync Actions
[04-30-18 1:33:48 pm] | CMD:
			lftp -e 'set xfer:log 1; set xfer:log-file "/opt/fog/log/fogreplicator.Dell_Default_Win10.transfer.fognode.log";set ftp:list-options -a;set net:max-retries 10;set net:timeout 30; mirror -c --parallel=20 -R --ignore-time -vvv --exclude ".srvprivate" "/images/Dell_Win10" "/images/Dell_Win10"; exit' -u fog,[Protected] IPADRESS
[04-30-18 1:33:48 pm] * Started sync for Image Dell_Default_Win10
[04-30-18 1:33:48 pm] * Found Image to transfer to 1 node
[04-30-18 1:33:48 pm] | Image Name: pbu_win7_19042018
[04-30-18 1:33:49 pm] | pbu_win7_19042018: No need to sync d1.fixed_size_partitions file to fognode
[04-30-18 1:33:49 pm] | 3 0 /images/pbu_win7_19042018/d1.fixed_size_partitions ftp://XXX@IPADRESS/images/pbu_win7_19042018/d1.fixed_size_partitions
[04-30-18 1:33:49 pm] | Files do not match.
[04-30-18 1:33:49 pm] * Deleting remote file: /images/pbu_win7_19042018/d1.fixed_size_partitions
[04-30-18 1:33:49 pm] | 1048576 0 /images/pbu_win7_19042018/d1.mbr ftp://XXX@IPADRESS/images/pbu_win7_19042018/d1.mbr
[04-30-18 1:33:49 pm] | Files do not match.
[04-30-18 1:33:49 pm] * Deleting remote file: /images/pbu_win7_19042018/d1.mbr
[04-30-18 1:33:49 pm] | 1048576 0 /images/pbu_win7_19042018/d1.mbr ftp://XXX@IPADRESS/images/pbu_win7_19042018/d1.mbr
[04-30-18 1:33:49 pm] | Files do not match.
[04-30-18 1:33:49 pm] * Deleting remote file: /images/pbu_win7_19042018/d1.mbr
[04-30-18 1:33:49 pm] | 190 0 /images/pbu_win7_19042018/d1.minimum.partitions ftp://XXX@IPADRESS/images/pbu_win7_19042018/d1.minimum.partitions
[04-30-18 1:33:49 pm] | Files do not match.
[04-30-18 1:33:49 pm] * Deleting remote file: /images/pbu_win7_19042018/d1.minimum.partitions
[04-30-18 1:33:50 pm] | 190 0 /images/pbu_win7_19042018/d1.minimum.partitions ftp://XXX@IPADRESS/images/pbu_win7_19042018/d1.minimum.partitions
[04-30-18 1:33:50 pm] | Files do not match.
[04-30-18 1:33:50 pm] * Deleting remote file: /images/pbu_win7_19042018/d1.minimum.partitions
[04-30-18 1:33:50 pm] | 15 0 /images/pbu_win7_19042018/d1.original.fstypes ftp://XXX@IPADRESS/images/pbu_win7_19042018/d1.original.fstypes
[04-30-18 1:33:50 pm] | Files do not match.
[04-30-18 1:33:50 pm] * Deleting remote file: /images/pbu_win7_19042018/d1.original.fstypes
[04-30-18 1:33:50 pm] | 15 0 /images/pbu_win7_19042018/d1.original.fstypes ftp://XXX@IPADRESS/images/pbu_win7_19042018/d1.original.fstypes
[04-30-18 1:33:50 pm] | Files do not match.
[04-30-18 1:33:50 pm] * Deleting remote file: /images/pbu_win7_19042018/d1.original.fstypes
[04-30-18 1:33:50 pm] | pbu_win7_19042018: No need to sync d1.original.swapuuids file to fognode
[04-30-18 1:33:50 pm] | pbu_win7_19042018: No need to sync d1.original.swapuuids file to fognode
[04-30-18 1:33:50 pm] | 190 0 /images/pbu_win7_19042018/d1.partitions ftp://XXX@IPADRESS/images/pbu_win7_19042018/d1.partitions
[04-30-18 1:33:50 pm] | Files do not match.
[04-30-18 1:33:50 pm] * Deleting remote file: /images/pbu_win7_19042018/d1.partitions
[04-30-18 1:33:50 pm] | 190 0 /images/pbu_win7_19042018/d1.partitions ftp://XXX@IPADRESS/images/pbu_win7_19042018/d1.partitions
[04-30-18 1:33:50 pm] | Files do not match.
[04-30-18 1:33:50 pm] * Deleting remote file: /images/pbu_win7_19042018/d1.partitions
[04-30-18 1:33:51 pm] | 7931434 0 /images/pbu_win7_19042018/d1p1.img ftp://XXX@IPADRESS/images/pbu_win7_19042018/d1p1.img
[04-30-18 1:33:51 pm] | Files do not match.
[04-30-18 1:33:51 pm] * Deleting remote file: /images/pbu_win7_19042018/d1p1.img
[04-30-18 1:33:51 pm] | 7931434 0 /images/pbu_win7_19042018/d1p1.img ftp://XXX@IPADRESS/images/pbu_win7_19042018/d1p1.img
[04-30-18 1:33:51 pm] | Files do not match.
[04-30-18 1:33:51 pm] * Deleting remote file: /images/pbu_win7_19042018/d1p1.img
[04-30-18 1:38:51 pm] | 91071384023 0 /images/pbu_win7_19042018/d1p2.img ftp://XXX@IPADRESS/images/pbu_win7_19042018/d1p2.img
[04-30-18 1:38:51 pm] | Files do not match.
[04-30-18 1:38:51 pm] * Deleting remote file: /images/pbu_win7_19042018/d1p2.img

The last entry “/images/pbu_win7_19042018/d1p2.img” is now being checked by sha512sum.

@sebastian-roth here the results

du --version
du (GNU coreutils) 8.2

sha512sum --version
sha512sum (GNU coreutils) 8.21

du -b /images/pbu_win7_19042018/d1p2.img /images/sojus_win10_09012018/d1p2.img

91071384023     /images/pbu_win7_19042018/d1p2.img
90351730508     /images/sojus_win10_09012018/d1p2.img

ls -al /images/pbu_win7_19042018/d1p2.img /images/sojus_win10_09012018/d1p2.img

-rwxr-xr-x 1 fog fog 91071384023 Apr 25 11:05 /images/pbu_win7_19042018/d1p2.img
-rwxr-xr-x 1 fog fog 90351730508 Apr 25 11:24 /images/sojus_win10_09012018/d1p2.img

After waiting a while today (I started the node in the morning) now there are about 5 sha512sum processes
These sha512sum are blocking the website on the master. Timeout 504
Will shutdown the node overnight.

31667  3364 sh -c sha512sum '/images/sojus_win10_09012018/d1p2.img' | awk '{print $1}'
 3364  3365 sha512sum /images/sojus_win10_09012018/d1p2.img
26029  4053 sh -c sha512sum '/images/sojus_win10_09012018/d1p2.img' | awk '{print $1}'
 4053  4054 sha512sum /images/sojus_win10_09012018/d1p2.img
 3462  4295 sh -c sha512sum '/images/sojus_win7_20042018/d1p2.img' | awk '{print $1}'
 4295  4296 sha512sum /images/sojus_win7_20042018/d1p2.img
22534  4548 sh -c sha512sum '/images/sojus_win7_20042018/d1p2.img' | awk '{print $1}'
 4548  4549 sha512sum /images/sojus_win7_20042018/d1p2.img
14808  5149 sh -c sha512sum '/images/sojus_win7_26042018/d1p2.img' | awk '{print $1}'
 5149  5150 sha512sum /images/sojus_win7_26042018/d1p2.img

@sebastian-roth

 4029 18135 sh -c sha512sum '/images/pbu_win7_19042018/d1p2.img' | awk '{print $1}'
18135 18136 sha512sum /images/pbu_win7_19042018/d1p2.img
18496 18506 sh -c sha512sum '/images/sojus_win10_09012018/d1p2.img' | awk '{print $1}'
18506 18507 sha512sum /images/sojus_win10_09012018/d1p2.img

@tom-elliott where can I check hashes to compare between master and node? Are they saved somewhere?
Any logfiles if something is going wrong between master and node?

Did a fsck and nothing suspicious.
Machine is running on ESXi. Did a check of the vmdk aswell.
The harddisc where the VM is located also has no errors.

I am doing a resync of images/snapins to the storagenode.

Any ideas?

OS: Ubuntu 14.04
FOG: 1.5.2

My storage node runs several sha512sum processes (for long time now). Didn’t interupt these processes but they seem block the usability of the fog webpage. Now I desided to stop them and the web page was back on track. An image is around 100G. How long should the sha512sum process normally take?

Uninstalled and reinstalled the location plugin.
Assigned hosts to the locations and deploy now works like a charm.

Thanks a lot @Tom-Elliott

Here you go.

Server

• FOG Version: 1.3.1-RC-6
• OS: 14.04.1-Ubuntu

Description

I created a deploy task with 6 hosts using the location plugin. The hosts get the image from 3 different nodes.
On two hosts I am getting this error. Both have the same “location node”.

Server

• FOG Version: 1.3.1-RC-5
• OS: 14.04.1-Ubuntu

Description

Just noticed that the image replication isn’t working. I restarted all machines and services with no success.
Checked fogreplication.log and it tells that the sync started for the missing image. But I can’t recognize any transfers between the nodes.

[01-11-17 9:30:33 am]  * Found Image to transfer to 2 nodes
[01-11-17 9:30:33 am]  | Image Name: sojus_win7_10012017
[01-11-17 9:30:34 am]  * Starting Sync Actions
[01-11-17 9:30:34 am]  | CMD:
                        lftp -e 'set ftp:list-options -a;set net:max-retries 10;set net:timeout 30;  mirror -c -R --ignore-time -vvv --exclude "dev/" --exclude "ssl/" --exclude "CA" --delete-first '/images/sojus_win7_10012017' \''/images/sojus_win7_10012017'\'; exit' -u fog,[Protected] X.X.X.X
[01-11-17 9:30:34 am]  * Started sync for Image sojus_win7_10012017
[01-11-17 9:30:35 am]  | CMD:
                        lftp -e 'set ftp:list-options -a;set net:max-retries 10;set net:timeout 30;  mirror -c -R --ignore-time -vvv --exclude "dev/" --exclude "ssl/" --exclude "CA" --delete-first '/images/sojus_win7_10012017' \''/images/sojus_win7_10012017'\'; exit' -u fog,[Protected] X.X.X.X
[01-11-17 9:30:35 am]  * Started sync for Image sojus_win7_10012017

Then I started the FOGImagereplicator using

sudo /usr/bin/php -q /opt/fog/service/FOGImageReplicator

Thats the output I get:

PHP Warning:  fopen(sojus_win7_10012017./opt/fog/log/fogreplicator.transfer.fognode.log): failed to open stream: No such file or directory in /var/www/fog/lib/service/fogservice.class.php on line 252
PHP Warning:  fwrite() expects parameter 1 to be resource, boolean given in /var/www/fog/lib/service/fogservice.class.php on line 257
PHP Warning:  fclose() expects parameter 1 to be resource, boolean given in /var/www/fog/lib/service/fogservice.class.php on line 261
PHP Warning:  proc_open(sojus_win7_10012017./opt/fog/log/fogreplicator.transfer.fognode.log): failed to open stream: No such file or directory in /var/www/fog/lib/service/fogservice.class.php on line 730
PHP Warning:  fopen(sojus_win7_10012017./opt/fog/log/fogreplicator.transfer.node-3.log): failed to open stream: No such file or directory in /var/www/fog/lib/service/fogservice.class.php on line 252
PHP Warning:  fwrite() expects parameter 1 to be resource, boolean given in /var/www/fog/lib/service/fogservice.class.php on line 257
PHP Warning:  fclose() expects parameter 1 to be resource, boolean given in /var/www/fog/lib/service/fogservice.class.php on line 261
PHP Warning:  proc_open(sojus_win7_10012017./opt/fog/log/fogreplicator.transfer.node-3.log): failed to open stream: No such file or directory in /var/www/fog/lib/service/fogservice.class.php on line 730

The log files fogreplicator.transfer.fognode.log and fogreplicator.transfer.node-3.log don’t exist in /opt/fog/log.
However I see the files fogreplicator.log.transfer.fognode.log and fogreplicator.log.transfer.node-3.log which exist.

@Tom-Elliott

Found the error. After creating a new snapin and enabling it on creation, I rechecked the snapin and found that the checkbox Snapin Enabled was empty. Set a check mark, restarted replication and it uploaded the snapisn to the nodes.