storage node sha512sum at 100% CPU/HDD usage

[Tom Elliott]

@mp12 I’m not seeing the same issue. I suppose it could be how large the files are but that still seems unlikely. Is it possible the Lftp command is stuck itself? On the transferring node you can see running lftp processes via ps -ef|grep lftp and I’m going to guess there is none running. Maybe disk space on the receiving node is out?

[mp12]

@tom-elliott there is no lftp running. I am having one default image that passes the check normally. The filed1p3.img is around 11G.

[05-18-18 7:29:19 am] | Image Name: xxx_Default_Win10
[05-18-18 7:29:19 am] | xxx_Default_Win10: No need to sync d1.fixed_size_partitions file to node
[05-18-18 7:29:19 am] | xxx_Default_Win10: No need to sync d1.mbr file to node
[05-18-18 7:29:20 am] | xxx_Default_Win10: No need to sync d1.minimum.partitions file to node
[05-18-18 7:29:20 am] | xxx_Default_Win10: No need to sync d1.original.fstypes file to node
[05-18-18 7:29:20 am] | xxx_Default_Win10: No need to sync d1.original.swapuuids file to node
[05-18-18 7:29:20 am] | xxx_Default_Win10: No need to sync d1.original.uuids file to node
[05-18-18 7:29:20 am] | xxx_Default_Win10: No need to sync d1.partitions file to node
[05-18-18 7:29:21 am] | xxx_Default_Win10: No need to sync d1p1.img file to node
[05-18-18 7:29:21 am] | xxx_Default_Win10: No need to sync d1p2.img file to node
[05-18-18 7:30:49 am] | xxx_Default_Win10: No need to sync d1p3.img file to node
[05-18-18 7:30:52 am] | xxx_Default_Win10: No need to sync d1p4.img file to node
[05-18-18 7:30:52 am] * All files synced for this item.

Free disk space around 70G. I will move one image to free up some space and reboot both maschines.

[Tom Elliott]

@mp12 Any word on this? It really seems there could be a disk usage type issue, though your free space does appear to fit the size needed for that image. Then again, if there’s other images also trying to replicate at the same time, this could pose a bit of a problem as well.

[mp12]

@tom-elliott I will set up a new node and check if something goes wrong in replication. If not I will remove the old node.

[Tom Elliott]

Any word on this @mp12? I want to nail this one down, though you already admitted that it seems a little better. Hopefully a new node will help out a bit?

Thanks for the feedback.

[mp12]

@tom-elliott
just wanted to see if things improved over the weekend. But the new node has the same problem as discribed before. I will recheck the master and try a Zstd split.

[mp12]

@Tom-Elliott after replicating 492 files and waiting one night, I can say that nothing goes wrong with small files. The check of the 200M files takes 2-3 seconds. Large files are still not checked correctly. I don’t know exactly at what size the test fails. A file with 11G is checked correctly. Files from 85G lead to errors.

[mp12]

@Tom-Elliott after I noticed that you can’t distribute splitfiles via multicast, I discarded this possibility.

[Sebastian Roth]

Should be solved in latest as we have re-worked the replication stuff a fair bit.

[mp12]

@Sebastian-Roth
Thanks for the reply! Did an update to 1.5.5.1 two weeks ago. Everything works fine so far!

[jeffersondv]

This post is deleted!