I tested and uploaded another version: this time it’s the latest Partclone with the fix suggested by @Sebastian-Roth, i.e.

diff --git a/src/ddclone.c b/src/ddclone.c
index 04f8775..cab5386 100644
--- a/src/ddclone.c
+++ b/src/ddclone.c
@@ -30,7 +30,9 @@ void read_super_blocks(char* device, file_system_info* fs_info)
        }
        strncpy(fs_info->fs, raw_MAGIC, FS_MAGIC_SIZE);
        fs_info->block_size  = PART_SECTOR_SIZE;
-       //fs_info->device_size = get_partition_size(&src);
+#ifdef IMG
+       fs_info->device_size = get_partition_size(&src);
+#endif
        fs_info->totalblock  = fs_info->device_size / PART_SECTOR_SIZE;
        fs_info->usedblocks  = fs_info->device_size / PART_SECTOR_SIZE;
        close(src);

The symbol IMG is defined in Makefile when building partclone.imager and prevents this change from being built into partclone.dd. For now, I’ve successfully captured a disk from one machine and restored it to another one. After the coming Microsoft Patch Day (probably over the next weekend) I am planning to capture another disk image with this and deploy it to my pool in multi-cast mode.

This time, I also built the 64-bit kernel 4.19.94 and used it together with this image. It’s here.

@Quazz @Sebastian-Roth
The two awk lines changed in this commit are underquoted:
https://github.com/FOGProject/fos/commit/5f0e7b5f7d77986b0e0019dd23c875c6d40ec869

@shruggy said in 1.5.7.89: partclone doesn't capture an image in dd mode: wrong options in fog.upload:

After the coming Microsoft Patch Day (probably over the next weekend) I am planning to capture another disk image with this and deploy it to my pool in multi-cast mode.

I did it last weekend and the results are mixed. Yes, the image was successfully captured and then restored to 36 PCs in multi-cast. But: On five hosts I got this error message after restoring the image:

Reattempting to update database: Failed

The image was restored successfully on those hosts nevertheless. Only the FOG database wasn’t updated. All 36 PCs are identical hardware.

In the Imaging Log the End column for those five hosts says:

-0001-11-30 00:00:00

while the Duration column says:

2020 years 1 month 18 days 15 hours 35 minutes 43 seconds

It looks like somehow the data for Start timestamp got written into Duration?

@jhumpf I don’t use the FOG client on Linux, but these are my scripts for CentOS 6:
/etc/rc.d/rc.local:

#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local
. root/ckhostname
exit 0

/root/ckhostname:

#!/bin/sh
network=/etc/sysconfig/network
ipaddr=$(hostname -I)
shostname=$(hostname)
rhostname="$(dig +noall +answer +short -x $ipaddr)"
rhostname=${rhostname%.}
if [ "$rhostname" != "$shostname" ]; then
  sed -i "/^HOSTNAME=/s/=.*/=$rhostname/" "$network"
  hostname $rhostname
fi

I plan to retire them as I’m migrating the hosts to CentOS 8.

hostname -I may not give you what you want depending on how your system is configured, instead you can try something like ip -4 -br addr|awk -F'[ /]+' '/UP/,$0=$3' or, more cautiously worded:

ip -4 -br addr | awk '$2=="UP" {sub(/\/.*/,"",$3); print $3}'

@Sebastian-Roth said in FOG v1.5.7 on Ubuntu 18.04 random problem multicast:

@Tom-Elliott Have you ever seen a PC failing to proceed from one partition to the other in multicast?

IIRC, I had this same problem once or twice, after one of FOG upgrades, probably half a year ago. I guess that prompted me to try out the dev-branch back then, the problem went away and I stayed on the dev-branch since then.

@tec618 What happens if you deploy unicast to that PC that failed to pick up partition 8?

In my case, it never happened in the unicast mode. I have a setup with 4 legacy primary MBR partitions. NTFS (500 MB) - NTFS (237 GB) - ext2 (500 MB) - LVM2 (237 GB). d1.partitions:

label: dos
label-id: 0x871158f2
device: /dev/sda
unit: sectors

/dev/sda1 : start=        2048, size=     1024000, type=7, bootable
/dev/sda2 : start=     1026048, size=   499081216, type=7
/dev/sda3 : start=   500107264, size=     1024000, type=83
/dev/sda4 : start=   501131264, size=   499083264, type=8e

IIRC, the freeze up happened after restoring the 3rd partition.

@george1421 said in [dev-branch] multicast: for some hosts DB not updated after restore:

@shruggy How many systems do you typically image at the same time with multicast? How much memory do you have on the fog server?

Usually, it’s 36 systems at once. The setup is similar to @tec618’s: FOG on a VM with 4GB RAM, but both the VM and the hosting server run CentOS 7, and it’s Xen, not KVM. PHP 7.3 from Remi’s repo.

@Quazz @Sebastian-Roth
The two awk lines changed in this commit are underquoted:
https://github.com/FOGProject/fos/commit/5f0e7b5f7d77986b0e0019dd23c875c6d40ec869

@Sebastian-Roth You can mark this as solved now. I didn’t go with the adjustments you suggested, though: just wanted to try first the configuration suggested at https://www.sitepoint.com/php-fpm-tuning-using-pm-static-max-performance and it worked.

Here is an excerpt from my current /etc/php-fpm.d/www.conf (the changed lines are the first two and the last):

pm = static
pm.max_children = 40
pm.start_servers = 5
pm.min_spare_servers = 5
pm.max_spare_servers = 35
pm.max_requests = 500

I have a pool of 38 identical hosts.

See this post.
Apache logs don’t contain anything of note.
PHP-FPM log during (or shortly after) multicast restore sessions sometimes contains these warnings:

[04-Jan-2020 16:38:01] NOTICE: [pool www] child 29241 started
[05-Jan-2020 02:54:37] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 17 total children
[05-Jan-2020 02:54:38] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 16 children, there are 0 idle, and 22 total children
...
[25-Jan-2020 18:00:58] NOTICE: [pool www] child 9916 started
[25-Jan-2020 18:54:59] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 17 total children
[25-Jan-2020 18:55:00] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 16 children, there are 0 idle, and 22 total children
[25-Jan-2020 18:55:01] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 27 total children

Output of egrep '^pm\.(start|min|max)' /etc/php-fpm.d/www.conf

pm.max_children = 50
pm.start_servers = 5
pm.min_spare_servers = 5
pm.max_spare_servers = 10
pm.max_requests = 2000

Output of free -h on FOG server

              total        used        free      shared  buff/cache   available
Mem:           3,9G        362M        372M        8,6M        3,2G        3,5G
Swap:          4,0G          0B        4,0G

Output of lscpu | egrep '^Core|^Socket'

Core(s) per socket:    2
Socket(s):             1

Output of ps --no-headers -o rss,cmd -C php-fpm|awk '{sum+=$1}END{print sum/NR/1024"M"}'

21.2727M

Output of mysql -u root fog -e 'select l.taskID,s.tsName from taskLog as l,taskStates as s where l.taskStateID=s.tsID and l.id between 3282 and 3348 order by l.taskID'

+--------+-------------+
| taskID | tsName      |
+--------+-------------+
| 1701   | In-Progress |
| 1701   | Complete    |
| 1702   | In-Progress |
| 1702   | Complete    |
| 1703   | In-Progress |
| 1703   | Complete    |
| 1704   | In-Progress |
| 1704   | Complete    |
| 1705   | In-Progress |
| 1706   | In-Progress |
| 1706   | Complete    |
| 1707   | In-Progress |
| 1707   | Complete    |
| 1708   | In-Progress |
| 1708   | Complete    |
| 1709   | In-Progress |
| 1709   | Complete    |
| 1710   | In-Progress |
| 1710   | Complete    |
| 1711   | In-Progress |
| 1711   | Complete    |
| 1712   | In-Progress |
| 1712   | Complete    |
| 1713   | In-Progress |
| 1713   | Complete    |
| 1714   | In-Progress |
| 1714   | Complete    |
| 1715   | In-Progress |
| 1715   | Complete    |
| 1716   | In-Progress |
| 1716   | Complete    |
| 1717   | In-Progress |
| 1717   | Complete    |
| 1718   | In-Progress |
| 1718   | Complete    |
| 1719   | In-Progress |
| 1719   | Complete    |
| 1720   | In-Progress |
| 1721   | In-Progress |
| 1721   | Complete    |
| 1722   | In-Progress |
| 1722   | Complete    |
| 1723   | In-Progress |
| 1723   | Complete    |
| 1724   | In-Progress |
| 1724   | Complete    |
| 1725   | In-Progress |
| 1725   | Complete    |
| 1726   | In-Progress |
| 1726   | Complete    |
| 1727   | In-Progress |
| 1727   | Complete    |
| 1728   | In-Progress |
| 1728   | Complete    |
| 1729   | In-Progress |
| 1730   | In-Progress |
| 1730   | Complete    |
| 1731   | In-Progress |
| 1732   | In-Progress |
| 1732   | Complete    |
| 1733   | In-Progress |
| 1733   | Complete    |
| 1734   | In-Progress |
| 1735   | In-Progress |
| 1736   | In-Progress |
| 1736   | Complete    |
+--------+-------------+

@shruggy said in 1.5.7.89: partclone doesn't capture an image in dd mode: wrong options in fog.upload:

After the coming Microsoft Patch Day (probably over the next weekend) I am planning to capture another disk image with this and deploy it to my pool in multi-cast mode.

I did it last weekend and the results are mixed. Yes, the image was successfully captured and then restored to 36 PCs in multi-cast. But: On five hosts I got this error message after restoring the image:

Reattempting to update database: Failed

The image was restored successfully on those hosts nevertheless. Only the FOG database wasn’t updated. All 36 PCs are identical hardware.

In the Imaging Log the End column for those five hosts says:

-0001-11-30 00:00:00

while the Duration column says:

2020 years 1 month 18 days 15 hours 35 minutes 43 seconds

It looks like somehow the data for Start timestamp got written into Duration?

@Sebastian-Roth said in Development FOG not capturing image - PartClone update:

@shruggy Would you mind telling us which version of partclone this is?

It’s the latest git shapshot:
https://github.com/Thomas-Tsai/partclone/tree/58d138da6e0f473acd80a7c9b8544d104377d50f
with the fix you suggested:

$ cd git/fos/Buildroot/package/partclone
$ ls
Config.in  partclone-0.3.12.patch  partclone.imager.patch  partclone.mk
$ cat partclone.imager.patch
diff --git a/src/ddclone.c b/src/ddclone.c
index 04f8775..cab5386 100644
--- a/src/ddclone.c
+++ b/src/ddclone.c
@@ -30,7 +30,9 @@ void read_super_blocks(char* device, file_system_info* fs_info)
 	}
 	strncpy(fs_info->fs, raw_MAGIC, FS_MAGIC_SIZE);
 	fs_info->block_size  = PART_SECTOR_SIZE;
-	//fs_info->device_size = get_partition_size(&src);
+#ifdef IMG
+	fs_info->device_size = get_partition_size(&src);
+#endif
 	fs_info->totalblock  = fs_info->device_size / PART_SECTOR_SIZE;
 	fs_info->usedblocks  = fs_info->device_size / PART_SECTOR_SIZE;
 	close(src);

I haven’t yet pulled the latest FOS, however. So it doesn’t contain the changes to func.sh by @Tom-Elliott i.e. no special case for partclone.imager.

On the other note, when using unpatched Partclone 0.3.12 I also sometimes got segmentation fault, but didn’t report it. The case when Partclone finished successfully, but actually didn’t capture the partition was the more common outcome. I’d say, somewhere in the relation of 4 (empty) successes to 1 segfault, IIRC. And I didn’t notice any pattern to segfaults then (e.g. it could happen with or without compression).

@Quazz has just fixed it in the source.

And I suppose the errors won’t show up if you set the image type to “Multiple Partition Image - Single Disk (Not Resizable)”, since the awk script gets only invoked during partition resizing.

@ty900000 Could you please try my init.xz with patched Partclone and say if it works for you?