FOG: Not detecting target disks correctly (/dev/sda vs /dev/xvda)

[JonesDK]

And that was it - no other errors or strange things (saying the guy with 0 sucessfull deployments)

[george1421]

@JonesDK Now at this point you are at an error and a breakpoint. If you hit ctrl-c it should toss you back to the fos linux command prompt. At this time key in the lsblk command again so we can see the partitions as it created them on disk.

The second front will be back on the fog server the developers will need to see the output of these posted here.

cat /images/goldenimage8/d1.partitions
cat /images/goldenimage8/d1.fixed_size_partitions
cat /images/goldenimage8/d1.fixed_size_partitions

Now looking at your original post I’m confused because on the golden image the partition “content” size doesn’t match what FOG captured. These are the compressed contents of the partitions. You see partition 1 has 121.7MB in compressed size. Partition 2 has 74bytes in size. So something happened during image capture.

-rwxrwxrwx. 1 root root 121741198 Dec 29 22:55 d1p1.img
-rwxrwxrwx. 1 root root 74 Dec 29 22:55 d1p2.img

If you look at the output of the df -h command on the reference image partition 1 [ d1p1.img ] should be ~120MB and partition 2 [d1p2.img] should be ~1GB.

So based on this I’m going to say something happened with the image capture when partition 2 was being uploaded to the fog server.

So what I’ll recommend is that you abort the debug deploy and terminate the task on the FOG server web ui then schedule another image capture of your reference image. Go ahead and tick the debug box when you schedule the capture task against your reference image. Once at the FOS Linux command prompt key in fog and single step through the capture process. I’m expecting an error to be thrown on the second partclone screen. When partclone throws an error is just writes the error where ever the cursor is on the screen. Its a bit of a mess, but the developers will need to see that error.

Just be aware the developers are on holiday until after the first of the year. They do check in now and again, but not on any consistent basis. So just be aware if its something that they need to dig into their response may be delayed. I can say they are getting to release FOG 1.5.8 in January so if this is a problem in the code they will want to get it sorted out before 1.5.8 is released.

[JonesDK]

first reply

[JonesDK]

cat d1.partitions

label: dos
label-id: 0x038fb017
device: /dev/sda
unit: sectors

/dev/xvda1 : start=        2048, size=     2097152, type=83, bootable
/dev/xvda2 : start=     2099200, size=     8386560, type=8e

cat d1.fixed_size_partitions

2

[JonesDK]

@george1421 said in FOG: Not detecting target disks correctly (/dev/sda vs /dev/xvda):

df -h

Here u go

Filesystem           Size  Used Avail Use% Mounted on
devtmpfs             1.9G     0  1.9G   0% /dev
tmpfs                851M     0  851M   0% /dev/shm
tmpfs                851M   17M  835M   2% /run
tmpfs                851M     0  851M   0% /sys/fs/cgroup
/dev/mapper/cl-root  3.5G  1.3G  2.3G  36% /
/dev/xvda1           976M  126M  783M  14% /boot
tmpfs                171M     0  171M   0% /run/user/0

[george1421]

@JonesDK As I posted last, lets go ahead and recapture your golden image in debug mode to try to understand why it only copied 74 bytes of your root partition.

Also this is outside of the issue you have at the moment and just a few comments

1. If you recreate your golden image, but manually provision the disk using standard partitions instead of an LVM volume FOG will be able to expand the root partition to the size of the target drive. The centos 8 default is to create a LVM volume (kind of a partition inside of a partition). But for a dedicated VM standard partitions will work just fine if you only have 4 or less partitions (including the boot partition).

2. Centos as well as RHEL have a configuration script called kickstart (not to be confused with the windows kixstart batch processor). If you close one eye and squint with the other it kind of works like MDT does for windows, but not really. It may not fit your needs here, but just be aware its available for centos.

3. Since you are a windows guy, linux has a much older equivalent batch processor call the shell. There are a number of command shells out there, but the most common is called bash. Think of the bash sell as a cross between a DOS batch file programming and Powershell. You can use bash as well as some of the other command line utilities to dynamically configure your centos server. If you use a FOG postinstall script, you can write FOG variables to a config file during imaging and then have your configuration script read that config file to aid in configuring the target computer. Linux has something equivalent to the windows services applet structure. The current command line version is called systemctl that is for systemD type systems. There is an older services manager for SysV type systems. (hang with me here, I am going to a point). If you place a bash file in /etc/init.d it will be executed when the system boots. So if you name the file in /etc/init.d something like S99Configure (the name is not arbitrary) S means to run at startup. The 99 is just a sequential number. The scripts in that directory are run alphabetically. So S99 will be run after a script that starts S30. The rest of the text is just used to idenitfy the script and to also aid in running order. It is allowed to have 2 or more scripts that start with S99… as long as the entire name is unique. So to configure the system on the first startup after imaging create a bash script /etc/init.d/S99Configure and put the bash commands in it to customize your target image. The last line of the bash script should be to remove the S99Configure script so it only runs once.

[Sebastian Roth]

@JonesDK Great you have added so many details in your initial post already, well done! Reading through the whole topic over and over I find it a bit hard to figure out where some of the information (results from commands executed) stem from. So please bear with me if I get some things wrong.

From what I see the main issue here is that FOG/FOS does not handle LVM. This is something on the list that we never found the time to actually implement and most users work around it by using/switching to standard partition layout. That said, I still fancy the idea of properly implementing LVM support some time in the future.

Now about “/dev/sda vs /dev/xvda”: The device names on Linux systems depend on the subsystem used in general. If you run a VM in VirtualBox you usually have sda same as if you have SCSI/SATA/HD/SSD drives in physical machines. I am not sure but I think this is the same for VMware and maybe as well in Hyper-V as they all use some kind of (emulated) SCSI subsystem layer. Now Xen (XCP-ng) uses a differend subsystem (xen_blk) and therefore has different device names like xvda. And there are more, like /dev/nvme0n1 and /dev/mmcblk0… FOS should be able to “convert” from one to the other and vice versa. It does not really convert but simply ignores the sda/xvda/nvme/mmcblk information but only enumerates disks and partitions. There is no need to manually edit d1.partitions d1.minimum.partitions!

-rwxrwxrwx. 1 root root 74 Dec 29 22:55 d1p2.img

The size of the image file shows that FOS is not able to capture the LVM partitions housed within the second partition of your golden master.

All in all I’d say it would be nice to at least detect LVM and print out an error on capture already! I will see if I can squeeze that in before we push out the next FOG release.

[george1421]

Cross linking posts since the issues are almost the same: https://forums.fogproject.org/topic/14078/1-5-7-89-partclone-doesn-t-capture-an-image-in-dd-mode-wrong-options-in-fog-upload

[shruggy]

Update. Image updated and can be tested now.

@JonesDK I’m the topic starter from the post cross-linked by @george1421. Here is the FOS image I built. Could you please test it and report if it works for you?

I’ve only built an 64-bit image. If your FOG server is on CentOS put it into the /var/www/fog/service/ipxe directory (backup the old init.xz beforehand). Change the ownership to be the same as of init_32.xz file in there and don’t forget to restore the SELinux context:

cd /var/www/fog/service/ipxe
sudo mv init.xz init.xz.bak
sudo mv ~/Downloads/init.xz . 
sudo chown --reference=init_32.xz init.xz
sudo restorecon init.xz

Now try to capture the image with FOG. On my system I was using the image type “Multiple Partition Image - Single Disk (Not Resizable)” and as image manager “Partclone Zstd” with compression level 7.

[Quazz]

@shruggy partclone.imager will generate partclone like files using a dd-like approach and is the one used in FOG when selecting raw image.

[Sebastian Roth]

@JonesDK More details on this here: https://github.com/FOGProject/fos/issues/34

[Sebastian Roth]

@JonesDK Thanks to @shruggy we found out that the new partclone version used introduced a bug that breaks raw (!) capturing of partitions. While I didn’t even know people use FOG to capture LVM in raw mode it’s definitely good we found this. More details in this here: https://github.com/FOGProject/fos/issues/35

[shruggy]

@JonesDK Sorry for the delay. You can try it out now (as described below).

[Sebastian Roth]

Latest inits should have all the fixes included. Marking as solved.