Cloning loops forever

Xibeaga · Nov 20, 2018, 10:03 PM

Hi!

My clone (capture) loops forever.

I tried these 3 things - more than once

1st) i checked all passwords, I walked through https://wiki.fogproject.org/wiki/index.php/Troubleshoot_FTP

2nd) i installed 1.5.5 over 1.5.0

3rd) i checked all passwords - incl. the linux-user “fog”

What happens:

when clone is finished, it starts from beginning. Computer does NOT(!) reboot, it simply starts to clone again.

the /images/dev/00… looks like:
drwxrwxrwx 2 fog root 96 Nov 20 22:56 .
drwxrwxrwx 4 fog root 4096 Nov 20 22:04 …
-rw-r–r-- 1 root root 0 Nov 20 22:56 d1.has_grub
-rw-r–r-- 1 root root 1048576 Nov 20 22:56 d1.mbr
-rw-r–r-- 1 root root 20 Nov 20 22:56 d1p1.img.000
-rw-r–r-- 1 root root 182 Nov 20 22:56 d1.partitions

Give attention to the d1p1.img.000 entry.
It starts with size 0, then goes to 20, and thats it. it never grows bigger.
i can change whatever permissions what ever i want, no change.

when ist starts over, the MAC-Dir is deleted and newly made with root:root and thats it.

Any help is welcome!
regards

Sebastian Roth · Nov 21, 2018, 8:17 PM

@Xibeaga Sound interesting. Can’t remember anyone reporting such an issue where it would loop without rebooting in between.

Give attention to the d1p1.img.000 entry.

That is fine as images are named like this while capturing is still going. It should rename d1p1.img.000 to d1p1.img if partclone finishes properly. If it doesn’t then it would fail out with an error message.

FOG does actually loop through all your partitions and possibly disks (if you have more than one and settings are to capture all).

Can you please take a video or describe more in detail what exactly FOG does on your client so we might figure out why and where it loops. Video would definitely help a lot!

Xibeaga · Nov 25, 2018, 9:24 AM

Hi There!

Uh! It took me a while to make a video.

Since i installed 1.5.5 the behaviour changed a bit: now the computer reboots everyx time it wants to get a new clone, but loops anyway.

Where can I put it? Its got 80MB. Maybe i can send it by “wetransfer.com”?

Here is the description:

you can see 3 Windows:
upper left: serial console to the computer that shall be captured
upper rigth: „watch -la /images/dev/00…“

these are the events at specific timestamps:
0:29 computer boots (its headless, only serial console is available to watch it)
1:47: directories+files are created
1:50: clone starts
3:28 reboot
4:54 ftp test

Sebastian Roth · Nov 25, 2018, 6:06 AM

@Xibeaga Yes, please upload onto wetransfer, dropbox or whatever you have access to. Looking forward to watch the whole video. I am pretty sure we can figure this out.

Xibeaga · Nov 25, 2018, 10:14 AM

Dropbox
here ii is:
https://www.dropbox.com/s/8iuhvmlwuepwihw/fog.webm?dl=0

Sebastian Roth · Nov 25, 2018, 7:33 AM

@Xibeaga Alright, good you took that video because it’s very different to setups we usually see, mainly because you don’t seem to have more than just the boot and pre kernel output on screen which makes debugging more difficult - not impossible though. What kind of headless machine is that? Maybe we can also help you get at least the cloning output on screen as well. Possibly just needs some kernel options enabled or maybe even just kernel parameters passed on boot (which FOG can do). If it is KVM then possibly console=hvc0 might help…

Thinking a bit more about it I reckon I have a better idea for you: using a debug task and SSH into it would be just right for your case. You see all the output there. The only issue is that we don’t have an initial password set in the FOS environment which prevents you from simply SSH into it. But this can be fixed. Run the following commands (tested on CentOS 7, hope this is working for you as well):

sudo -i
cd /var/www/html/fog/service/ipxe/
cp init.xz init_mod.xz
unxz init_mod.xz
mkdir loopmnt/
mount -o loop init_mod loopmnt/
vi loopmnt/etc/shadow

Sorry but chroot-ing into that environment to properly passwd fails with FATAL: kernel too old on my CentOS 7 machine as the host kernel seems to be too old to match our FOS glibc. But if you simply edit the shadow file by hand it does the same thing. Change the line root:*:... to root:$1$.6iRZggR$nSNK0KYm9aFQPo4Lh.6ZH1:... for password Mojo@work. Close and save, then:

umount loopmnt/
rmdir loopmnt/
xz -C crc32 -9 init_mod
chown fog:apache init_mod.xz
mv init.xz init_orig.xz && mv init_mod.xz init.xz

Now schedule an upload task for this client but tick the checkbox for debug this time. Wait a little and the try to SSH into it. As we generate new SSH host keys on every FOS boot you might want to use this: ssh -o "UserKnownHostsFile=/dev/null" -o "StrictHostKeyChecking=no" root@ip.of.client.host

When you are logged into the client machine just start the upload task by running the command and stepping through: fog

As well you might answer a couple of questions just to make finding the right spots in the script code easier for me:

cat /images/dev/00…/d1.partitions as well as file /images/dev/00…/d1p1.img.000 and post output here
Compression is set to ZSDT/GZIP?
Image Type, non-resizable single disk multiple partitions I guess, right?

Xibeaga · Nov 25, 2018, 2:40 PM

Dear Sebastian!

Wow! Any kind of display at headless machines would be great!
It is an APU-Board.
https://www.pcengines.ch/apu3c4.htm

enabling the sserial console withj kernel params would be cool!

Your guess: single disk multi partitions is correct

Your suggestion lead to this: Everything went fine. In debug-mode the clone was successful.

After the successful cloning in debug-mode
I tried again with non-debug but did not change the moidified ramdisk back, cloning worked

After the successful cloning I re-did with the original ramdisk: cloning worked

I have no idea whats going on.
thx!
R

root@1stNAS:/images/dev/000db94a39a8# file d1p1.img.000 
d1p1.img.000: gzip compressed data, last modified: Sun Nov 25 20:22:26 2018, from Unix

cat d1.partitions 
label: dos
label-id: 0x42d3de51
device: /dev/sda
unit: sectors

/dev/sda1 : start=        2048, size=   449218560, type=83
/dev/sda2 : start=   449220608, size=    39176560, type=82

And here comes the log:

[Sun Nov 25 root@fogclient ~]# fog
 * Running post init scripts.........................Done
 * Press [Enter] key to continue


   ==================================
   ===        ====    =====      ====
   ===  =========  ==  ===   ==   ===
   ===  ========  ====  ==  ====  ===
   ===  ========  ====  ==  =========
   ===      ====  ====  ==  =========
   ===  ========  ====  ==  ===   ===
   ===  ========  ====  ==  ====  ===
   ===  =========  ==  ===   ==   ===
   ===  ==========    =====      ====
   ==================================
   ===== Free Opensource Ghost ======
   ==================================
   ============ Credits =============
   = https://fogproject.org/Credits =
   ==================================
   == Released under GPL Version 3 ==
   ==================================
   Version: 1.5.5
 * Press [Enter] key to continue

 * Verifying network interface configuration.........Done
 * Press [Enter] key to continue

 * Checking Operating System.........................Windows XP
 * Checking CPU Cores................................4
 * Send method.......................................NFS
 * Attempting to check in............................Done
 * Press [Enter] key to continue

 * Mounting File System..............................Done
 * Press [Enter] key to continue

 * Checking Mounted File System......................Done
 * Press [Enter] key to continue

 * Checking img variable is set......................Done
 * Press [Enter] key to continue

 * Preparing to send image file to server
 * Preparing backup location.........................Done
 * Press [Enter] key to continue

 * Setting permission on /images/000db94a39a8........Done
 * Press [Enter] key to continue

 * Removing any pre-existing files...................Done
 * Press [Enter] key to continue

 * Using Image: nsa193_ssd_msata
 * Looking for Hard Disk.............................Done
 * Press [Enter] key to continue

 * Reading Partition Tables..........................Done
 * Press [Enter] key to continue

 * Using Hard Disk: /dev/sda
 * Now FOG will attempt to capture the image using Partclone
 * Press [Enter] key to continue

 * Processing Hard Disk: /dev/sda
 * Saving original disk/parts UUIDs
 * Press [Enter] key to continue


 * Saving Partition Tables (MBR).....................Done
 * Press [Enter] key to continue
Done
 * Press [Enter] key to continue

 * Processing Partition: /dev/sda1 (1)
 * Press [Enter] key to continue

 * Using partclone.extfs
 * Press [Enter] key to continue

Cloned successfully.
 * Image Captured
 * Press [Enter] key to continue

 * Processing Partition: /dev/sda2 (2)
 * Press [Enter] key to continue

 * Saving swap partition UUID
 * Saving UUID (bf7eaf1e-a5fc-4a3a-aa1d-3fc55096e747) for (/dev/sda2)
 * Stopping FOG Status Reporter......................Done
 * Press [Enter] key to continue

 * Task Complete
 * Updating Database.................................
Done
 * Press [Enter] key to continue
[Sun Nov 25 root@fogclient ~]#

@Sebastian-Roth said in Cloning loops forever:

ssh -o “UserKnownHostsFile=/dev/null” -o “StrictHostKeyChecking=no” root@

Sebastian Roth · Nov 25, 2018, 8:46 PM

@Xibeaga said in Cloning loops forever:

I have no idea whats going on.

Haha, me neither…

Any kind of display at headless machines would be great!

I remember when installing CentOS on an APU I had to add kernel parameters for console output. Can’t remember which from the top of my head but a little research is saying that vga=off console=ttyS0,115200n8 should do the trick. Add that to the host’s settings (Host Kernel Parameters) and see if you can get some output.

Beside that, using SSH is a great alternative I guess.

Xibeaga · Nov 25, 2018, 9:20 PM

Hello Sebastian!
Cloning works again, maybe some sun flares

Your kernel-params for sercon are working great!
Thanks a lot for your help!

Best regards,
R.

Tom Elliott · Nov 26, 2018, 1:51 AM

@Xibeaga actually, no. There’s been very few this past year.

Cloning loops forever

157

12.0k

17.3k

155.2k