error 39 premature end no such file or directory

iamyogo

Hi All,

Very new here and to FOG but just wanted to say it’s a great piece of software and well done to the devs…

With that out of the way:

I’m having an issue where I’m getting this error (error 39 premature end no such file or directory) right at the end of an image deployment (about 98%-ish of the way through), but only if the image is captured with partclone-Zstd , it’s fine if captured with gzip…

Details, I’m just gonna dump everything I can tell you:

Image Source: VMWare Workstation 15.5 (multiple snapshots)
Image Traget: Lenovo X1 Carbon gen 2
Windows 10 x64 OS
Source is UEFI
Target is UEFI
FOG Version: 1.5.9-RC2 (also tried version before, RC1 something before update was released)
Kernel: 4.19.123 (also tried 118 before I tried updating to see if it was resolved)
Image Type: Single Disk Resizable
Partition: Everything
Compression: 20
Image Manager: Partclone-Zstd (fails) Partclone-gzip (works)
Size on Client: 14GB
Size on Server: 6GB

Captured 3 with Zstd times to make sure it was not a capture issue… all 3 times completed successfully
Deployed 3 times to ensure it wasn’t a 1-off error

Let me know what you want me to do to get you logs or something…

In my google travels, I came across this post which i thought was similar… but my image file doesn’t have any tacs/dashes in the name, and my image is windows …

cheers…

EDIT: I should say, that the error is 100% reproducible …

george1421

@iamyogo Ok you will need to do a little research. FOG captures images to /images/dev/<mac_address> directory. Once the capture is done the fog client logs into the FOG server using the FTP program and issues a move (mv) command to move the uploaded files from /images/dev/<mac_address> to /images/<image_name> directory.

What you need to do is

See if the /images/dev/<mac_address> exists. Normally if the capture process works correctly after the capture is done there should be no directories in /images/dev that appear to be mac addresses.
See if the directory /images/<image_name> exists and has files in it.

The file name for the image must be something that is compatible with the linux file system because the image name in the web ui will be the directory name in the linux file system.

iamyogo

@george1421

ok, but why would be ok with partclone-gzip and not partclone-Zstd ? Surely the process is the same, just a differing compression method?

All the files complete capture successfully…

files captured in dev<deviceMAC>
files moved to images<imagename>

d1.fixed_size_partitions
d1.mbr
d1.minimum.partitions
d1.original.fstypes
d1.original.swapuuids
d1.partitions
d1p1.img
d1p2.img
d1p3.img
d1p4.img

imagename is ZstdTest

Error Screens during deploy:

Testing VM-2020-05-19-13-34-32.png

Testing VM-2020-05-19-13-34-38.png

here are the log files from the client (/var/log)
partclone.log
messages.txt
tmpoutput.txt

george1421

@iamyogo OK based on the recent pictures it appears there is something wrong with d1p4.img (the 4th partition image file). Either it doesn’t exist or the image capture ended before the file was completely closed OR the target disk is just a bit smaller than the source media. Single disk resizable works well until it runs into a partition that can’t be resized such as an encrypted partition or a non resizable partition following a resizable partition.

Lets see the output of you cat ing these files.
d1.fixed_size_partitions
d1.minimum.partitions

Lets also see the output of ls -la /images/ZstdTest

Also from your source image if its MS Windows based lets see what the disk configuration looks like for this source disk. Please in include a screen shot.

george1421

@iamyogo said in error 39 premature end no such file or directory:

partclone-gzip and not partclone-Zstd

That is a good question. I can tell you that with FOG 1.5.8 partclone and zstd were updated with the buildroot update.

If you want to try something for the developers. Download this file https://fogproject.org/binaries1.5.7.zip and extract init.xz from the file.

You will want to rename the original file, but move this init.xz /var/www/html/fog/service/ipxe directory on the fog server. Then recapture and deploy with zstd. See if this corrects the issue. The file you downloaded is the virtual hard drive for the FOS Linux but for 1.5.7.

Tom Elliott

While the error is there, is it actually failing to do anything? The best of my knowledge, the “error” is a warning, and things will continue moving forward after the minute timeout. This is just to give time to let people know there may be an issue.

I see this error sporadically on my images as well (Same image, same model type -sometimes even the same machine itself- will give the warning), but it is just a warning and nothing seems amiss. I think it’s something to do with the version (that I was running at the time) of ZSTD and the compression. It would probably miss a null byte or something at the end, and because the decompression engine didn’t find the “end” of the file, it assumes it was just cut off (hence the premature end message presented.)

iamyogo

@george1421 said in error 39 premature end no such file or directory:

@iamyogo OK based on the recent pictures it appears there is something wrong with d1p4.img (the 4th partition image file). Either it doesn’t exist or the image capture ended before the file was completely closed OR the target disk is just a bit smaller than the source media. Single disk resizable works well until it runs into a partition that can’t be resized such as an encrypted partition or a non resizable partition following a resizable partition.

Lets see the output of you cat ing these files.
d1.fixed_size_partitions
d1.minimum.partitions

Lets also see the output of ls -la /images/ZstdTest

Also from your source image if its MS Windows based lets see what the disk configuration looks like for this source disk. Please in include a screen shot.

I’ll have a look at the re-capture re-deploy with the updated binaries in the morning as it’s a bit late …

cat of d1.fixed_size_partitions

1:2:3

cat of d1.minimum.partitions

label: gpt
label-id: B36636E6-1FA7-44D2-BE92-E8DEB414A4CA
device: /dev/sda
unit: sectors
first-lba: 34
last-lba: 209715166
sector-size: 512

/dev/sda1 : start=        2048, size=     1083392, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=6077D8BD-DEDC-463B-A3EE-E13587685F6A, name="Basic data partition", attrs="RequiredPartition GUID:63"
/dev/sda2 : start=     1085440, size=      202752, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=74EFE12E-9127-41E7-B3E6-7871C9F0AEB6, name="EFI system partition", attrs="GUID:63"
/dev/sda3 : start=     1288192, size=       32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=84B74C6A-DECC-4817-889F-F1063D08C574, name="Microsoft reserved partition", attrs="GUID:63"
/dev/sda4 : start=     1320960, size=    30081480, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=EC9F736A-DFE5-4071-B134-DB8817378C65, name="Basic data partition"

ls -la /FOG_images/ZstdTest

total 3869412
drwxrwxrwx  1 root root        272 May 19 13:24 .
drwxrwxrwx+ 1 root root        162 May 19 22:47 ..
-rwxrwxrwx  1 root root          6 May 19 13:17 d1.fixed_size_partitions
-rwxrwxrwx  1 root root    1048576 May 19 13:17 d1.mbr
-rwxrwxrwx  1 root root        878 May 19 13:17 d1.minimum.partitions
-rwxrwxrwx  1 root root         15 May 19 13:17 d1.original.fstypes
-rwxrwxrwx  1 root root          0 May 19 13:17 d1.original.swapuuids
-rwxrwxrwx  1 root root     148222 May 19 13:17 d1p1.img
-rwxrwxrwx  1 root root    8609547 May 19 13:17 d1p2.img
-rwxrwxrwx  1 root root    1373451 May 19 13:17 d1p3.img
-rwxrwxrwx  1 root root 3951075328 May 19 13:24 d1p4.img
-rwxrwxrwx  1 root root        878 May 19 13:17 d1.partitions

here is the diskpart readout from the source system:

@Tom-Elliott said in error 39 premature end no such file or directory:

While the error is there, is it actually failing to do anything? The best of my knowledge, the “error” is a warning, and things will continue moving forward after the minute timeout. This is just to give time to let people know there may be an issue.
I see this error sporadically on my images as well (Same image, same model type -sometimes even the same machine itself- will give the warning), but it is just a warning and nothing seems amiss. I think it’s something to do with the version (that I was running at the time) of ZSTD and the compression. It would probably miss a null byte or something at the end, and because the decompression engine didn’t find the “end” of the file, it assumes it was just cut off (hence the premature end message presented.)

The deployed image fails to boot… no repair windows no nothing… it’s as if it was a cleaned disk… perhaps that’s the clue? Is it not reading/writing the gpt or something to do with the resizing as @george1421 said? should I try to image as Zstd fixed size and see if that rectifies or narrows it down?

I mounted the vdisk (partition 4) and everything (all the normal windows stuff) seems to be there, though I’m not going to go through and diff/compare it to a normal install…

iamyogo

ok, I got curious with the alternative binaries… same result … re-captured with the new ones, deployed with the new ones, same same … premature end

iamyogo

and just for fun, i tried the same with the older init.xz (capture -> deploy) with the non-resizable option, no dice … same

george1421

@iamyogo Ok so we know now the version of zstd and partclone changing between version 1.5.7 and 1.5.8 doesn’t have an impact on ability to capture/deploy. (just to be clear the newer version of partclone can capture and deploy older versions of partclone captured images, but the older version of partclone can’t reader the newer versions format. So that’s why you need to recapture).

So the issue has to be in the source image or the target computer’s hardware. So what size hard drive are you deploying this image to?

iamyogo

@george1421 said in error 39 premature end no such file or directory:

So the issue has to be in the source image or the target computer’s hardware. So what size hard drive are you deploying this image to?

100GB, source and target are the same VM to rule out incompatibility …

And I tried again this morning and this time i got the repair windows screen… (resizable)

then I tried with the fixed size option and success! … still errored, but the system booted! (newer init.xz)

iamyogo

In any case… i’m going to stick with gzip, as that seems the most flexible and reliable… just thought I’d let you know

Sebastian Roth

@iamyogo I would say let’s try to see if different versions of Zstd all show the same issue. Download a FOS init build with the latest Zstd github commit as of now.

I can also build inits with older versions of Zstd to find out when the issue might have started. Are you keen to figure this out and maybe help find a bug in Zstd - if there is one…

iamyogo

@Sebastian-Roth

Sure … my kids are going back to school next week, so i’ll have a bit more free time to troubleshoot …

Sebastian Roth

@iamyogo Looking forward to hear if the latest version is doing any better.

Update: Just compiled a few more inits…

1.3.7
1.3.8
1.4.0
1.4.1
1.4.2
1.4.3
1.4.4 - the default we currently use in our inits, just to have the list completed
commit 17226f59

Please try those and report back if they all show the same issue or not.

Sebastian Roth

@iamyogo I’d be keen to hear if any of these did not show the error.

roarcore

Hi guys,

Was having the same issue: partclone-Zstd failing at 98%. Managed to resolve it by using compression: 6 instead of compression: 18 and recapturing the image. Attached images show the error I was getting.

CreativeSeeds

Hello

I had a problem when deploying an image with the FOG

Error message :
read ERROR: No such file or directory
Args Passed: /images/<image>/d1p2.img /dev/<disk>*

For me, it was a problem of user rights on the FOG

Before :
drwxrwxrwx. 2 root root 190 Jun 29 15:07 <image>

After:
drwxrwxrwx. 2 fogproject root 190 Jun 29 15:07 <image>

I had to do a “chown” on my image folder in the /images directory to add the owner to “fogproject”

When creating a new image in the FOG GUI, the folder rights management is not well handled

Sincerely

error 39 premature end no such file or directory

209

12.1k

17.3k

155.3k