Image replication loop - incorrect file size calculation

[Mark Shelton]

Hi,
I have FOG 1.3.0 RC4 SVN 5941 running on 3 servers, connected via WAN. For the last few revisions, image replication has been looping. It takes a few days to replicate over the WAN and when finished starts all over again.
Upon further investigation of the logs, I have realised that it is because the replication process incorrectly calculates the local file size. Please find attached two shapshots. One of the local image filesize of the largest file being 8306809333 bytes. The other image shows the log displaying the remote filesize as the correct number, but the local size incorrectly as -283125259. So the replication starts all over again.

[Tom Elliott]

@Wayne-Workman It’s requesting the filesize from the remote server using ftp. Locally it is using the filesize php command.

I’ve created an arch agnostic filesize getter which should fix this issue.

Will be in RC-6.

[Tom Elliott]

Just a guess, but a 32 bit operating system?

[Wayne Workman]

Also, please update to RC-5.

[Mark Shelton]

@Tom-Elliott Yes 32 bit. However it reads the remote image size accurately which is also 32 bit.

[Tom Elliott]

@Mark-Shelton And both nodes are at the same version?

[Mark Shelton]

@Wayne-Workman I will try that, but this issue has been happening for a few releases. I’ve not noticed anyone else reporting the problem so not sure if it has been addressed.

[Mark Shelton]

@Tom-Elliott Yes all are FOG 1.3.0 RC4 SVN 5941. All have Ubuntu 14.04 installed. Interesting that the local file size is always incorrect for the large files. This has been happening for a while, I’ve only just realised why.

[Wayne Workman]

@Mark-Shelton said in Image replication loop - incorrect file size calculation:

Yes 32 bit. However it reads the remote image size accurately which is also 32 bit.

But, it’s requesting the data from the remote server, which is 64 bit I presume. The remote server is the one doing the measuring on it’s files.

32 bit OSs have a file size limit of 2GB. There’s nothing we can do about that.

[Tom Elliott]

@Wayne-Workman It’s requesting the filesize from the remote server using ftp. Locally it is using the filesize php command.

I’ve created an arch agnostic filesize getter which should fix this issue.

Will be in RC-6.

[Mark Shelton]

@Wayne-Workman It worked in the past. I have always used 32 bit filesystems, and the remote server is also 32 bit.

[Mark Shelton]

@Tom-Elliott Thanks Tom. I’ll wait a few days for RC-6. Regards, Mark

[Tom Elliott]

@Mark-Shelton basically, the remote was working because I get the size using a raw from look at the content. This is why the remote worked even on 32 bit remotes. However, I was using the filesize php function to get the local filesize.

The ftp essentially just reads the size as a string. So displaying it is proper as it’s not converting to an integer. The local was returning an integer.

[Mark Shelton]

@Tom-Elliott I’m so pleased you found the issue so quickly! I’ve had to turn off replication for the last number of releases and upload them when at the sites. Didn’t want to post about it until I had done some investigation and had evidence of the exact issue.

[Wayne Workman]

Fedora project isn’t making 32 bit builds anymore. What does that tell you about the next release of RHEL and CentOS? I doubt Debian would ever stop building 32 bit versions though, so Ubuntu may or may not have a future in 32 bit.

[Mark Shelton]

@Wayne-Workman Thanks, yes I may move to 64 bit. However it’s good that my issue has highlighted the inconsistency in the code which sounds like it only shows up in 32 bit OS’s. I’m guessing there mustn’t be too many others with my setup out there, otherwise this would have been reported a while ago. Glad its a relatively easy fix, I’ve rebuilt these boxes a few times recently and didn’t wish to have to do it again. Regards, Mark