SOLVED Failing to Update Database after capture of image is complete on the client

  • Hello Everyone,

    I am get a Updating Database … FAILED error message when trying to run a capture task on a client. The image is being captured successfully.

    I am running FOG version 1.5.6 with CentOS 7. Currently, I have the firewalld stopped and disabled and selinux disabled.

    I have tried the following using the troubleshooting article on the wiki:

    1. Ensure /images has the appropriate permissions
    2. Ensure the storage management, tftp server and local fogproject user have the same password
    3. Verified the filesystem where /images resides is not ext3
    4. Tried the capture task with a managed and unmanaged switch as well

    I am going through a process of elimination at this point and any help would be much appreciated.

  • Senior Developer

    @tgidwani Wow. Really glad you figured that out. I have done a lot of networking but I wouldn’t have expected the gateway setting to cause such an error. In theory the gateway information is only ever used when the host tries to connect to a machine that is not within it’s own subnet. So gateway shouldn’t play a role here as long as both were in the same subnet.

    Are you able to replicate the issue when putting the GATEWAY= line back into the config??

  • @Sebastian-Roth When I was experiencing said error the intermediary directory was never being created.

    It is working now. As mentioned in my previous reply asking about the default gateway, I think it has something to do with the Gateway configured in my network interface’s configuration file.

    Within the config file, GATEWAY= was pointing to an IP address that was not assigned to any machine on the current network when I moved networks. When I removed the GATEWAY= line from the config and ifdown and ifup my network interface, the update database error was resolved.

    Does FOS when communicating with the FOG server go through the default gateway even when both the client running FOS and the FOG server are on the same network?

  • Senior Developer

    @tgidwani Ok, lets try to focus on the following warning in the logs: ftp_put(/images/dev/ac1f6bb0690e): failed to open stream: No such file or directory

    What FOG does is it boots the FOS (FOG operating system) on the clients and mounts /images/dev directory (NFS export) to write the new captured image into /images/dev/<MAC-ADDRESS> - in your case the client MAC should be AC:1F:6B:B0:69:0E - please double check just to make sure there is no culprit in this. Now after the image capture is finished FOS calls back to the FOG server and tells it to move that newly captured image from /images/dev/<MAC-ADDRESS> to /images/<IMAGE-NAME> and this is exactly the point where things seem to fail for you.

    Thinks to check:

    • When capturing an image, do you see files being written to /images/dev/ac1f6bb0690e -> if not we need to figure out why!!
    • When the capture is finished, do you still have those files in place? They should still be there if rename/move failed. -> Then try to rename/move those manually via FTP to hopefully see why it fails. Connect via FTP as below and run: rename /images/dev/ac1f6bb0690e /images/Imagename

  • This post is deleted!

  • @Sebastian-Roth Good to know about some of the warnings. Here are the screenshots:



  • Senior Developer

    @tgidwani Looking at out code more closely I noticed that most of those ftp_... warnings are perfectly fine. FTP is not able to remove a directory that isn’t empty yet. So we do some kind of recursive delete which leads to print out all of the “ftp_delete” and “ftp_rmdir” messages which are not important. I’ll still remove those in the next release we get less confusion.

    But the “ftp_rename” and “ftp_put” warnings make me think there is something wrong with your NFS shares. Just guessing here but let’s give it a try. Please run exportfs -v as root on your FOG server and post output here. As well take a picture of the Storage Node setting in the web UI and post a picture here.

  • Senior Developer

    @tgidwani Ahhh, I see, got that wrong, sorry. I will take a look at our code again to see if I can figure out what might be causing this.

    Just to be sure, Do you still get the full list of exact same ftp_… messages in the log everytime even when SELinux is disabled/permissive?

  • @Sebastian-Roth My apologizes for the confusion. I meant I cleared them only today to eliminate the possibility of looking at stale logs entries.

    I do get new log entries every time I run a capture. I continue getting the same error every time. Update database failed with the same ftp_put failure. Would you like me to re-run the capture? Since I have not changed anything I am not why the problem would go away?

    Do you think it is a version issue? I am running 1.5.6? Does updating the database process depend upon DNS or a default gateway by chance?

  • Senior Developer

    @tgidwani said in Failing to Update Database after capture of image is complete on the client:

    I cleared all three error log files today

    Don’t you get new log entries when doing a new capture? There is no possible way we can help you without getting more evidence on what exactly is causing this. Logs are usually the best way to figure this out.

  • @Sebastian-Roth I cleared all three error log files today before posting the screenshots here. With or without SeLinux disabled I keep getting the same failed to update database error.

  • Senior Developer

    @tgidwani Alright, FTP is not an issue I am fairly sure now. Seems like I was miss-lead by the messages we saw in the php-fpm log file. Those probably came from the time before you disabled SELinux… I reckon.

    So do you still see it failing on database update at the end of the task?

    Please check the log files again but this time only take a look at the last 10 lines: tail -10 /var/log/php-fpm/www-error.log

  • @Sebastian-Roth Here is the screenshot:


  • Senior Developer

    @tgidwani Seems like we need to work this out manually. Use a ftp command line utility (Windows or Linux) and connect to the FOG server using the credentials you see in /var/www/html/fog/lib/fog/config.class.php.

    Try connecting, then create and rename a directory just to see if things work like expected:

    shell> ftp
    Connected to
    220 (vsFTPd 3.0.3)
    Name ( fogproject
    331 Please specify the password.
    230 Login successful.
    Remote system type is UNIX.
    Using binary mode to transfer files.
    ftp> pwd
    257 "/home/fogproject" is the current directory
    ftp> cd /images/dev
    250 Directory successfully changed.
    ftp> mkdir test12345
    257 "/images/dev/test12345" created
    ftp> rename test12345 12345test
    350 Ready for RNTO.
    250 Rename successful.
    ftp> rm 12345test
    250 Remove directory operation successful.
    ftp> quit
    221 Goodbye.

  • @Sebastian-Roth I followed all the steps in the mentioned article when setting up my CentOS 7 VM and before installing FOG.

    Here is the screenshot:


    When I got the update database failure, I disabled and stopped firewalld and set selinux=disabled in /etc/selinux/config and restarted my CentOS 7 VM. Re-ran the capture process but still continued to get the same failure of updating database.

  • Senior Developer

    @tgidwani said in Failing to Update Database after capture of image is complete on the client:

    with CentOS 7

    Why didn’t I see this any earlier?!??!

    Please run command getenforce on your FOG server and post output here.

    Sure you wanna read this:

  • @Sebastian-Roth Here is the screenshot:


  • Senior Developer

    @tgidwani Please as well run grep USERNAME /var/www/html/fog/lib/fog/config.class.php and post results here.

  • @Sebastian-Roth Appreciate your help and quick response. Here is the screenshot:


  • Senior Developer

    @tgidwani Please run this command on your FOG server and post output here: ls -alRt /images/