Imaging Jobs Freezing



  • Server
    • FOG Version: 1.4.2
    • OS: Ubuntu Server 16.04.2 LTS
    Description

    Recently, when I image computers the imaging job will randomly hang or freeze. The times on the client all stop as well as on the server. No errors or warnings are displayed nor seen on the server’s log files. The GUI is still usable and the server is still pingable. The only errors I can find are in the Apache Error Log on the FOG GUI. Below is what is present:

    [Mon Jun 05 10:01:46.164311 2017] [php7:warn] [pid 4300] [client 192.168.150.19:52636] PHP Warning: file_get_contents(/sys/class/net/bonding_masters/operstate): failed to open stream: No such file or directory in /var/www/fog/status/bandwidth.php on line 82

    I was running version 1.4.0 when the issue started and I have since upgraded to 1.4.2 which did not resolve the issues. Has anyone ever seen this before?

    Thanks,

    Anthony



  • @Tom-Elliott
    @george1421 after updating to the latest version the deployments to the NCR still fail. I am able to image PCs and laptops so I will just use a different solution for the NCRs and leave it at that. Thanks everyone for all of your help and assistance!

    Thanks,

    Anthony



  • @Tom-Elliott
    @george1421 Thank you for clarifying. I am still getting the same error message when I try to update the client. Is there some where else this password could be messed? I did install the latest FOG update, so I am at 1.4.4. I will try a deployment and let you know if I have any success.

    Thanks,

    Anthony


  • Senior Developer

    The settings under TFTP Server is from the days of PXE Boot (plain and simple). These were used to define the TFTP Server and the fog username/password were used to upload the latest pxe file when tasking a client machine. This, then, was also used for updating the kernels. I just don’t have a logical way to achieve a more dynamic means of updating these things so these “settings” are now used during updating kernels.


  • Moderator

    @atarone Well my last post was way off base, you are not capturing an image but just trying to update the client settings.

    There is another place the fog user account info is hidden in the fog settings under tftp server. I can’t say for absolute why this is there. I’m not saying that is your issue either. Its just strange why you have your root issue. I don’t think its related to updating the client settings.



  • @george1421 I confirmed the passwords and they do match. I can connect via a Windows FTP client, but when I run the update I still get the error listed below.

    Thanks,

    Anthony


  • Moderator

    @atarone This message is typically generated during an upload if/when someone mucks about with the linux user called fog. This is a service account that is owned by the FOG backend and should not be used for general system administration. If you happened to change this password or the installer got this account out of sync you will see the above error. I’m not saying that is your case, because what you have is a bit unique.

    But I would start by inspecting the /opt/fog/.fogsettings file for the password, then make sure in the web gui that the storage node password matches. If they all do then connect to the fog server using a windows box ftp client. Use the user ID (fog) and password found in the .fogsettings file. Confirm that you can login.



  • @Sebastian-Roth
    @Tom-Elliott

    Yes before it freezes I am not able to switch. On another interesting note, I tried updating to the latest client and i get the error message below:

    0_1498572351557_FOG ScreenShot.jpg

    Could this be related?

    Thanks,

    Anthony


  • Developer

    @atarone Before or after it freezes??



  • @Sebastian-Roth Correct. I am not able to switch when using CTRL-ALT-Fx.


  • Developer

    @atarone You mean you aren’t able to switch even when the imaging did not freeze yet? Yeah, Ctrl+Alt+Fx is the key…



  • @Sebastian-Roth Thanks! What is the key combination to switch VTY lines? I am unable to switch them using CTRL-ALT-Fx or any other combination of those keys.

    Thanks!


  • Developer

    @Tom-Elliott You are absolutely right about SSH being the more advanced method to get access to such a client. But in this case when network connection is lost or it actually freezes it’s quite handy I suppose. On the other hand, I see that we never ever had such a case yet. So maybe just leave it.


  • Senior Developer

    @Sebastian-Roth I don’t think it would. I think it’s just the access to those terminals can be rather limited, which is all the more reason I added the openssh utils. Anybody can remote in much easier than have a device that’s having issues right next to them the whole time.

    Using the openssh elements of it all allow us (devs and what not) to remote in and ssh in to see the machine too.

    Pair the postinit scripts with a means to associate the root password and you don’t even, fully, need debug mode to test things (though I’ll admit you’d be strained for time to get information).


  • Developer

    @atarone Yes, no problem. Find a fresh version of both 32 bit and 64 bit in the same place.

    @Tom-Elliott What do you think about adding a virtual terminal to the official initrds? Would this use too much resources on the clients for no reason?



  • @Sebastian-Roth Do you have a VT2 version for init_32.xe? I can only use the NCR and they are 32 bit.


  • Developer

    As well, could you please try this: Boot the client into deploy task using the new init.xz as normal. As FOG starts to prepare the disk for imaging (before the blue partclone screen) switch to VT2 (Ctrl+Alt+F2) and run this command: tail -f /var/log/messages. Just let it sit there. You should see all (kernel) messages coming in. Maybe this will give us a hint on what’s causing the hang. Please take a picture and upload here.

    Unfortunately you can’t see when it freezes while you are in VT2 but you can run a ping from another machine to check if the client is still alive…


  • Developer

    @atarone said in Imaging Jobs Freezing:

    I am unable to switch between the VTYs.

    So the client really seems to fully freeze. What if you hit caps lock by the way. Does the LED on the keyboard change state when it hangs? Just want to make sure…



  • @Sebastian-Roth “I think I have found part of it given this output. Please let me know.” I though that last output capture being blank may have been a problem, but you said all looks good to you. I am unable to switch between the VTYs.

    Thanks,

    Anthony


  • Developer

    @atarone said in Imaging Jobs Freezing:

    The deployments started freezing again with the new init.xz and the old one.

    Are you able to switch between virtual terminal one and two (as described earlier) when deployment freezes?

    The numbers in the outputs you posted look pretty ok to me. I can’t see where things are going wrong here yet.

    I think I have found part of it given this output. Please let me know.

    What do you mean by that?


 

404
Online

5.4k
Users

12.6k
Topics

118.9k
Posts