@Sebastian-Roth and @george1421
Started the debug task and it’s stuck at loading the init.

@Sebastian-Roth said in Surface Go 2 (microsoft surface usb-c ethernet adapter) won't boot bzImage on fog 1.5.9.2 and Getting database update errors:

find /var/www/html/fog -type f -exec grep “Machine Details” {} /dev/null ;

 find /var/www/html/fog -type f -exec grep "Machine Details" {} /dev/null \;
/var/www/html/fog/management/languages/zh_CN.UTF-8/LC_MESSAGES/messages.po:msgid "Machine Details"
/var/www/html/fog/management/languages/pt_BR.UTF-8/LC_MESSAGES/messages.po:msgid "Machine Details"
/var/www/html/fog/management/languages/it_IT.UTF-8/LC_MESSAGES/messages.po:msgid "Machine Details"
/var/www/html/fog/management/languages/es_ES.UTF-8/LC_MESSAGES/messages.po:msgid "Machine Details"
/var/www/html/fog/management/languages/fr_FR.UTF-8/LC_MESSAGES/messages.po:msgid "Machine Details"
/var/www/html/fog/management/languages/de_DE.UTF-8/LC_MESSAGES/messages.po:msgid "Machine Details"
/var/www/html/fog/management/languages/en_US.UTF-8/LC_MESSAGES/messages.po:msgid "Machine Details"
/var/www/html/fog/management/languages/messages.pot:msgid "Machine Details"
/var/www/html/fog/lib/reg-task/taskqueue.class.php:            sprintf("%s:-\n", _('Machine Details')) => '',

@Sebastian-Roth I don’t have any custom scripts at the fog level.
I also don’t think this is an issue with the surface device. I need to image another machine today and see if it’s happening everywhere or just on this one machine.
I’m gonna run a debug deploy on the surface this morning and a fresh image on another new machine.

@george1421 This was my first deployment since upgrading to fog 1.5.9 from 1.5.8 (technically 1.5.9.2 from 1.5.9 RC 2 I think)

I did switch to the 5.6.18 kernel yesterday and that got things through the imaging as well, but still had the database error. I’m working from home today so I can’t do any testing on the device till tomorrow, when I will gladly give a debug a try. I’m also wondering if there’s a matching init for the 5.6.18 kernel?

I can pull up logs and check the database and such today if it would help.

I double checked my sql settings in the .fogsettings file and all was correct.
Not sure what’s going on now with the database update error

I just upgraded to fog 1.5.9 (technically 1.5.9.2 from the dev-branch)
I went to image a new surface go and it got stuck on bzImage. I tried updating the bzImage to the latest (that was probably the one included 4.19.143) and there was no change.

I reverted to 4.18.11 (it was the closest to 4.18.6 I saw in the official kernels, which is what I looked for after reading this post https://forums.fogproject.org/topic/12465/microsoft-surface-go-usb-c-to-ethernet-adapter-compatibility/5 probably didn’t need to go that far back )

So maybe we took out the microsoft driver in the latest update? These were imaging fine in 1.5.8

Wait, now I’m getting an error after the image finished…

So I didn’t find any clear documentation on how this is done. So we probably need to add it to the wiki @Wayne-Workman

Probably to here https://wiki.fogproject.org/wiki/index.php?title=Managing_FOG#Setting_up_Printers_With_Fog_Printer_Management

even something as simple as

updating this section

To create a new printer definition click on the Printer icon on the system menu bar. Then on the left hand menu, click on Add New Printer. The form you are presented with will require you to enter:

Printer Model - This must match the name in the INF file.
Printer Alias - This can be anything you wish and it is what the end user will see.
Printer Port - This is something like LPT1:, or IP_1.1.1.2.
Printer INF File - This is the path to the INF file for the printer driver.
Printer IP - (optional) This is ip address of an IP based printers only, this can take the form of 1.2.3.4:9100 or 1.2.4.5. If the port doesn’t exist already, it will create one named IP_x.x.x.x, where x.x.x.x is the ip address. That is what should be entered in the port field.

After all the required information is entered, click on the Add Printer button.

With this under the Printer IP definition

Printer Config File - (Optional) This is the local or remote path to a .dat file that will be imported to set the printers configurations (i.e. tray count, add-on modules, model specific settings, etc.). You can create this file by

1. manually configuring a printer via the printer properties gui
2. running this command on the same computer RUNDLL32 PRINTUI.DLL,PrintUIEntry /Ss /n"Printer Name" /a "C:\Path\To\Save\ConfigFile.dat m f g p

Of course if someone wants to create the evolution of the printer manager I suggested, it would have to be re-documented. But people would still need to know how to create the config file.

TL;DR

Try putting the .dat file the same place you put your inf driver file for the printer and reference them on the same path. i.e. either a local path on the client computer or a publicly accessible file share.

As I mentioned though, I don’t use the fog print manager anymore. So I don’t have a way to test it currently.

Found some code, sorta remember how it works

Well I found the pull request commit on the fog client github from when @Joe-Schmitt put in the code I wrote for adding the config file https://github.com/FOGProject/fog-client/commit/57af21cdbbe79a9bd403056b1ea88f88af722ecf.
Both the posts I referenced earlier were for before this code was added, so they probably won’t be any help in figuring out where that config file is supposed to go.

And then I found where that code is now, on line 124 of this file fog-client https://github.com/FOGProject/fog-client/blob/master/Modules/PrinterManager/Windows/WindowsPrinterManager.cs,
which looks like this

public override void Configure(Printer printer, bool verbose = false)
        {
            if (printer.Type == Printer.PrinterType.Network)
            {
                Log.Entry(LogName, $"Invoking add {printer.Name} for all users");
                PrintUI($"/in /n \"{printer.Name}\"", verbose);
            }

            if (string.IsNullOrEmpty(printer.ConfigFile)) return;

            Log.Entry(LogName, "Configuring " + printer.Name);
            PrintUI($"/Sr /n \"{printer.Name}\" /a \"{printer.ConfigFile}\" m f g p", verbose);
        }

As far as I can tell/remember from the code that it takes the given path and puts it into the command, which is the csharp version of the one you already know. So, I believe, it’s taking whatever you put in that field as a string and putting it in the command.

So if you put C:\printerConfig.dat in your config field it would run RUNDLL32 PRINTUI.DLL,PrintUIEntry /Sr /n"Printer Name" /a "C:\printerConfig.dat" m f g p after it adds the printer, which should import your config.

So it could either be a file on a share the system account (that runs the fogservice) has access to, or a file that is already local on the client computer. So I think it works the same way the inf does where the file just needs to be somewhere that is helpful.

I feel like we (or probably @Joe-Schmitt ) also made a helper tool for creating the dat files, and I see a folder in the code called PrinterManagerHelper here https://github.com/FOGProject/fog-client/tree/master/PrinterManagerHelper but I don’t remember how that works.

Ways to deploy printers

To deploy it a bit more smoothly you might…

Print Share

Consider creating a public samba share (or at least give it permissions for system/machine accounts) for sharing an organized structure of print drivers and config files. Then point each printer field as something like \\printshare\printerName\printerName-config.dat \\printShare\printerName\printerName-driver.inf

Snapin

You could also create a simple snapin for each printer, that copies the driver and dat files to some folder that works with your company’s organization. Like a C:\ProgramData\FogPrinters folder or something like that. And then maybe making a scheduled task or another snapin to delete them after fog adds the printers (no need to take up space on client computers with config files)

Feature Request

Maybe, if it’s desired by people, we could add functionality to copy files from the snapins folder (or a new fogserver folder) to a temporary folder while the printer gets installed and then deletes them from the temp folder. Once the printer is added I believe the driver gets put in the system driver store and the config file is no longer needed as its been imported. That would make the print functions of fog a bit more integrated and you could just manage the files in fog like you do with snapins.

But this would require

• Creating a new server folder
  • at least a subfolder in the existing snapins, i.e /opt/fog/snapins/printers or even just make a /opt/fog/printers.
• A modification to the printer gui page
  • Instead of specifying a path to the inf and config file, browse buttons and use existing dropdowns like the snapins have
• A code change to the printer manager
  • make it work like the snapins code. where it copies down the files from the snapins folder, runs it, then deletes it. Just make it be from a printer folder that works like the snapins.
• Probably something else too I’m not thinking of. But if lots of people use the printer manager, could be a useful tool.

@Jonathan-Cool So I found my old post where I mentioned those exact commands you put in your original post
https://forums.fogproject.org/topic/6540/adding-custom-printer-configuration plus it has a bat script (which is a language I don’t even use anymore since I moved everything to powershell)

So it is the .dat file that you want but where it resides, I’m trying to find that info.

I was a bit long winded when I tried to help someone else with something related here https://forums.fogproject.org/topic/6615/printer-problems/5

I am sure I made some sort of documentation post somewhere though just gotta find it and then make sure it gets put somewhere more findable

@Jonathan-Cool So I helped a lot with that field’s existence. I actually don’t use that feature anymore since I switched our infrastructure to just use a print server to deploy printer connections and settings (point being, I don’t actually remember how to use it…). But I’m pretty sure I have an old forum post explaining how to create a setting file from a configured printer. Standby, I’ll see if I can find it.

@Sebastian-Roth
This is what I’m doing to fix the problem via the api, which seems to work most of the time. A couple times either the fogservice or my code caused the host to stop showing up in the api/gui list of hosts but the host would still be in the database. When that happens I have to reset the host encryption fields and the pending field in the database hosts table for each affected host and then restart the fog service on the host and it shows back up again.

below is powershell code

$mac = get-activeMacAddress; #function that filters output of get-netadapter to the currently active adapter's mac
$hostObj = Get-FogHost -macAddr $mac; #function from fogapi module can be found here https://github.com/darksidemilk/FogApi/blob/master/FogApi/Public/Get-FogHost.ps1
if ($hostObj.pending -ne '0') { 
                            Write-Warning "The host is pending or not explicitly set to not pending in fog, adjust host to be approved";
                            Reset-HostEncryption -fogHost $hostObj; #function from fogapi module that resets the encryption fields of a host see https://github.com/darksidemilk/FogApi/blob/master/FogApi/Public/Reset-HostEncryption.ps1
                            Set-FogOU -hostObj $hostObj -force; #custom internal function, will share applicable contents below
                        }

this is what Set-FogOU does

function Set-FogOU {
<#
    .SYNOPSIS
    Set the proper OU on the fog host based on the hostname and group membership
    
    .DESCRIPTION
    Gets the current fog host infomration and then uses get-oustr to get
    the OU it should be in based on group membership and the hostname
    
    .PARAMETER jsonData
    The jsondata string used to set the ou with the fog api
    
    .PARAMETER hostObj
    the hostobject of the foghost gotten from get-foghost
    
    .PARAMETER OUstr
    The optional prewritten fogstr if you want to provide it
    
#>
    
    [CmdletBinding()]
    param (
        [string]$jsonData,
        $hostObj,
        [string]$OUstr,
        $progressID=1,
        $parentID=-1,
        [switch]$force
    )

    process {
        if($null -eq $hostObj) {
            Write-Verbose 'no host given, getting current host...';
            $mac = Get-ActiveMacAddress
            $hostObj = Get-FogHost -macAddr $mac
        }
        Write-Verbose 'getting fog group...';
        $group = Get-FogGroup $hostObj.id; #function from fogapi see https://github.com/darksidemilk/FogApi/blob/master/FogApi/Public/Get-FogGroup.ps1
        $OUstr = Get-OUStr -group $group -compName $hostObj.Name; #function that creates correct OUstring from the group name
            
            $ADPass = ""; #edited for privacy, this is gotten with powershell pscredential encryption methods
            Write-Verbose "OU string is $OUstr, setting to properties of host object, also enabling and enforcing domain join...";
            $hostObj.ADDomain = "domain"; #edited for privacy
            $hostObj.ADUser = "domainUser"; #edited for privacy
            $hostObj.ADPass = $ADPass;
            $hostObj.ADOU = $OUstr;
            $hostObj.useAD = 1;
            $hostObj.enforce = 1;
            $hostObj.pending = '0'; #made it a string instead of an int as it seemed to make a difference
            Write-Verbose 'creating json string from new host ou settings...';
            $jsonData = $hostObj | Select-Object id,ADDomain,ADUser,ADPass,ADOU,useAD,enforce,pending | ConvertTo-Json;
            Update-FogObject -type object -coreObject host -IDofObject $hostObj.id -jsonData $jsonData; #fogapi function to run the update/set api command     
    }

Basically just doing a PUT api command on the host object to reset the encryption fields and then another one to set the pending field to '0' and to put all the OU settings back. The json created by the command defining $jsonData looks like this (slightly edited for privacy)

{
    "id":  "1593",
    "ADDomain":  "domain",
    "ADUser":  "domainUser",
    "ADPass":  "secretPassword",
    "ADOU":  "OU=IT,OU=Workstations,DC=domainName,DC=com",
    "useAD":  1,
    "enforce":  1,
    "pending":  "0"
}

@Sebastian-Roth I am also getting hosts becoming pending for no apparent reason. My best guess at the moment is that it’s related to the service as it only seems to happen when the service is running. I’d rather like to keep it running.

@Sebastian-Roth I saw it happen again this morning. In at least one case (I have scenarios where it’s happening that this doesn’t match up with) it seems to happen right after a snapin hash doesn’t match.
I attached the fog.log section where the host went through Authenticated -> start snapins ->Snapin hash mismatch -> Invalid host

How my hashes keep getting mismatches is a whole other question because I use the exact same file for every snapin. And said file hasn’t changed in 2 years.

------------------------------------------------------------------------------
--------------------------------Authentication--------------------------------
------------------------------------------------------------------------------
 8/25/2020 7:12:24 AM Client-Info Version: 0.12.0
 8/25/2020 7:12:24 AM Client-Info OS:      Windows
 8/25/2020 7:12:24 AM Middleware::Authentication Waiting for authentication timeout to pass
 8/25/2020 7:12:24 AM Middleware::Communication Download: http://fogserver/fog/management/other/ssl/srvpublic.crt
 8/25/2020 7:12:24 AM Data::RSA FOG Server CA cert found
 8/25/2020 7:12:24 AM Middleware::Authentication Cert OK
 8/25/2020 7:12:24 AM Middleware::Communication POST URL: http://fogserver/fog/management/index.php?sub=requestClientInfo&authorize&newService
 8/25/2020 7:12:24 AM Middleware::Response Success
 8/25/2020 7:12:24 AM Middleware::Authentication Authenticated


 8/25/2020 7:12:24 AM Middleware::Communication URL: http://fogserver/fog/management/index.php?sub=requestClientInfo&configure&newService&json
 8/25/2020 7:12:24 AM Middleware::Response Success
 8/25/2020 7:12:24 AM Middleware::Communication URL: http://fogserver/fog/management/index.php?sub=requestClientInfo&mac=9C:7B:EF:C3:90:84||00:00:00:00:00:00:00:E0&newService&json
 8/25/2020 7:12:25 AM Middleware::Response Success
 8/25/2020 7:12:25 AM Middleware::Communication URL: http://fogserver/fog/service/getversion.php?clientver&newService&json
 8/25/2020 7:12:25 AM Middleware::Communication URL: http://fogserver/fog/service/getversion.php?newService&json

 8/25/2020 7:12:25 AM Service Creating user agent cache
 8/25/2020 7:12:25 AM Middleware::Response Module is disabled globally on the FOG server
 8/25/2020 7:12:25 AM Middleware::Response Module is disabled globally on the FOG server
 8/25/2020 7:12:25 AM Middleware::Response Module is disabled globally on the FOG server
 8/25/2020 7:12:25 AM Service Initializing modules

------------------------------------------------------------------------------
---------------------------------ClientUpdater--------------------------------
------------------------------------------------------------------------------
 8/25/2020 7:12:25 AM Client-Info Client Version: 0.12.0
 8/25/2020 7:12:25 AM Client-Info Client OS:      Windows
 8/25/2020 7:12:25 AM Client-Info Server Version: 1.5.9-RC2
 8/25/2020 7:12:25 AM Middleware::Response Success
------------------------------------------------------------------------------


------------------------------------------------------------------------------
----------------------------------TaskReboot----------------------------------
------------------------------------------------------------------------------
 8/25/2020 7:12:25 AM Client-Info Client Version: 0.12.0
 8/25/2020 7:12:25 AM Client-Info Client OS:      Windows
 8/25/2020 7:12:25 AM Client-Info Server Version: 1.5.9-RC2
 8/25/2020 7:12:25 AM Middleware::Response Success
------------------------------------------------------------------------------


------------------------------------------------------------------------------
--------------------------------HostnameChanger-------------------------------
------------------------------------------------------------------------------
 8/25/2020 7:12:25 AM Client-Info Client Version: 0.12.0
 8/25/2020 7:12:25 AM Client-Info Client OS:      Windows
 8/25/2020 7:12:25 AM Client-Info Server Version: 1.5.9-RC2
 8/25/2020 7:12:25 AM Middleware::Response Success
 8/25/2020 7:12:25 AM HostnameChanger Checking Hostname
 8/25/2020 7:12:25 AM HostnameChanger Hostname is correct
 8/25/2020 7:12:25 AM HostnameChanger Attempting to join domain
 8/25/2020 7:12:25 AM HostnameChanger Host already joined to target domain
------------------------------------------------------------------------------


------------------------------------------------------------------------------
---------------------------------SnapinClient---------------------------------
------------------------------------------------------------------------------
 8/25/2020 7:12:25 AM Client-Info Client Version: 0.12.0
 8/25/2020 7:12:25 AM Client-Info Client OS:      Windows
 8/25/2020 7:12:25 AM Client-Info Server Version: 1.5.9-RC2
 8/25/2020 7:12:25 AM Middleware::Response Success
 8/25/2020 7:12:25 AM SnapinClient Running snapin DeviceForm - Desktop
 8/25/2020 7:12:25 AM Middleware::Communication Download: http://192.168.100.117//fog/service/snapins.file.php?mac=9C:7B:EF:C3:90:84||00:00:00:00:00:00:00:E0&taskid=5462
 8/25/2020 7:12:26 AM SnapinClient C:\Program Files (x86)\FOG\tmp\chocoPkgSnapin.ps1
 8/25/2020 7:12:26 AM Bus Emmiting message on channel: Notification
 8/25/2020 7:12:26 AM SnapinClient Starting snapin
 8/25/2020 7:13:00 AM SnapinClient Snapin finished
 8/25/2020 7:13:00 AM SnapinClient Return Code: 0
 8/25/2020 7:13:00 AM Bus Emmiting message on channel: Notification
 8/25/2020 7:13:00 AM Middleware::Communication URL: http://fogserver/fog/service/snapins.checkin.php?taskid=5462&exitcode=0&mac=9C:7B:EF:C3:90:84||00:00:00:00:00:00:00:E0&newService&json
 8/25/2020 7:13:00 AM SnapinClient Running snapin mitel-pm
 8/25/2020 7:13:00 AM Middleware::Communication Download: http://192.168.100.117//fog/service/snapins.file.php?mac=9C:7B:EF:C3:90:84||00:00:00:00:00:00:00:E0&taskid=5475
 8/25/2020 7:13:01 AM SnapinClient C:\Program Files (x86)\FOG\tmp\chocoPkgSnapin.ps1
 8/25/2020 7:13:01 AM Bus Emmiting message on channel: Notification
 8/25/2020 7:13:01 AM SnapinClient Starting snapin
 8/25/2020 7:14:35 AM SnapinClient Snapin finished
 8/25/2020 7:14:35 AM SnapinClient Return Code: 0
 8/25/2020 7:14:35 AM Bus Emmiting message on channel: Notification
 8/25/2020 7:14:35 AM Middleware::Communication URL: http://fogserver/fog/service/snapins.checkin.php?taskid=5475&exitcode=0&mac=9C:7B:EF:C3:90:84||00:00:00:00:00:00:00:E0&newService&json
 8/25/2020 7:14:35 AM SnapinClient Running snapin office2010-Std
 8/25/2020 7:14:35 AM Middleware::Communication Download: http://192.168.100.117//fog/service/snapins.file.php?mac=9C:7B:EF:C3:90:84||00:00:00:00:00:00:00:E0&taskid=5476
 8/25/2020 7:14:36 AM SnapinClient C:\Program Files (x86)\FOG\tmp\chocoPkgSnapin.ps1
 8/25/2020 7:14:36 AM Bus Emmiting message on channel: Notification
 8/25/2020 7:14:36 AM SnapinClient Starting snapin
 8/25/2020 7:22:15 AM SnapinClient Snapin finished
 8/25/2020 7:22:15 AM SnapinClient Return Code: 0
 8/25/2020 7:22:15 AM Bus Emmiting message on channel: Notification
 8/25/2020 7:22:15 AM Middleware::Communication URL: http://fogserver/fog/service/snapins.checkin.php?taskid=5476&exitcode=0&mac=9C:7B:EF:C3:90:84||00:00:00:00:00:00:00:E0&newService&json
 8/25/2020 7:22:15 AM SnapinClient Running snapin outlook2016
 8/25/2020 7:22:15 AM Middleware::Communication Download: http://192.168.100.117//fog/service/snapins.file.php?mac=9C:7B:EF:C3:90:84||00:00:00:00:00:00:00:E0&taskid=5477
 8/25/2020 7:22:16 AM SnapinClient C:\Program Files (x86)\FOG\tmp\chocoPkgSnapin.ps1
 8/25/2020 7:22:16 AM SnapinClient ERROR: Hash does not match
 8/25/2020 7:22:16 AM SnapinClient ERROR: --> Ideal: 817B881A4D07DCFF51F67A47D9595DDE59D010FE0C018BDF5B57C502CB83A14F3085AF48987B392E8BF38E2C6B29D094055BDFFB6BBCD5647697D64AA4BDD668
 8/25/2020 7:22:16 AM SnapinClient ERROR: --> Actual: 5271D1B9F35EA9FC2F1E3CE474C20641A15B37A02035E7E8210DFB513518CBC79FDFD25092A41491C2AB02AF8E0913C99819C6066B9883F398D90C5D045E4A1D
 8/25/2020 7:22:16 AM Middleware::Communication URL: http://fogserver/fog/service/snapins.checkin.php?taskid=5477&exitcode=-1&mac=9C:7B:EF:C3:90:84||00:00:00:00:00:00:00:E0&newService&json
------------------------------------------------------------------------------


------------------------------------------------------------------------------
--------------------------------PrinterManager--------------------------------
------------------------------------------------------------------------------
 8/25/2020 7:22:16 AM Client-Info Client Version: 0.12.0
 8/25/2020 7:22:16 AM Client-Info Client OS:      Windows
 8/25/2020 7:22:16 AM Client-Info Server Version: 1.5.9-RC2
 8/25/2020 7:22:16 AM Middleware::Response Module is disabled globally on the FOG server
------------------------------------------------------------------------------


------------------------------------------------------------------------------
--------------------------------PowerManagement-------------------------------
------------------------------------------------------------------------------
 8/25/2020 7:22:16 AM Client-Info Client Version: 0.12.0
 8/25/2020 7:22:16 AM Client-Info Client OS:      Windows
 8/25/2020 7:22:16 AM Client-Info Server Version: 1.5.9-RC2
 8/25/2020 7:22:16 AM Middleware::Response Success
 8/25/2020 7:22:16 AM PowerManagement Calculating tasks to unschedule
 8/25/2020 7:22:16 AM PowerManagement Calculating tasks to schedule
------------------------------------------------------------------------------


------------------------------------------------------------------------------
----------------------------------UserTracker---------------------------------
------------------------------------------------------------------------------
 8/25/2020 7:22:16 AM Client-Info Client Version: 0.12.0
 8/25/2020 7:22:16 AM Client-Info Client OS:      Windows
 8/25/2020 7:22:16 AM Client-Info Server Version: 1.5.9-RC2
 8/25/2020 7:22:16 AM Middleware::Response Success
 8/25/2020 7:22:16 AM Middleware::Communication URL: http://fogserver/fog/service/usertracking.report.php?action=login&user=SHP-WKS-6096\AdlAdmin&mac=9C:7B:EF:C3:90:84||00:00:00:00:00:00:00:E0&newService&json
------------------------------------------------------------------------------

 8/25/2020 7:22:16 AM Service Sleeping for 94 seconds
 8/25/2020 7:23:50 AM Middleware::Communication URL: http://fogserver/fog/management/index.php?sub=requestClientInfo&configure&newService&json
 8/25/2020 7:23:50 AM Middleware::Response Success
 8/25/2020 7:23:50 AM Middleware::Communication URL: http://fogserver/fog/management/index.php?sub=requestClientInfo&mac=9C:7B:EF:C3:90:84||00:00:00:00:00:00:00:E0&newService&json
 8/25/2020 7:23:50 AM Middleware::Response Invalid host
 8/25/2020 7:23:50 AM Middleware::Communication URL: http://fogserver/fog/service/getversion.php?clientver&newService&json
 8/25/2020 7:23:51 AM Middleware::Communication URL: http://fogserver/fog/service/getversion.php?newService&json

 8/25/2020 7:23:51 AM Service Creating user agent cache
 8/25/2020 7:23:51 AM Middleware::Response Module is disabled globally on the FOG server
 8/25/2020 7:23:51 AM Middleware::Response Module is disabled globally on the FOG server
 8/25/2020 7:23:51 AM Middleware::Response Module is disabled globally on the FOG server

@Sebastian-Roth I found that there was some need for database cleanup as I had some invalid entries. I’ll paste in my internal notes on what I did. Sadly I am still getting random pending hosts. I think that maybe something in the new version of the fog client, 0.12.0 I believe, is making it so some hosts that may have previously had some sort of invalid data in the database making them not viewable are becoming pending and then fixed. Though some of them were valid beforehand. So I still don’t know the cause of the behavior but I at least made some api commands to workaround the issue when it happens during deployment. I Also found that there were int, string, and null data types in my hostPending field. i.e. some hosts had a blank/null field, some matched a where hostPending = '1' some match hostPending = 1 some matched hostPending = 0 s and some matched hostPending = '0'
After testing an approval of a host in the gui, I manually set all hosts in the database to hostPending = '0' to add universal. I tried to find the code in the fog project where hosts are set to pending/approved but don’t think I found every place. It did look like when they are added via pxe (the method I usually use) it sets the pending to $null. But I was having trouble following it all perfectly. I’d be interested in seeing everything that determines whether a host should become pending or not if you’re willing to help me find all the code where it happens.

in the actual database there are 26 more entries than what we see in the gui or api.
The hostPending field in the database seems to have multiple ways of being set. some are 0 some are ‘0’ some are 1 and some are ‘1’
There must be something else that makes it show up or not in the gui or api as pending or at all.
Currenty seeing 7 hosts with pending of ‘1’ but only one of them shows up in pending hosts/ in the gui/api at all.

MariaDB [fog]> select hostID,hostname,hostPending,hostCreateDate,hostLastDeploy,hostImage from hosts where hostPending in ('1');
+--------+-----------------+-------------+---------------------+---------------------+-----------+
| hostID | hostname        | hostPending | hostCreateDate      | hostLastDeploy      | hostImage |
+--------+-----------------+-------------+---------------------+---------------------+-----------+
|   1519 | DESKTOP-G4VN9T5 | 1           | 2016-11-11 10:22:53 | 0000-00-00 00:00:00 |         0 |
|   1710 | ARROWHE-AS1BQVC | 1           | 2019-11-14 15:51:54 | 0000-00-00 00:00:00 |         0 |
|   1719 | SCH-Wks-6077    | 1           | 2020-08-04 21:10:17 | 0000-00-00 00:00:00 |         0 |
|   1715 | mss-dcamTmill2  | 1           | 2020-08-19 07:44:06 | 0000-00-00 00:00:00 |         0 |
|   1729 | MD-Bcode-Test   | 1           | 2020-08-11 20:18:17 | 0000-00-00 00:00:00 |         0 |
|   1721 | MSS-LTS-6080    | 1           | 2020-08-07 09:11:26 | 0000-00-00 00:00:00 |         0 |
|   1725 | imp-wks-6085    | 1           | 2020-08-20 08:51:21 | 0000-00-00 00:00:00 |         0 |
+--------+-----------------+-------------+---------------------+---------------------+-----------+
7 rows in set (0.00 sec) 

ran

update hosts set hostPending = '0'; 

Which updated all 286 rows, but no change on the api number of hosts but the pending hosts list link has now disappeared on the gui. Going to set all hosts that have no image id set to the stable image and see if that changes anything

Ran

update hosts set hostImage = 27 where hostImage != 29; 

to set all hosts that aren’t set to the dev image to the stable image

The api is still showing 257 hosts, the database shows 283 rows in the hosts table, and then in the fog gui there are 274 members of the everyone group that was just checked and found to indeed have all possible hosts in it.

did some clean up on groups that are no longer used and deleted hosts within that couldn’t be brought up in the host view. Then emptied the everyone group and re-added all available hosts. The everyone group count and api group count are now the same but the database is still showing 283 hosts.

ran these commands to clean up the database of stale records

DELETE FROM `hosts` WHERE `hostID` NOT IN (SELECT `hmhostID` FROM `hostMAC`);
DELETE FROM `hostMAC` WHERE `hmhostID` NOT IN (SELECT `hostID` FROM `hosts`);
DELETE FROM `snapinAssoc` WHERE `saHostID` NOT IN (SELECT `hostID` FROM `hosts`);
DELETE FROM `groupMembers` WHERE `gmHostID` NOT IN (SELECT `hostID` FROM `hosts`); 

Got the code from this post when searching for forum posts concerning having a hostID of 0 https://forums.fogproject.org/topic/8359/fog-registration-error-hostname-with-that-name-already-exists/20

Database now shows 265 entries and api shows 256

used some set-clipboard and -join “,” magic to create a csv list to paste into the mariadb console and search for all hosts that are not being found in the api. Found that one of the hosts is my tablet and found that the fog log showed it as ‘not encrypted’. ran the following to do a database level client encryption reset on the 9 missing hosts

update hosts set hostsectoken="",hostpubkey="",hostsectime ="0000-00-00 00:00:00" where hostid not in (83,1645,1657,1500,1639,138,78,77,1596,1647,1469,1531,1604,1513,1605,1634,147,1543,1467,1607,1606,1466,1512,1490,1495,1631,109,1665,101,1534,1533,1659,1562,1660,1633,1628,154,1541,16,1552,82,1551,1706,1708,1714,151,156,152,157,150,1470,1574,1062,107,1679,76,72,85,137,1640,1520,86,1581,80,1540,1594,1676,130,1501,1526,1535,1545,1559,1590,1616,1627,1626,1527,1601,1686,1693,91,1644,1672,358,141,100,1553,302,1477,1479,1667,1646,93,19,92,1661,1670,155,1489,1664,142,124,1692,127,1554,1673,1724,1725,1598,24,1690,1578,1585,1705,1663,136,1656,1701,1577,1557,1736,1593,1674,103,22,1503,66,95,1652,1472,1635,133,1458,110,96,1532,1516,1678,1641,1536,1544,1481,1480,1658,111,395,1564,1563,1702,1599,1675,1682,1697,1698,1515,1671,1681,1685,1666,1560,1468,1695,1696,1720,1704,1632,1456,1497,1680,1653,1649,105,1619,1482,34,1618,140,131,25,1546,69,1688,1474,1473,1476,1475,1683,1700,1699,1711,1712,1713,1707,1735,1737,1738,1691,1517,448,1638,1614,280,1662,108,13,1502,1493,1576,21,31,1491,1630,128,1580,94,1494,1492,1629,29,1556,1510,1498,119,1558,1740,1588,1643,1496,1642,116,1612,37,1620,1739,1621,1624,129,1623,126,1622,1625,98,1689,1694,153,1709,1597,1548,1655,1684,1488,1611,90,1668,88); 

there were 2 unknown hosts in the list of 9 issue hosts

+--------+-----------------+-------------+
| hostid | hostname        | hostPending |
+--------+-----------------+-------------+
|   1519 | DESKTOP-G4VN9T5 | 0           |
|   1710 | ARROWHE-AS1BQVC | 0           |
|   1723 | imp-wks-6081    | 0           |
|   1719 | SCH-Wks-6077    | 0           |
|   1703 | IT-Tab-6026     | 1           |
|   1729 | MD-Bcode-Test   | 0           |
|   1718 | MSS-Lap-6076    | 0           |
|   1728 | ops-cb-6087     | 0           |
|   1721 | MSS-LTS-6080    | 0           |
+--------+-----------------+-------------+ 

So ran these sql commands to remove them from the database completely.

DELETE FROM `hosts` WHERE `hostID` IN (1519,1710);
DELETE FROM `hostMAC` WHERE `hmhostID` IN (1519,1710);
DELETE FROM `snapinAssoc` WHERE `saHostID` IN (1519,1710);
DELETE FROM `groupMembers` WHERE `gmHostID` IN (1519,1710);  

@Sebastian-Roth It’s been a while I realize but it’s started happening again. I think I found the issue in an internal script that was causing part of the issue during imaging but I fixed that. (Had to do with approving pending macs automatically after imaging a computer). But now I am still getting hosts that are in production randomly becoming pending hosts.

@george1421 I see that in the code but this is a method I use to get past dhcp on the daily. I think it’s going out to dhcp just to make sure the machine has an ip address and is on the network and then connects to the tftp server. But since it already has the efi boot file it’s not using dhcp to try to go find it, which is the part I believe he needs to skip. But if my thought doesn’t work then the default.ipxe method you outlined is worth a shot.

I would add to your method that in the case that file isn’t seen (like if it’s a super picky uefi bios) you could create a efi partition instead of fat32 with diskpart But I would bet your method there would work.

You could also use something like refind or grub2win on a usb and embed the ipxe.efi file and set it as a boot option. This would completely skip the dhcp portion of the pxe boot as you’ll arleady have the pxe file and it should skip straight to booting to the fog tftp server.

You could also include adding your chosen boot loader to your devices as part of your provisioning process (setupcomplete/firstlogoncommands etc.) so that you wouldn’t require pxe booting or a special usb to re-image a computer once deployed.

An easy way to test if booting straight to the ipxe.efi file would work is if your devices support either a built-in uefi shell or have the ability to choose a boot file in the bios settings. In that case you can just get a normal usb and simply put that ipxe.efi file on it (located on your fog server at /var/www/fog/service/ipxe/ipxe.efi I believe or it may be at /tftp or both). If all of that sounded complicated let me know the model of the computers and I’ll see if I can figure out if they have that capability.

There’s also the dnsmasq option which sets up a sort of proxy dhcp but it’s been a long time since I had to do that, but it makes it work as if you had the dhcp options set up. The only issue you might run into is if your dhcp server is on a faster network connection than your fog server, the dhcp server will beat your fog server to to responding to the dhcp requests.

Another thought, if the reason you can’t change the dhcp options is because it’s currently pointing to wds, you could also see if you’d be allowed to add policies to the dhcp server that would check for the specific devices and then boot just them to the fog server. Some example of this can be found in this wiki article https://wiki.fogproject.org/wiki/index.php?title=BIOS_and_UEFI_Co-Existence but you would have to do more research to find a way for the policies to detect the specific devices.

I will gladly go into more detail on any of these options if you want to give one a try.

@Sebastian-Roth I updated and imaged a computer. It didn’t happen during imaging again, yay!

However, I just saw another host that was imaged back in February come up as pending for no apparent reason. It still has its image history, inventory, membership, snapin history, login history, etc. It is missing its assigned image (it was assigned before the image’s display name was changed) and it is missing its active directory OU settings. I have not taken it out of pending in case there’s something more to look at for debugging purposes. I’m working remotely at the moment and it seems the computer is turned off, otherwise I’d grab the fog.log in case there’s something in there that hasn’t been truncated that might help.

I just figured out that sending stuff out with the image doesn’t work as well as one would like. None of the fog hosts seem to be working and the like. I try to make it grab the new image off one of the clients and it doesn’t actually grab them.

What do you mean by this? Do you see anything in the C:\fog.log on the clients? Or are you saying that they aren’t imaging correctly?

For that example snapin, if you’re just wanting that file to show up in C:\users a simple snapin pack. I’d take a look at the link @george1421 gave on snapinpacks and make a zip with a script that copies that file to C:\users.

As a simpler test you could create a powershell or batch script that just makes a hello world text file and see if that works.

i.e. powershell

"Hello World!" | Out-File -encoding oem -filePath C:\users\public\Desktop\hello.txt -force;

So put that into a file called hello.ps1 and make a new snapin with the Powershell template and upload the simple script. The snapin read-only command at the bottom should look like this
powershell.exe -ExecutionPolicy Bypass -NoProfile -File hello.ps1
Then add it to a host and deploy it as a single snapin task and see if it works.
You can deploy it then if you have access to the host you can run this in powershell to open up a dynamic version of the fog log to watch what’s happening on the client

cat C:\fog.log -wait

##cat is an alias for get-content. You can also do this with Get-FogLog if you install the FogApi powershell module

@Sebastian-Roth I updated to the RC2 from the master branch. Gonna be imaging a machine (well a virtual machine) shortly. I’ll let you know how it goes