@sebastian-roth But if you test it, what’s the point of us Testers? =p

Here’s the information from the debug session with the new init. Looks mostly the same. I’m going to try installing windows with default partitions the old fashioned way and I’ll see if image capture works by chance.

@sideone Are you getting different IP addresses in the snapin download url and the communication url? i.e. is the communication URL going to your master and the download going to a storage node? Maybe a storage node has something goofy in its storage node config?

The communication url is derived from the c:\program files (X86)\Fog\settings.json file where it uses the Https flag, server name, and webroot values to construct that base url i.e. http://10.xx.xx.xx/fog
I’m not sure if the download url is combo of the settings.json file with info from the server or if it’s just from the server or what. But your download url is missing the http/https and is also has an extra slash.

P.S. you don’t really need to hide the ip of the fog server. It’s an internal subnet only ip address, it doesn’t expose anything to say what ip you’re using. Do whatever makes you feel safe, just saying is all.

@librarymark That’s the way I do it or at least the way I created my groups originally, but both options are available to be flexible. I mean if I did it now I would use the powershell module I just made for the api along with the PS module for active directory and create and populate groups based on OU’s, could even make it sync with a script in a scheduled task on a windows server.

But I digress. I did in fact confirm the bug of phantom hosts when creating a group from the hosts screen. I selected the top 3 hosts (id’s of 83, 26, and 135) and created a new group with the tool at the bottom of the list. Then I went to the group to find 5 members, the 3 I chose plus 2 more with ids of 1531 and 1535. I’m including the ids in case there’s some possible correlation to be seen to troubleshoot the bug.

Add printer configuration to printer management. i.e. a sharp copier that requires you to manually click auto configuration for the printer to update it’s settings to match the printer.
I already have this scripted and imagine it would be pretty easy to add to the existing printer management. I would gladly add and test the code myself if someone could point me in the right direction.

So lets say you have added a printer and configured it to all the correct settings, like how many input trays, types of output trays, maybe it requires putting in a security pin or it has a punch module. And it looks like this for example.

Then to save those configuration settings you simply need to run this command in a command prompt
RUNDLL32 PRINTUI.DLL,PrintUIEntry /Ss /n"Printer Name" /a Path\ConfigFile.dat m f g p
and then to configure the printer once it’s been added with the saved config
RUNDLL32 PRINTUI.DLL,PrintUIEntry /Sr /n"Printer Name" /a Path\ConfigFile.dat m f g p

I imagine that the way the print manager works is similar to the script I use to add a printer with the similar command line inputs.
For example, I can use the following script (manually adding in the parameters in the call of installPrinter in the main function) to install a printer. I could probably easily change it to take arguments into the execution like snapin arguments taken from the printer information. Which I imagine is somewhat how the add a printer works.

@ECHO off

call :main
exit

:main
	REM Function to call other functions and run the installation process

	call :setVars
	call :funcHead "Welcome to the Printer installer!"
	REM inputs: 1 - Printer Port Name, 2 - printer ip or hostname 3- driverPath 4 - printer name 5 - printer model 6 - config file 7 - raw or lpr
	call :installPrinter 

	EXIT /B

:setVars
	REM function for setting script variables, typically for directories

	call :funcHead "Setting script variables"

	set share="\\networkShare\Printers\Drivers"
	net use "%share%"
	
	echo. done!

	EXIT /B

:installPrinter
	REM Function to add a new Printer
	REM inputs: 1 - Printer Port Name, 2 - printer ip or hostname 3- driverPath 4 - printer name 5 - printer model 6 - config file 7 - raw or lpr

	call :funcHead "Installing Printer %~4"

	call :printerPort %~1 %~2 %~7
	call :printerDriver "%~3"
	call :addPrinter "%~4" "%~5" %~1
	call :configPrinter "%~6" "%~4"

	echo. done installing printer %~4!

	EXIT /B

:printerPort
	REM function for adding a printer port
	REM var inputs 1 - port name 2 -hostname or ip address 3 -port type (raw or lpr)

	call :dots
	echo. Creating the printer port...
	
	IF %~3==lpr (
		Cscript %WINDIR%\System32\Printing_Admin_Scripts\en-US\Prnport.vbs -a -r %~1 -h %~2 -o lpr -q lp -n 515
	) ELSE ( REM raw
		Cscript %WINDIR%\System32\Printing_Admin_Scripts\en-US\Prnport.vbs -a -r %~1 -h %~2 -o raw -n 9100 
	) 
	
	echo. done!
	call :dots

	EXIT /B

:printerDriver
	REM function to add the driver input 1=full driver path

	call :dots
	
	echo. Adding printer driver...
	PNPUTIL -i -a "%~1"
	echo. done!
	
	call :dots

	EXIT /B

:addPrinter
	REM add the printer to the created port
	REM 1 - printer name 2 - printer model associated with driver 3 - port name
	
	call :dots
	
	echo. adding printer to network port...
	Cscript %WINDIR%\System32\Printing_Admin_Scripts\en-US\Prnmngr.vbs -a -p "%~1" -m "%~2" -r %~3
	echo. done!
	
	call :dots

:configPrinter
	REM add any special printer configurations
	REM 1 - config file path 2 - printer name

	call :dots
	
	echo. Configuring Printer...
	REM To create a config file for a printer, use the following syntax
	REM RUNDLL32 PRINTUI.DLL,PrintUIEntry /Ss /nPrinterName /a ConfigFilePath.dat m f g p
	RUNDLL32 PRINTUI.DLL,PrintUIEntry /Sr /n"%~2" /a %~1 m f g p
	echo. done!
	
	call :dots

	EXIT /B

:dots
	REM just echoing dots in a Function instead of copy pasting them so that it's consistent
	echo ......................................................................
	EXIT /B

:funcHead
	REM A simple function for displaying a consistent header at the start of functions
	call :dots
	echo. %~1
	call :dots
	EXIT /B

I would rather use the fog print management with the added ability to import printer config files, as it is the only thing it doesn’t do that my scripting method does. Granted I could just set this idea up as a snapin, but I don’t have the remove printer functionality in my script as is. So What can I do to add this functionality in.

So here’s some good news, image capture works!
I did just a default multiple partition install of windows and it uploaded no problem.
I had still set the host primary disk to /dev/nvme0n1 in the gui.
I’m going to try re-downloading the image to the same computer
Then I’ll try to downloading the image I actually need on it that is single disk resizable.

Thanks for all the help thus far.

@Tom-Elliott @Sebastian-Roth @george1421

I gave something else a try. I looked at the commits included in the dev_branch since 1.5.9 and found that Tom had added the machine details array in the taskqueue.class.php file.

First thing I tried was adding a primary user to the host, in case that being empty was somehow a problem. No dice.

Looking at it, it’s just a prettier more informative email array. But the parameters in that object must be used somewhere else, because I reverted the file to its last commit (made a .old version of it on my server, made a new version of it with the previous version’s contents) just to see if that change maybe broke something.

My debug task was waiting for me to hit enter at another updating database attempt, so I hit enter, and viola!

So now we gotta figure out why in the world that change in the email sent would break the database update to say the task is completed.

edit:
I debugged it a bit further. Restored the newer version of the file and started commenting things out one at a time to confirm the problem.

The code here is where the problem lies (line 293-315 of taskqueue.class.php found at https://github.com/FOGProject/fogproject/blob/99bb17b168e929796a8d76526f73bcfdac11bb4b/packages/web/lib/reg-task/taskqueue.class.php)

$Inventory = self::$Host->get('inventory');
        $mac = self::$Host->get('mac')->__toString();
        $ImageName = $this->Task->getImage()->get('name');
        $ImageStartTime = self::niceDate($this->Task->get('checkInTime'));
        $ImageEndTime = self::niceDate();
        $duration = self::diff($ImageStartTime, $ImageEndTime);
        $Snapins = implode(',', (array)$SnapinNames);
        $email = array(
            sprintf("%s:-\n", _('Machine Details')) => '',
            sprintf("\n%s: ", _('Host Name')) => self::$Host->get('name'),
            sprintf("\n%s: ", _('Computer Model')) => $Inventory->get('sysproduct'),
            sprintf("\n%s: ", _('Serial Number')) => $Inventory->get('sysserial'),
            sprintf("\n%s: ", _('MAC Address')) => $mac,
            "\n" => '',
            sprintf("\n%s: ", _('Image Used')) => $ImageName,
            sprintf("\n%s: ", _('Snapin Used')) => $Snapins,
            "\n" => '',
            sprintf("\n%s: ", _('Imaged By')) => $engineer,
            sprintf("\n%s: ", _('Imaged For')) => $primaryUser,
            sprintf("\n%s: ", _('Imaging Started')) => $ImageStartTime,
            sprintf("\n%s: ", _('Imaging Completed')) => $ImageEndTime,
            sprintf("\n%s: ", _('Imaging Duration')) => $duration
        );

If I comment out the last 3 pieces of the array and try again, the database updates and all is well again

$Inventory = self::$Host->get('inventory');
        $mac = self::$Host->get('mac')->__toString();
        $ImageName = $this->Task->getImage()->get('name');
        $ImageStartTime = self::niceDate($this->Task->get('checkInTime'));
        $ImageEndTime = self::niceDate();
        $duration = self::diff($ImageStartTime, $ImageEndTime);
        $Snapins = implode(',', (array)$SnapinNames);
        $email = array(
            sprintf("%s:-\n", _('Machine Details')) => '',
            sprintf("\n%s: ", _('Host Name')) => self::$Host->get('name'),
            sprintf("\n%s: ", _('Computer Model')) => $Inventory->get('sysproduct'),
            sprintf("\n%s: ", _('Serial Number')) => $Inventory->get('sysserial'),
            sprintf("\n%s: ", _('MAC Address')) => $mac,
            "\n" => '',
            sprintf("\n%s: ", _('Image Used')) => $ImageName,
            sprintf("\n%s: ", _('Snapin Used')) => $Snapins,
            "\n" => '',
            sprintf("\n%s: ", _('Imaged By')) => $engineer,
            sprintf("\n%s: ", _('Imaged For')) => $primaryUser
            //sprintf("\n%s: ", _('Imaging Started')) => $ImageStartTime,
            //sprintf("\n%s: ", _('Imaging Completed')) => $ImageEndTime,
            //sprintf("\n%s: ", _('Imaging Duration')) => $duration
        );

@Sebastian-Roth
Ok let me try to explain at least my idea in a simpler way. I tend to use too many words.

So lets say you have a department like accounting. This department all needs access to the same printers, software, and needs to be in the same OU.
I want to be able to add a host to the accounting group that already has all those settings saved.
So when I add a host to the accounting group I can then apply all the saved printers, OU settings, snapins, and image with a single action.

Does that make sense?

I think @george1421’s desires are a little more complex then mind, but a similar idea.

So downloading a multi-partition image that was uploaded from an nvme drive works on an nvme drive. But I can’t download one that was made before I updated the init
I’m going to try uploading and downloading a resizable image and then I’ll also try reuploading my main image to see if uploading it using the new init makes some sort of difference.

@george1421 I have seen some strange behavior with 1’s and 0’s related to the hostPending field in the database that I’m still working out with @Sebastian-Roth . I believe there’s some notes buried somewhere in my post here https://forums.fogproject.org/topic/14607/weird-host-behavior-some-disappearing-losing-primary-mac-some-suddenly-needing-approval where I discovered I had some hosts says pending = 1 some pending = ‘1’ and then pending = 0 and pending = ‘0’.
I ended up making all of them string values (‘1’ or ‘0’ rather than 1 or 0).
It doesn’t seem to be universally true throughout everything but I have seen it.

@nehsa
Also, I also want to know how to make a scheduled task via the api. I’ve messed with that idea and haven’t been able to figure out the json needed. My workaround has been to schedule a reboot in windows using a combination of powershell date math (to figure out seconds till a given time) and shutdown.exe /r /t $seconds then I stop the fog service on the host (so it doesn’t reboot the host) and queue the task for right now.
It would be much more elegant to do a delayed scheduled task.

I think the required fields may be something like

 {
     "taskTypeID": 1,
     "type":"S",
     "scheduleTime":"1585616580"
}

I have that commented out in some code but sadly didn’t write down what that time meant… but regardless I’m pretty sure it didn’t actually work. I think I probably just scheduled a task and did a get call on tasks to see what it looked like. If I didn’t, that’s what I should have done…
@Tom-Elliott might have some insight on scheduling a task via the api

@Junkhacker Well deploy is where speed is more important to me.
I gave 19 a go with the split and it actually had some weird error on the first deploy, I was in a meeting in my office so I didn’t really get to see the whole thing, but then it auto-retried the task and worked proper the second time. 2 minutes 18 seconds. The image size on the server seems to still be bigger than pigz was with 7.1 GB for a 15 GB image instead of 6.7 G for the old 18 GB image. But, don’t need to be too picky about it.

@Tom-Elliott Two minor issues I noticed. The first time (the slow deploy) I queued the image after doing a full host inventory via the pxe boot on the host. I had not specified any snapins and it randomly added like 10 of them. The second time I queued it from the gui and deleted the snapins and the problem didn’t repeat itself. However in both instances the drive didn’t resize itself.

@Sebastian-Roth No problem. I’m happy to help.
So I can’t upload a resizable image. It says
“Problem opening /dev/nvme0n1p for reading”

I’m going to try uploading my base image as a multi-partition and see if I can download that one.

@nehsa I finally figured out scheduled tasks via the api.
I have it written out with a powershell example (with verbose output so you can see the POST url used and the json data) in post
https://forums.fogproject.org/post/139328

Give that a looksee and let me know if you want more examples. Also in that Thread @Tom-Elliott explained what some of the needed fields for a scheduled or crontask do.

@JJ-Fullmer Well it wasn’t actually phpipam that was slowing it down.
I had some ssl logs running for a website running on that server. It was tracking access logs for every access of the ssl cert and site. Which was also adding up to over 400 GB of logs. I changed the apache configuration of the site to log much less. That solved the problem. So again, nothing wrong with zstd. Just wanted to have the right answer in case someone else happens to see a constant 5-8 Mbps transmit and receive on a storage node.

Well it only seems to work when it was uploaded from nvme drive. That’s kind of odd.

@Sebastian-Roth The variables you don’t recognize are part of post installation scripts, I haven’t modified any php classes.

I will see if I can get that output on monday, we’ve now deployed all of our surface go 1’s back to their users but I can probably nab one for a few minutes if I get in early enough.

I can also try a different ipxe boot file for good measure. I use ipxe.efi maybe snponly.efi will load things differently and yield different results

@mousepl Are you thinking you would want to be able to do this on the fly, or would you want to deploy the snapin on all hosts that have it assigned to it?
By on the fly I mean, select specific hosts from the snapin then deploy, that would be a bit more difficult to implement, but maybe adding a button for deploying on all hosts with a snapin is possible.

If you’re familar with powershell or willing to learn. You could create a function/script that does this with the api module.

i.e.

# install the module
Install-Module FogApi;
# Follow the setup instructions found in the links in my signature. i.e. get the fogapi token and user api token from your web gui
Set-FogServerSettings -fogServer 'serverName' -fogapitoken 'insertServerAPItokenHere' -fogusertoken 'insertFogUserAPItokenHere'

Assuming those steps are done for the user running the function

function Start-SnapinDeploy {
[CmdletBinding()]
param(
    #define the snapin name you want to deploy
    $snapinName
)
#get all the snapins from the server
$allSnapins = Get-FogSnapins;
#get all the host/snapin associations from the server
$snapAssocs = Get-FogObject -type object -coreObject snapinassociation;
#get the snapin you want to deploye in a single object
$snapinToDeploy = $allSnapins | Where-Object name -match $snapinName;
#find all the hosts that have that snapin associated
$hostsWithSnapin = $snapAssocs.snapinassociations | Where-Object snapinID -match "$($snapinToDeploy.ID)"
#loop through the hosts that have that snapin assigned and start a task for that snapin for each of them
$hostsWithSnapin.hostID | ForEach-Object {
    $hostID = $_;
    $json = (@{
        "taskTypeID" = 13
        "deploySnapins"=$snapinToDeploy.ID
    } | ConvertTo-Json);
    New-FogObject -type objecttasktype -coreTaskObject host -jsonData $json -IDofObject $hostID;
    }
}
# call the function to deploy a snapin named office365
Start-SnapinDeploy -snapinName 'office365'

I may be off on the json creation as I haven’t tested a single snapin deploy, so the required body data might be different. I just kinda altered the way I use the start-fogsnapins function https://github.com/FOGProject/fog-community-scripts/blob/master/PowershellModules/FogApi/FogApi/Public/Start-FogSnapins.ps1 for this example. I typically only use fog snapins for the initial deployment at imaging.

But as you can see from the example, while things are linked together, the linkage is really meant to go from host to snapin. Snapins are really meant to be deployed at imaging time, which is why they are all queued when you a image a host. The idea being to include software provisioning with your OS images. The feature request you described would be useful I agree, but one might argue that that makes more sense for a software deployment tool, where fog is an imaging tool.

It can be used as a software deployment tool, but that’s just not how it’s designed.

That’s all really just a disclaimer to say it may be a bit for this to show up as a new feature. It’s possible and it would be useful. I’m all for anything linked in a database having linked buttons in each spot in a gui. But creating that button may just be easier said than done, but may eventually be done.

You could in theory create a gui using the powershell module and built in powershell command show-command and some dynamic paramters that grab all the available snapins, it would take a bit of work but you could make a gui prompt with drop downs/multi-selector for fog host(s), snapins, then a deploy button. Point being, just because it isn’t in the web gui, doesn’t mean the capability doesn’t already exist.

I hope that helps

@Tom-Elliott I will gladly test them!
I’m currently trying the inits that came from the most recent btsync and they seem to be working too for the hardware inventory task anyway, haven’t tried an image yet.
Where are the new inits to test? Oh wait you posted while I was typing this, I’ll download those and give them a go right now.

@mosi Are you saying you recreated the problem on a VM with 3 nics or the problem doesn’t happen in that case?
Also, are all nics connected to the network? Are they in a nic team/bond/aggregate link? Connected to different subnets? only 1 connected?

EDIT: I tested this in a VM and recreated the problem. It didn’t matter if 1 or all adapters were connected, if 3+ exist on an unregistered host it behaves as @mosi describes.

UPDATE: I also tried some previous bzImage and init versions, all the same behavior.

According to the official page from microsoft https://www.microsoft.com/en-us/windows/windows-11-specifications it just says “secure boot capable” I guess we’ll just have to wait till it’s released to insiders to get some real world information.