Posts made by JJ Fullmer

JJ Fullmer

Should also mention I’m using the latest available linux kernel version for the test 6.6.1

JJ Fullmer

Some possible usb options found when searching UFS in kernel config.
Seems like we need the UFSCHD driver defined here https://www.kernel.org/doc/html/latest/scsi/ufs.html
One generic option for UFS was enabled at the top level.

Symbol: PHY_MTK_UFS [=n]                                                                                        │
  │ Type  : tristate                                                                                                │
  │ Defined at drivers/phy/mediatek/Kconfig:30                                                                      │
  │   Prompt: MediaTek UFS M-PHY driver                                                                             │
  │   Depends on: (ARCH_MEDIATEK || COMPILE_TEST [=n]) && OF [=n]                                                   │
  │   Location:                                                                                                     │
  │     -> Device Drivers                                                                                           │
  │ (1)   -> PHY Subsystem                                                                                          │
  │         -> MediaTek UFS M-PHY driver (PHY_MTK_UFS [=n])                                                         │
  │ Selects: GENERIC_PHY [=y]                                                                                       │
  │ Selected by [n]:                                                                                                │
  │   - SCSI_UFS_MEDIATEK [=n] && SCSI_UFSHCD [=y] && SCSI_UFSHCD_PLATFORM [=y] && ARCH_MEDIATEK && RESET_CONTROLLE

Symbol: PHY_QCOM_QMP_UFS [=n]                                                                                   │
  │ Type  : tristate                                                                                                │
  │ Defined at drivers/phy/qualcomm/Kconfig:89                                                                      │
  │   Prompt: Qualcomm QMP UFS PHY Driver                                                                           │
  │   Depends on: PHY_QCOM_QMP [=n]                                                                                 │
  │   Location:                                                                                                     │
  │     -> Device Drivers                                                                                           │
  │ (2)   -> PHY Subsystem                                                                                          │
  │         -> Qualcomm QMP PHY Drivers (PHY_QCOM_QMP [=n])                                                         │
  │           -> Qualcomm QMP UFS PHY Driver (PHY_QCOM_QMP_UFS [=n])                                                │
  │ Selects: GENERIC_PHY [=y]

 Symbol: PHY_SAMSUNG_UFS [=n]                                                                                    │
  │ Type  : tristate                                                                                                │
  │ Defined at drivers/phy/samsung/Kconfig:32                                                                       │
  │   Prompt: Exynos SoC series UFS PHY driver                                                                      │
  │   Depends on: OF [=n] && (ARCH_EXYNOS || COMPILE_TEST [=n])                                                     │
  │   Location:                                                                                                     │
  │     -> Device Drivers                                                                                           │
  │ (3)   -> PHY Subsystem                                                                                          │
  │         -> Exynos SoC series UFS PHY driver (PHY_SAMSUNG_UFS [=n])                                              │
  │ Selects: GENERIC_PHY [=y]

This top level was enabled

 Symbol: SCSI_UFSHCD [=y]                                                                                        │
  │ Type  : tristate                                                                                                │
  │ Defined at drivers/ufs/Kconfig:6                                                                                │
  │   Prompt: Universal Flash Storage Controller                                                                    │
  │   Depends on: SCSI [=y] && SCSI_DMA [=y]                                                                        │
  │   Location:                                                                                                     │
  │     -> Device Drivers                                                                                           │
  │ (4)   -> Universal Flash Storage Controller (SCSI_UFSHCD [=y])                                                  │
  │ Selects: PM_DEVFREQ [=y] && DEVFREQ_GOV_SIMPLE_ONDEMAND [=y] && NLS [=y]

This one was not enabled, enabling it in a test

Symbol: SCSI_UFSHCD_PLATFORM [=y]                                                                               │
  │ Type  : tristate                                                                                                │
  │ Defined at drivers/ufs/host/Kconfig:30                                                                          │
  │   Prompt: Platform bus based UFS Controller support                                                             │
  │   Depends on: SCSI_UFSHCD [=y] && HAS_IOMEM [=y]                                                                │
  │   Location:                                                                                                     │
  │     -> Device Drivers                                                                                           │
  │       -> Universal Flash Storage Controller (SCSI_UFSHCD [=y])                                                  │
  │ (6)     -> Platform bus based UFS Controller support (SCSI_UFSHCD_PLATFORM [=y])

Also enabling these other choices

 --- Universal Flash Storage Controller                                                    │ │
  │ │                   [*]   Universal Flash Storage BSG device node                                             │ │
  │ │                   [*]   PCI bus based UFS Controller support                                                │ │
  │ │                   [ ]     DesignWare pci support using a G210 Test Chip                                     │ │
  │ │                   [*]   Platform bus based UFS Controller support                                           │ │
  │ │                   []     Cadence UFS Controller platform driver

JJ Fullmer

Microsoft recently released a “business only” surface Go 4.
This go around they switched from having an NVME controller option to only offering a UFS based option.
See also https://www.microsoft.com/en-us/surface/business/surface-go-4
In the past models you could get either eMMC based storage or NVME based storage.

UFS, being as it has previously not been a common pc form factor storage option, is not supported in the Fog Kernel (I’m on 6.1.22)

I imagine for me to get this thing deployed right now, I’ll need to go build a custom kernel.
Perhaps we can add UFS storage support in the kernel going forward?

JJ Fullmer

A minor bugfix update has been released…and then something more was caught (aliases for functions weren’t being exported) and another version was released right after

Release notes: https://github.com/darksidemilk/FogApi/releases/tag/2310.6.3
PSGallery: https://www.powershellgallery.com/packages/FogApi/2310.6.3

JJ Fullmer

@Lenain Just a little FYI.
I don’t know when I’ll have time to test it out fully before putting it into production, but the idea of adding this to the module was not forgotten and I started a branch for it while I was working on some other bug fixes
https://github.com/darksidemilk/FogApi/commit/03c0342cff759ef272a64bbb91cbb1b5999bab9c

JJ Fullmer

@typotony Sorry I’m late to the party.

I haven’t built a “helper” command yet for removing fog images, but I do have Get-FogImages

https://fogapi.readthedocs.io/en/latest/commands/Get-FogImages/

If you setup the fogapi powershell module, you can use that to get a list of the current images.

You can delete the records of the fog images from the database via

Remove-FogObject

i.e. this would delete ALL your images

$images = get-fogimages
$images | Foreach-object {
  Remove-fogobject -type object -coreObject image -idofobject $_.data.id
}

That little snippet is untested but should be enough to get you started.

I’m pretty sure the api command won’t remove the files from the server though.

However, I believe the path to the files, or at least the relative path is in a child property of the data property when you run Get-FogImages.
So if you set this up to run on the linux based fog server using the powershell api module, you could script something that runs in a cronjob to delete from the database and then the files.

i.e. (again untested as I’m away from my test environment, so the property names are guesses, but it should point you in the right direction)

Get-FogImages | % {
  if ($_.data.name -eq "removeMe") {
    Remove-fogobject -type object -coreObject image -idofobject $_.data.id
    rm -fr $_.data.path
  }
}

This, if run on the fog server where you have configured the fogapi module (see my signature for doc links), would find images named “removeMe” and then delete them from the database and then the files from the server.

JJ Fullmer

@Sebastian-Roth Sadly I have not actually done this.
I believe that it is possible and @tom-elliott may have some insight on accessing the api over https. I believe he has some forum posts elsewhere discussing it. I think there may be some fog server side configuration needed to enable https api. For sure you need to have the root CA certificate that issued your fog server web certificate trusted on the machine issuing the commands. In theory that should be all there is to it, but I haven’t done a ton of testing on it, and while it’s on my to-do list, it’s a bit far down the list. Not to say SSL/TLS isn’t very important even behind a firewall, I just haven’t got there yet.

@glequeau In theory, if you can get all the certificates trusted correctly and can access a fog api url in a browser over https, then it should work for the powershell commands as well. That’s how it typically works with powershell and api commands.

That was long, here’s a short version:

Go to your https fog site
view the certificate being used and view the trust chain
Download the root and any intermediate CA certificates
Install them on your machine as trusted root ca certs
Try the api commands again with the https url set

If that doesn’t work, then we just have more development needed on https api commands

You could also try editing your locally installed version of the fogapi module. Specifically the invoke-fogapi command (probably at C:\program files\windowspowershell\modules\fogapi\2303.5.33\fogapi.psm1) then search for that command in the compiled version of the file.

You would add the -SkipCertificateCheck switch to all calls to invoke-restmethod and invoke-webrequest which would bypass that error and still use the ssl connection, but ignore errors about untrusted certificates

JJ Fullmer

@Sebastian-Roth I have a work in progress concept for permanent links. But things have gotten busy and I haven’t had much free time to get it working in full.
Short version is that we’ll be able to create uuids for all pages and that can be used as a permanent link even if it moves elsewhere in the doc.

JJ Fullmer

@Lenain That’s great to hear, glad we could get it figured out.

JJ Fullmer

@JJ-Fullmer I think I have determined this is a problem in an internal process with the api. Gonna try to narrow it down some more.

JJ Fullmer

@KaiHerlemann
You should be able to do this without too much trouble for just the basic ssl communication like you do for any other linux server. I would suggest at install time NOT selecting ssl and then altering the config to use your custom certificate. I don’t think we’ve implemented custom certs into the install process. I use an internal CA and custom cert on the fog server for https web access, I configured it in apache after installing fog. I think there are some other places that get ssl enabled when you select ssl in the installer.
@Sebastian-Roth might know more on how the client uses certificates.

JJ Fullmer

@UserBxl Do you have an unattend.xml file with sysprep?
Did you happen to open the microsoft store or any microsoft store apps before running sysprep as that can cause issues.

This error typically means there’s an error in the unattend file or something went wrong with sysprep.

When you see the error you may be able to hit shift+f10 to bring up a cmd console and you can try to look at sysprep log files, i.e. notepad C:\Windows\System32\sysprep\Panther\setupact.log there’s a few places they may reside, see also https://learn.microsoft.com/en-us/windows-hardware/manufacture/desktop/deployment-troubleshooting-and-log-files?view=windows-11

That may help point you in the right direction.

Some common ones I’ve run into

Needing to provision store apps properly and make sure they are never opened before running sysprep. I run this powershell before sysprep as well

Write-Verbose "Removing current user packages"
		Get-AppxPackage | Remove-AppxPackage -EA 0;
#see also https://docs.microsoft.com/en-us/windows/application-management/apps-in-windows-10
		$allowedApps = @(
			"Microsoft.549981C3F5F10", #cortana
			"Microsoft.DesktopAppInstaller",
			"Microsoft.GetHelp",
			"Microsoft.HEIFImageExtension",
			"Microsoft.HEVCVideoExtension",
			"Microsoft.MicrosoftEdge.Stable",
			"Microsoft.Microsoft3DViewer",
			"Microsoft.MicrosoftOfficeHub",
			"Microsoft.MicrosoftStickyNotes",
			"Microsoft.MPEG2VideoExtension",
			"Microsoft.MSPaint",
			# "Microsoft.Office.OneNote",
			"Microsoft.ScreenSketch",
			"Microsoft.StorePurchaseApp",
			"Microsoft.VCLibs",
			"Microsoft.VP9VideoExtensions",
			"Microsoft.WebMediaExtensions",
			"Microsoft.WebpImageExtension",
			"Microsoft.Windows.Photos",
			"Microsoft.WindowsAlarms",
			"Microsoft.WindowsCalculator",
			"Microsoft.WindowsCamera",
			# "microsoft.windowscommunicationsapps",
			# "Microsoft.WindowsFeedbackHub",
			# "Microsoft.WindowsMaps",
			# "Microsoft.WindowsSoundRecorder",
			"Microsoft.WindowsStore",
			# "Microsoft.Xbox.TCUI",
			# "Microsoft.XboxApp",
			"Microsoft.XboxGameOverlay",
			"Microsoft.XboxGamingOverlay",
			# "Microsoft.XboxIdentityProvider",
			"Microsoft.XboxSpeechToTextOverlay",
			"Microsoft.YourPhone",
			"Microsoft.ZuneMusic",
			"Microsoft.ZuneVideo"
		)

		Write-Verbose "Removing provisioned appx packages that dont match allowed list"
		Get-AppxProvisionedPackage -Online | Where-Object { 
			$_.DisplayName -notin $allowedApps;
		} | ForEach-Object {
				if($_.PackageName) {
					Write-Verbose "Removing provisioned package $($_.DisplayName)"
					Remove-AppxProvisionedPackage -Online -packagename $_.PackageName;
				}
			}

Typo in the unattend i.e. missed a /> or put a setting twice
Typo in domain join fields like OU or domain
Specifying settings that aren’t compatible with one another (i.e. workgroup join and domain join)

Hope that helps to get you started.

JJ Fullmer

@Sebastian-Roth said in >FOG vs UEFI:

@jeremyvdv Maybe this is going to help you: https://wiki.fogproject.org/wiki/index.php/Windows_DHCP_Server

This is also on the new docs page https://docs.fogproject.org/en/latest/installation/network-setup/dhcp-server-settings/

@george1421 said in >FOG vs UEFI:

@jeremyvdv If you have a windows dhcp server AND you need to pxe boot both bios and uefi based computers you might want to review this wiki page to setup dhcp profiles so your dhcp server sends the proper boot file name based on the target computer: https://wiki.fogproject.org/wiki/index.php/BIOS_and_UEFI_Co-Existence#Using_Windows_Server_2012_.28R1_and_later.29_DHCP_Policy

This is also on the new docs page

https://docs.fogproject.org/en/latest/kb/how-tos/bios-and-uefi-co-existence/

JJ Fullmer

@Lenain I also made a github issue for making a function out of this
https://github.com/darksidemilk/FogApi/issues/7

If you’re feeling saucy feel free to fork the repo and give it a go and pull request it. I’ll get to it eventually, but it could be quite a while, life is rather busy at the moment.

JJ Fullmer

@Lenain Howdy, I’m the author of the FogApi powershell module and I’m happy to help.

A few questions

Are you trying to send/push/deploy an image with wol enabled or are you trying to create a wake only task?
Have you confirmed through other means that wake on lan is working in your environment, i.e. without additional switch configuration it typically doesn’t work across different subnets. It can, you just have to allow it.
Here’s another alternative in powershell, you can send a magic packet to a mac address with this function

function Send-WoL {
    param (
        $Mac,
        $port = 9
    )
    process {
 
        $MacByteArray = $Mac -split "[:-]" | ForEach-Object { [Byte] "0x$_"}
        [Byte[]] $MagicPacket = (,0xFF * 6) + ($MacByteArray  * 16)
        $UdpClient = New-Object System.Net.Sockets.UdpClient
        $UdpClient.Connect(([System.Net.IPAddress]::Broadcast),$port)
        $UdpClient.Send($MagicPacket,$MagicPacket.Length)
        $UdpClient.Close()
    }
}
Send-Wol -mac "12:34:56:78:90:aa"

If you’re trying to do a wake only task, the syntax will likely be different for the json you need to send

You might try creating a scheduled wake on lan task manually and then get the pending task from the api to see the fields
i.e. Get-FogObject -type objectactivetasktype -coreActiveTaskObject scheduledtask | select -expand data | select * -ExcludeProperty host | ConvertTo-Json

I ran that and in comparing those values and assuming you don’t want it to be scheduled, this should do the trick (I tested it and it did indeed do the trick)

$jsonData = @"
                    {
                        "taskTypeID":"14",
                         "wol":"1",
                        "other2":"-1",
                        "other4":"1",
                        "isActive":"1"
                    }
"@

New-FogObject -type objecttasktype -coreTaskObject host -jsonData $jsonData -IDofObject $hostID

The send-fogimage function was a good one to look at for an example, but pushing an image has more fields for the task than other tasks, some tasks only need the type id and what host to run it on. This one might even work without some of these variables.

Hope that helps

JJ Fullmer

Fog version: 1.5.10.4
Running on centOS 7

I have a number of hosts where the active directory information is randomly going poof.
The host gets imaged, it joins the domain with a post download script, and all is well. Then some time later I go check on things to find the information has gone away and no one has edited it.

I think it might have something to do with the fog service and adding pending macs, but I’m not sure. I have globally disabled adding new hosts via the fog client to see if that had any affect, but there was no change.

I’m unsure where to look at the moment to troubleshoot this further.

JJ Fullmer

@Sebastian-Roth I think you’re thinking of this post https://forums.fogproject.org/topic/16720/hp-elitebook-840-g9-cannot-deploy-image

JJ Fullmer

@lebrun78 Had another idea.
Since it does work with the system account in the psexec shell you try the scheduled task method again and have it use the system account. It would look something like below

Also, matching just ‘IPV4’ from the firmware boot options may not yield the most reliable results as not every device will name their network boot option like that. Another option that may work would be finding what your various models do use for pxe identification and script finding that one within this script, that would take some effort to maintain but would always get you the options you want
Another possibility is finding all possible choices and putting them all above the windows boot manager. For example, I have a custom built pc that has 3 options related to network boot (excluding ipv6) options, I could find each of those guids and put them all at the top of the list so each one will be tried at boot. I altered the script below to use that method matching the things I found in a couple places (‘IPV4’, ‘Network’, ‘pxe’) and I switched the matching up using a method I found here https://stackoverflow.com/questions/16903460/bcdedit-bcdstore-and-powershell to parse the enum of firmware into a powershell object, it’s not a perfect conversion to an object but it’s workable.

Another other option would be to get the tftpboot pxe files locally and add them to the efi partition and have it try a couple known ones, this method would work nicely as well especially on devices that don’t have built-in ethernet as they won’t have to support pxe boot to get to the bootfile that will get you into fog. But to make that work in a way that would support multiple pxe boot file options is a bit out of scope of this post.

#start the task 10 seconds after its defined
$trigger = New-ScheduledTaskTrigger -Once -at ((Get-date).AddSeconds(10))
$settings = New-ScheduledTaskSettingsSet -WakeToRun -Priority 0;
#make sure temp path exists, and use temp path to avoid issues with permissions on root of C
if (!(Test-Path 'C:\temp')) { mkdir 'C:\temp'; }
#use system account
$principal = New-ScheduledTaskPrincipal -UserId 'NT AUTHORITY\SYSTEM' -RunLevel Highest -LogonType ServiceAccount;
$sb = {
    $Configs   = New-Object -TypeName System.collections.generic.List['System.Object']
    $NameArray = @()
    $Pattern = '^(?<name>[a-z]*)?\s*(?<value>.*)?$'
    $firmware = (bcdedit /enum firmware);
    foreach ($item in $firmware ){
        if ( $item.trim() ){
            $res = [regex]::matches( $item, $pattern )
            if ( $res ){
                $Value = $res[0].Groups['value'].value
                $Name  = $res[0].Groups['name'].value
                if ( $Value ){
                    if ( $Name ){
                        "$item creating pso" | out-host;
                        $PSO = [PSCustomObject]@{
                            Name  = $Name
                            Value = $Value
                        }
                        $NameArray += $PSO
                    } else {
                        if ( $NameArray.count ){
                            ( $NameArray | Select-Object -last 1 ).Value += "; $Value"
                        }
                    }
                }
            }
        } else {
            if ( $NameArray ){
                $Configs.add(($NameArray))
                $NameArray = @()
            } 
        }
    }
    #the above loop from the example I found doesn't add the final item, so run this bit one more time
    if ( $NameArray ){
        $Configs.add(($NameArray))
        $NameArray = @()
    } 
    $netbootsipv4 = $configs | Where-Object { $_.Value[-1] -match "Network" -OR $_.Value[-1] -match "PXE" -OR $_.Value[-1] -match "IPV4" } | Where-Object { $_.Value[-1] -notmatch "IPV6" }
    #loop through the found netboot options and put each one at the top so all of them are before windows boot
    $result = New-Object -TypeName System.collections.generic.List['System.Object']
    $netbootsipv4 | ForEach-Object {
        $GUID = $_.Value[0];
        $bootOption = "Adding Desc: $($_.Value[1]) GUID: $GUID to boot first";
        $addBootToTop = (bcdedit /set "{fwbootmgr}" displayorder $GUID /addfirst);
        $result.add($bootOption);
        $result.add($addBootToTop);
    }
    #now that for sure all pxe boot options are first boots, try to find the active connection that matches and make it for sure the first boot option
    $upNets = (get-netadapter | Where-Object status -eq up).InterfaceDescription
    $upNets | ForEach-Object {
        #set the current item to a variable
        $adapterName = $_;
        #see if the active adapter's description matches any of the found ipv4 boot options
        $adapterBootEntry = $netbootsipv4 | Where-Object { $_.Value[-1] | Select-String -pattern $adapterName -SimpleMatch}
        if ($adapterBootEntry) {
            $guid = $adapterBootEntry.Value[0]
            $bootOption = "This interface is active, put it towards the top - Adding Desc: $($_.Value[1]) GUID: $GUID to boot first";
            $addBootToTop = (bcdedit /set "{fwbootmgr}" displayorder $GUID /addfirst);
            $result.add($bootOption);
            $result.add($addBootToTop);
        }
    }
    
    $newBootOrder = (bcdedit /enum "{fwbootmgr}");
    #make c:\temp if it doesn't exist
    if (!(Test-Path 'C:\temp')) { mkdir 'C:\temp'; }
    #log everything in a new C:\temp\firmware.log file
    $resultString = "Firmware: $($firmware | out-string)`n`nresult: $($result | out-string)`n`nNewBootOrder: $newBootOrder`n"
    New-Item -path C:\temp\firmware.log -itemType File -value $resultString -force;
}
$netBootScript = "C:\temp\netboot.ps1";
New-Item $netBootScript -value $sb.tostring() -force;
$action = New-ScheduledTaskAction -Execute "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -Argument "-File 'C:\temp\netboot.ps1'"
$task = New-ScheduledTask -Action $action -Description "update bcd" -Principal $principal -Trigger $trigger -Settings $settings;
$taskName = "boot-to-IPV4";
#register the task
Register-ScheduledTask -InputObject $task -TaskName $taskName;
# force the task to start
Start-ScheduledTask -TaskName $taskName;
# wait for the task to finish
while ((Get-ScheduledTask $taskName).State -eq 'Running') {
    Start-Sleep -Seconds 1;
}
#delete the task
Unregister-ScheduledTask -TaskName $taskName -Confirm:$false -EA 0;
#delete the created script file
remove-Item $netBootScript -force -ea 0;

I saved the above as a .ps1 file and made it a snapin and it worked for me when I tested it on a vm. However, it did make it so when fog booted to the hard disk via refind it wasn’t able to boot into windows. This may be related to my own custom configs in refind, but something to test for on a few different platforms to be on the safe side.
Wait wait, I made a small change to the logging and updated the snapin and then it didn’t work the second time. That’s odd… I recreated and couldn’t get it to work a second time. There’s something weird going on for sure. I know I double checked that things were not already configured to boot to the network the time it did work.
Granted, since it’s happening via scheduled task, if you have an AD environment you could deploy this script in a scheduledTask via a gpo. Add the file (the one created from the $sb in the script) to computers with file preferences and create the scheduled task that runs as the system account using a scheduled task gpo.

Will have to dig more into this on why the script isn’t doing anything via snapin, maybe @Sebastian-Roth can help a bit on troubleshooting the client side. And maybe other @testers could test using this script as a snapin too to see what results are being had elsewhere.

Then just as an fyi to anyone reading this, you can revert back to the normal windows boot manager with (bcdedit /set "{fwbootmgr}" displayorder "{bootmgr}" /addfirst);

JJ Fullmer

A minor bugfix update has been released

Release notes: https://github.com/darksidemilk/FogApi/releases/tag/2304.5.41
PSgallery Listing: https://www.powershellgallery.com/packages/FogApi/2304.5.41

JJ Fullmer

@lebrun78 As I mentioned that method would only work if the admin is logged in. If you’re running this at deploy time and have other firstlogoncommands that run via an unattend.xml file then it would work when the computer is first imaged to change the boot order back to pxe boot first.

There is a feature request that’s being considered for making it so the fogclient can handle changing the boot order, but no idea when there will be the time for that to be implemented. You’re welcome to fork that repo and give it a go

Personally I made an internal powershell module that queues the image using the FogApi powershell module (see my signature) and runs a remote powershell command on the machine I’m imaging to download the latest ipxe.efi (or snponly.efi) and I set that as the bootfile (I find it hard to match the network boot option as it’s labeled different on different platforms in my environment, but booting directly to the efi boot file always works).