@Zerpie You can also try booting from the ipxe shell, which if isn’t built in to the tablet as a boot option (sometimes it is sometimes it isn’t) then you can make a rEFInd usb disk and add all the ipxe efi boot options. Then you can create a startup.nsh script that will switch to the fs#: of the usb drive and then boot to whichever 32 bit ipxe file ends up working. It would be tricky and still involve usb drives but you could in theory make it work.

Another possibility would be to customize fog’s built in refind for those tablets if that happens to be booted to successfully (i.e. if boot to hard drive from the fog menu is working). You could change the default boot settings, I believe you can add some conditions to it, I know you can do it to have different times of the day have different default boot options. So one possibility would be to add the refind efi shell to the fog refind.conf boot options and make it the default during some time slot you are going to image and just also find a way to link a startup.nsh script. I haven’t actually tested this idea, it’s just another possibility if you want network boot to work. But all of that is nill if none of the ipxe efi boot files get you through bzimage32 boot.

@Tom-Elliott oh duh, pita, tee hee. I thought you might have just had something against pita’s the food or something.
I’m upgrading now and trying again.

Hi Friends,

We just got these new Z640 workstations and they come with these NVME 256GB SSDs plugged into a pci-e slot.
They show up in the bios, and when I boot to fog and check compatibility and partition information it says it’s compatible and all the partition info pops up no problem. One possible issue is that it assigns it to something like /dev/nvme instead of the standard /dev/sda

But then when I try to image or even just do a hardware inventory, FOG gives me a “HDD not found on system” error and then reboots after 1 minute.

Is this something that just isn’t supported? Do I need to enable something in a custom kernel? What am I missing here?

Thanks,
-JJ

P.S.
Fog info
Version svn 5676
bzImage Version: 4.3.0
bzImage32 Version: 4.3.0

Here is the code I use to create a snapin after publishing a chocolatey package to my repo.
I added the hashes after the problem started and it sometimes helps but it seems the behavior is slightly unpredictable and the hash record on fog still changes somehow.

        Write-Verbose "making sure package $global:packageName exists as fog snapin";
        if ( (Invoke-FogApi -uriPath "snapin/search/$global:packageName" -Method GET).count -eq 0){
            Write-Verbose "snapin does not exist, creating new snapin";
            $snapinScript = Get-Item "path\to\chocoPkgSnapin.ps1";
            $hash = ($snapinScript | Get-FileHash -Algorithm SHA512).Hash;
            $fileSize = $snapinScript.Length;
            $json = @{
                "name"="$global:packageName"
                "file"="chocoPkgSnapin.ps1"
                "args"="-pkgname $global:packageName"
                "reboot"=""
                "shutdown"=""
                "runwith"="powershell.exe"
                "runwithArgs"="-ExecutionPolicy Bypass -NoProfile -File"
                "protected"="0"
                "isEnabled"="1"
                "toReplicate"="1"
                "hide"="0"
                "timeout"="0"
                "packtype"="0"
                "storagegroupname"="default"
                "hash"="$hash"
                "size"="$fileSize"
            } | ConvertTo-Json;
            Invoke-FogApiChocoSnapin -uriPath 'snapin/new' -Method POST -jsonData $json -verbose;
        }
        else {
            Write-Verbose "Snapin already exists";
        }
        Write-Verbose "Updating hases for all snapins";
        $snapinScript = Get-Item "path\to\chocoPkgSnapin.ps1";
        $hash = ($snapinScript | Get-FileHash -Algorithm SHA512).Hash;
        $fileSize = $snapinScript.Length;
        $snapins = Get-FogObject -type object -coreObject snapin;
        $snapins.snapins | Where-Object file -match 'choco' | ForEach-Object { $data = @{
            "id" = "$($_.id)";
            "name" = "$($_.name)";
            "file" = "$($_.file)";
            "runwith"="powershell.exe";
            "runwithArgs"="-ExecutionPolicy Bypass -NoProfile -File";
            "args" = "$($_.args)";
            "protected"="0";
            "isEnabled"="1";
            "toReplicate"="1";
            "hide"="0";
            "timeout"="0";
            "packtype"="0";
            "reboot"="";
            "shutdown"="";
            "size"="$fileSize"
            "hash" = "$hash";
        } | convertto-json; 
        Update-FogObject -type object -coreObject 'snapin' -IDofObject $_.id  -jsonData $data -uri "snapin/$($_.ID)/
        Write-Verbose 'Done!';
        return;

Some of the snapins return 500 errors when I attempt to loop through them all and update their hash records.
Since that isn’t working I’m really hoping there’s some way to disable the hashing function, even if it’s some hackish way in the database or something.

@Tom-Elliott Oh good, I was about to post a horrifying screenshot of errors I just got.
Trying again

@george1421
Yes that’s how I got the hardware inventory to run without error.
But that doesn’t work for imaging. It acts likes it’s going to work, filling partitions and what not, then it just says database updated and reboots like it was finished instead of launching into partclone.

First let’s try to gather some more information. Leave the snapin as is for now and only schedule snapin tasks for clients and let those run.

1. Does it randomly fail on the same host? So if you schedule the same task on one host ten times in a row, does it run through all the time if it is fine on the first run?

I don’t typically try it more than once maybe twice, if it fails I just run the command manually. But I will give that a try. See if I can’t recreate it on my computer.

2. When it magically fails on a host can you please grab that host and take a look at C:\Program Files (x86)\FOG\tmp. I have not checked the code yet but I would hope the fog-client leaves a copy of that file there on failure. Now check if that file is zero size or truncated or anything like that.

It does not leave a copy, it does put a copy there and if you’re watching it you can copy it real quick or you can stop the fogservice before the hash check if you’re lucky. I think I did that once when I first discovered the issue but don’t recall the result, I remember it being confusing, I guess I’ll have to try this again too.

3. On snapin failure as well pay attention to the apache and PHP-FPM log files to see if there is an HTTP/PHP error on snapin download (see my signature).

Good thought will give those a post.

4. Please post the full fog-client log here in case of a failure so we can have a look. Sometimes there is something we see that leads us to the real problem.

Standby

If that doesn’t help we can go ahead an try monitor the database for changes of the hash. FOG has a deamon running that updates the snapin hashes from time to time. Unlikely but maybe this is causing an issue. You can take a look in that log file (webUI -> logs -> snapin log) and even disable that in another step just to see if it interferes.

I was hoping there is a way to disable it on the server, maybe something in the database or something, because I didn’t want to go through the trouble of recompiling the fog client. The other trick is that I really only use snapins during provisioning right after I image a computer. But I guess if I want it fixed I oughta suck it up.

The only other thing that may be a factor is I also add the snapins to the host and initiate the task from the api during provisioning. So It’s possible that it being queued from api causes this too, we’ll see if that’s the case though since I’ll be testing by deploying through the gui.

@Tom-Elliott Not quite working.
Got past the partition count but got a different error

I am also trying the linux live cd idea to see if it gets anything different and maybe if there’s a way to change it to something more standard

Just wanted to chime in with another report on a speed change between 1.5.7 and 1.5.8

1.5.7 ~22 GiB/min
1.5.8 ~11 GiB/min

This is on nvme drives, and we have a gigabit port aggregation on the main deploying node (in case you’re wondering how we got it going so fast).

However on 1.5.7 there was always a slow but steady drop in speed. It would start at 20-25 GiB/min and slowly drop GiB/min every couple seconds. But I never cared much since the ~20 GB image was done deploying in 2-3 minutes each time. In 1.5.8 it isn’t doing the speed drop and the overall time taken is about the same. It was just cycling between just below and just above 11 GiB/min (i.e. 10.58 - 11.03 or something along those lines) Looking at some of my recent imaging times just before and now after the upgrade to 1.5.8 they’re all at about 2 minutes 30 seconds. The only real variation appears to be the hardware being imaged, which is to be expected.

Point being, perhaps there isn’t actually a speed change but rather a more accurate overall average speed for the whole process instead of attempting a realtime speed? Or maybe just a generally more steady speed? Or just a better way of calculating the displayed imaging speed?

@Chris-Whiteley Maybe take a look in the web gui at the report viewer -> Imaging log and see if there’s actually a difference in time for your images deploying before and after the upgrade? I’m finding mine are all still within 0-30 seconds of the same time.

@Arrowhead-IT I tried resizing the partition manually in windows to the smallest that windows would allow to shrink to see if it by chance would get past the resize test before resizing, but it didn’t work either. It was a long shot.

Wasn’t able to boot to a live cd just yet but I tried some other stuff.

I tried changing the primary hard disk to /dev/nvme0
because I figure that may be the new /dev/sda
When I tried to image the computer I got a new error

“Erasing Current MBR/GPT Tables… The specified path is a character device!
Warning! GPT Main header not overwritten! Error is 22
Warning! MBR not overwritten! Error is 29
Corrupted partition table was erased. Everything should be fine now”

It does still get through the hardware inventory, but in either case it doesn’t pull and hard disk information.
I’m going to try /dev/nvme next to see if that works.

I also tried changing the bios sata disk mode from RAID to AHCI, but I don’t know if that effects anything on an nvme drive since it’s plugged into pci-express.

Thanks for all the help thus far.

-JJ

@george1421 While I agree and understand how it all works. I have found that we did get an increase in speed when we setup the aggregated adapter on the storage node. Even with just one client going. But perhaps that’s really just agreeing with your statements. As like on a highway if it was 1 lane, you often slowdown cause of the other slow drivers and perhaps I just opened up a metaphorical passing lane for fog images to go the full speed limit at. You do also have to consider all the switches it goes through and yada yada. And of course it’s all more complicated. Point being, I didn’t get a 2x boost when we aggregated the server link, but we did get a boost, so I wouldn’t deter anyone with the equipment capable of it from giving it a try.

@Tom-Elliott Great Success! It’s all good again.

I looked through the other post and gave debug mode a try,
I ran lsblk and got the same disk info except it didn’t have any mount information.
But is said nvme0n1 and partition nvme0n1p1

Also, here’s my ranting on unattend and sysprep that I can’t seem to stop myself from interjecting.

There are also multiple places you can put the unattend.xml file to get it to be used. I put it in 2 places C:\unattend.xml and C:\windows\system32\sysprep\unattend.xml I could have swore you had yours in a panther folder when I first read this, but it looks like I read that wrong or you edited while I was writing. I reference the one in the root of C in my call to sysprep and have it in the sysprep folder as a fail safe because it is a place that sysprep looks for the file. Is the unattend file doing what you expect? You have some winpe commands and partition creation commands in there. Are you using winPE in your process or is that just from the guide you found?

There are many ways that you can use sysprep and fog. The way I do it is

Install windows 10 onto vm
Hit Ctrl+shift+f3 at the oobe screen to enter audit mode
Run my base customization script (copies some custom files, copies some network drivers, installs some basic default programs)
Run my cleanup script (defrag, cleanmgr, sfc, dism, chkdsk, provision the metro/uwp apps to my liking…I’m thorough…)
Copy the unattend file and run sysprep with sysprep.exe /audit /reboot /unattend:C:\Unattend.xml

My unattend file is then set to run the Audit System pass, the Audit User pass, then the generalize pass and then shuts down. At that point I capture the image. I use the reseal property in unattend.xml to control which phase comes next.

Then when I deploy the image, it is a generalized image with no drivers but with lots of customization built in. Right after the image deploys the unattend file continues in specialize phase where, among many other things, the network drivers I copied earlier that apply to the hardware get added to ensure network connectivity. Then the oobe phase starts where all the oobe wizard stuff gets skipped because it is all already answered and then I use the firstlogoncommands section to start up a custom provisioning powershell module/script that sets everything up across a few reboots. I use the built-in admin user during those firstlogon setup pieces, I find that much easier and more reliable then setting up a separate user, the builtin admin usually has a better time with initial setup and having admin rights to do everything you want to, you can always disable the admin account once all is done.

That may all sound kinda complicated, but once you get it set up and scripted, it’s pretty simple.

My point in sharing all this, is you mentioned your new, and I want to let you know that there’s not just one way to handle all this. It’s good that you’re wanting to use sysprep at least. I was once a misguided soul and tried to find ways around it.

I see that you used a website to generate the answerfile. It looks like a decent starting point. I would reccomend making the whole thing yourself using the windows system image manager. It’s not quite as easy, but you’ll be able to see how much more you can do. https://docs.microsoft.com/en-us/windows-hardware/customize/desktop/wsim/windows-system-image-manager-technical-reference. Once you download the adk you can extract your windows 10 iso somewhere and load it up into the system image manager and start messing with it.

I could go on and on, but I don’t want to scare you away. Hopefully I haven’t already. It is completely possible to use somone elses or a web generated unattend file and image with fog without breaking any windows licensing or custom ids. I just felt, once I read through the documentation and gave it a try, that it was worth it to make it myself the official windows way. And I found that they have a lot of documentation on the topic. Here’s some more to look at, docs.microsoft.com is your friend in this adventure.

https://docs.microsoft.com/en-us/windows-hardware/customize/desktop/unattend/
https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/sysprep--system-preparation--overview

@Wayne-Workman
[Mon Jan 25 12:30:49.536526 2016] [:error] [pid 24884] [client 192.168.100.108:61538] PHP Fatal error: Uncaught exception ‘Exception’ with message 'FOGFTP: Failed to connect. Host: 192.168.100.100, Error: Undefined variable: whereA$

I couldn’t copy the whole error message from the terminal with ease, it appears to be ftp related. So I tried to ftp in only for it to not work on port 22 and on port 21 I get this message
OOPS: vsftpd: refusing to run with writable root inside chroot()

So this is a me accidentally running chmod -R 775 / instead of chmod -R 775 ./ and not cancelling it fast enough problem, not a fog problem at all.

Did a debug deploy
I found that fog saw the partition name as /dev/nvme0n11 instead of what was listed in gdisk -l /dev/nvme0n1p1
tried to change the name to no avail.

So if the root problem you want to solve is fog joining the domain I think I see an issue, but could be missing something. So in your script that starts sysprep you have it copy a setupComplete.cmd. I also see that you have it disable the fog service. Does setupcomplete re-enable and start the fogservice back up? If it is starting backup are you getting any error messages in C:\fog.log on the imaged machine?

I had set my snapins to a different directory (/images/snapins with /images being a symlink to /home/fogRootSystem/images)
But the original install had created the /opt/fog/snapins directory and put the ca ssl folders there.
But then it did it put the CA ssl folders with new keys in the new /images/snapins folder.

All day today I have been troubleshooting the client wondering why it suddenly stopped authenticating. Uninstalling and reinstalling, re-imaging, modifying the msi, resetting encryption data, running the fog install script with the --recreate-CA and --recreate-keys options. And nothing worked. Then I noticed the duplicate CA and SSL directories and got to thinking that maybe that’s not supposed to be a thing. So I moved the /opt/fog/snapins folder to my /images/snapins directory
mv -f /opt/fog/snapins /images/snapins then deleted the opt folder rm -fr /opt/fog/snapins then made a symlink ln -s /images/snapins /opt/fog/snapins Then I reran the fog installer like this ./installfog.sh --recreate-CA --recreate-keys -Y and now the client is working again (after reinstall on the client).

So a fix exists, but it seems that changing the snapins directory doesn’t work for the client and causes weird ca/ssl authentication errors.

It’s possible that something else I did fixed my issue, but I am 99.9% sure that the symlink and reinstall recreating the CA is what solved it.