Posts made by jonhwood360

@george1421

By and large I agree with you about the hash algorithm being irrelevant, however, some entities have requirements for minimum acceptable hashing for such kind of verification. I think a baseline of a choice between md5 and shasum baseline is fine, especially while development of the feature is ongoing (md5 is still widely used in forensics), however that will not always be the case and building support in for additional options might be better. The time cost involved is something the end user should accept when they select the additional complexity.

Just as a test, I timed sha 512 on my Windows 10 image. Here is the results:

Took about 2m 8s for 8 GB give or take. So the wait isn’t terrible at higher algorithm complexity. This test was done with a two virtual processor VM on a server with a bunch of other vms running.

Interestingly enough, the shasum utility on ubuntu can compare hashes to a text file for verification.

Also, I don’t think the hash need be taken inline with the imaging process, but be done post imaging, either on demand, or run in background automatically before image is made available for distribution.

Thoughts?

@tom-elliott said in Feature Request for FOG 1.6.X - Add image integrity verification check:

@jonhwood360
I’ll try to document better:

• 1. Why would we use this per host? Unless you mean per server containing images?

Sorry, I meant image, not host. However since fog does potentially have a distributed architecture, maybe a hash per image per image repository?

@tom-elliott I will assist as I am able to as per our private conversation. I agree a plugin would be good. Here are my thoughts:

• Plugin should probably allow the hash algorithm to be defined per image. That way a range of security requirements could be met. A drop down of available algorithms maybe?
• I agree with the database, although would this be a totally new database separate from the main fog one? If so I would wonder about query complexity and displaying that information on the image page in the gui?
• Does fog have a scheduler built in for timed tasks? If so, the plugin could maybe piggyback off of that to address the
  periodic update requirement

@george1421 Exactly, although I’d like to have the option for the algorithm used for the hash be user defined. I know some people may want Sha512, where as for some MD5 is sufficient for this purpose. A default of sha256 probably would be good as its more than the minimum (md5) but not as intense as 512.

edited: because I typed host instead of image. sorry for the confusion if any

Hi all,

One item I would like to see added is for when an image is captured for FOG to take a hash of some kind of the image files, and then have a view in the gui where you can see those hashes, and potentially ask it to do a check for those files against the captured hashes to verify they have not changed. The idea is so that if the image is altered outside of the image capture process, (as may be done by a hacker), the image could be flagged as changed outside of the normal capture process and would need to be hard overridden to be deployed.

Perhaps also a periodic check for image integrity as well with results that can be seen in the gui.

@george1421 said in Issue with EFI through PFSense Firewall:

@jonhwood360 said in Issue with EFI through PFSense Firewall:

so one thing I keep seeing in the packet capture which is weird, is that the router option (3) is set to 10.255.252.1 which is the gateway that the fog server uses, but is not the gateway for the external network. .

Well lets think about this for a minute. When I looked at the packet capture I did notice the lease times were different too.

Looking at the config file you provided both the lease times and route address are set correctly. Why are we seeing a difference in the packet captures from the configuration. Its almost like we have a different dhcp server for bios than uefi. Or two instances running with different config files on the FOG server.

Yep. The only thing I can figure is that somehow either the DHCP Relay or Arp Proxy in Pfsense is doing something weird, or the DHCP server is not processing my second range definition in its entirety (which is weird because booting into an OS proper, the exchange happens normally and gets the right settings).

Although in the mean time I have had to move along in my research so I moved the DHCP/ Bind DNS to PFsense and have configured it to have the correct options. EFI boot from fog server is working in this new configuration. I am leaving the existing configuration in place on the fog server though to return to this issue (which I think has value to try and solve).

@george1421

so one thing I keep seeing in the packet capture which is weird, is that the router option (3) is set to 10.255.252.1 which is the gateway that the fog server uses, but is not the gateway for the external network. .

and I figured out why the tftp-server-name line you gave me didn’t work. It was expecting a dns name, not an IP, there is a tftp-server-address option, which I set and did work (config loaded). This did not change anything though.

@george1421 said in Issue with EFI through PFSense Firewall:

@jonhwood360 But I assume it still doesn’t boot.

So without the config file alterations you don’t get the screen about filesize 0 bytes, but with the alterations you get that error. Just thinking about it a bit more I think we are on the right direction, maybe just not the right path.

Did you get a workstation pcap of the process where you received the 0 bytes filesize? I’m interested to know if dhcp options 66 and 67 were being set, where it was actually kind of working. If they were there with those settings and the settings were the key for EFI DHCP booting then we just need to work on why the settings were wrong. I didn’t get a chance last night to test this in my home lab where I have a ubuntu server to understand why the settings stopped isc-DHCP from booting. But I’ll look at it tonight if we can’t get it sorted out. The commands I gave you were from the isc-DHCP config file.

No no, I get the same error, just without the unprintable character.

So, yes option 66 (next-server) and 67 (filename) are being set. If I set the Endpoint VM to BIOS, it works fine in the external network.

If I put the endpoint in the same network as the DHCP server, both EFI and BIOS works fine.

I’ll check to make sure the DHCP settings between internal subnet and external subnet are the same (sans the obvious IP changes)

Thanks again for helping!

@george1421 said in Issue with EFI through PFSense Firewall:

@jonhwood360 OK I know this now. That unprintable character is what is causing the issue. I see that in some “university” dhcp servers. Let me look at your pcap again.

Just a note, once I reverted my config to my original without alterations, that character went away.

@george1421 said in Issue with EFI through PFSense Firewall:

Between next-server and class insert this line
option tftp-server-name 10.255.252.5;

Then for every instance of “filename…”; just below that line enter
option bootfile-name “<boot_file>”;

Where <boot_file> matches the boot file name issued by the filename command.

George,

Just wanted to take a moment to thank you for helping to troubleshoot this with me.

I did as you requested.

if the “option tftp…” line is enabled the dhcp server will fail to start. Isn’t it the case for isc-dhcp-server that the “next-server …” line is the equivalent?

if I disable the next-server line and just use “option tftp” it also fails to start:

if I leave the “option boot-file…” lines in and leave “next-server…” on and “option tftp…” off, the server starts

However there is no difference in boot either in a VM, or hardware. I did however snag this screenshot (sorry for the glare) from the laptop which might shed some light.

I then reverted my dhcpd.conf to without the changes you recommended and confirmed the same result, sans the square at the end of the filename. Still same PXE error.

So it appears that the dhcp exchange for EFI mode on the computers are happening, they are getting the right configuration data, but the data is getting mangled at some point.

Any thoughts?

@george1421 said in Issue with EFI through PFSense Firewall:

@jonhwood360 Well this pcap was unremarkable, other than we can see that there is no tftp download of the ipxe.efi file.

I did find something interesting looking at the previous pcap. For a bios computer the lease time is 6 hours, for a uefi the lease time is 4 hours. Once might think that if you are using the same dhcp server the lease time should be the same.

Now looking at the two offer packets side by side the values are identical, but I did notice something that could cause the issue. If you look at the picture below. Only the bootp protocol fields have been populated in the offer packet. What is missing is the dhcp boot protocol using dhcp options 66 and 67. The problem is its up to the target computer firmware which fields it wants to look at bootp or dhcp. Most dhcp servers that support pxe booting will populate both parts. This is because the client could chose bootp or dhcp.

So is this dhcp server configured by FOG or did you hand configure this dhcp server? The FOG configuration “should” provide both boot protocols.

The DHCP server was configured by fog, and I duplicated the entry for the internal subnet, and changed the relevant fields for the external one (attaching the conf).

Also I took another pcap, just with the EFI boot, but from both FOG and pfsense(External). (attached).

fogserver-EFI-timed-exchange.pcap
pfsense-EFI-timed-exchange-EXTERNAL-interface.pcap
dhcpd.conf.txt

Also, in the original pcap from the fogserver, only transaction ID 0xe25c58da is the EFI PXE exchange. The options you pointed out are set for that transaction as:

The other two transactions are when the initial tftp happens from the BIOS boot rom, and then the loading of iPXE it looks like.

@george1421 said in Issue with EFI through PFSense Firewall:

@jonhwood360 All of the broadcast stuff from your pcap looks good. What I don’t see is the tftp request from the client computer because that is using unicast messaging.

If you follow this guide and run tcpdump from the fog server we should be able to see the tftp pull request (hopefully). https://forums.fogproject.org/topic/9673/when-dhcp-pxe-booting-process-goes-bad-and-you-have-no-clue

So in your setup it appears you are using the FOG server for the dhcp server on the external network? This is not specifically necessary since pfsense can function as the dhcp server and supports dynamic pxe booting (uefi / bios). There is no need for the fog server to do this. Using the pfsense router also eliminates some complexity in the setup. You won’t need the dhcp-relay service either.

I do have to say I have seem some wonkiness with uefi booting in vmware. Where as sometimes it will not pxe boot on a warm restart but it will from a powered up state. I would repeat your testing with a physical machine from the external network just once to rule out vmware acting strange. But I have seen when it toggles between the EFI Network… and the uefi firmware either the boot file is not received or a bad boot loader is sent to the target computer.

Hi George,

I’ll grab that tcpdump shortly from the fog server.

The reason I am using the fog server for dhcp is: A.) keep all the deployment configuration in one place regardless of intermediary device, B.) Let the fog dhcp filters take effect during requests, and C.) integrate DNS with the DHCP so that automatic zone entries will work.

I have actually tried with a hardware device (laptop) as mentioned previously, with the same results as the VM.

So, let me get that tcpdump real quick… I’ll edit this post with it…

Here is the PCAP fogserver-BIOS-EFI-exchange.pcap

For the capture I first powered it on with BIOS set, which booted successfully. Then I powered off the machine, set it to EFI, and then powered it back on, and no boot.

@sebastian-roth said in Issue with EFI through PFSense Firewall:

@jonhwood360 Searching the web I found this topic: https://communities.vmware.com/t5/vSphere-Host-Client-Discussions/No-PXE-Boot-from-VM-with-EFI-Using-Microsoft-WDS/m-p/972385/highlight/true#M251

I just had some luck with this… On my vm guest, the default network adapter type was e1000e. I was regularly getting stuck at “Start PXE over IPv4”. I just removed the 1000e and setup a vmxnet network card. Great Success. I am uploading an image of my guest over PXE boot right now.

Be aware that there are many non-sense answers in that topic but this particular one does sound promising to me.

With the things you have tested so far (PXE boot in BIOS mode ok and UEFI PXE boot when hooked to the same network section) I would be fairly sure your setup is ok. It sounds like a PXE routing issue within vmWare to me. Though I am not sure.

Okay so I tested the following:

replaced all virtual network card hardware types with vmxnet3 in environment - no change in status. Still can BIOS PXE, but not UEFI PXE through pfsense firewall.

tried hardware laptop (realtek network adapter) - could not EFI PXE boot. When allowed to boot to OS, gets DHCP exchange fine. PXE booting the laptop through the OP rom setting on laptop also works.

I’ll dig into the potential for routing issues in the mean time. I just find it weird that bios PXE boot works, but EFI doesn’t work through the pfSense router. its like it can’t access the TFTP for some reason, or something is getting mangled.

Since this is happening on hardware and VM, I don’t think its a VMware issue persay. I’ll dig into the vmware network settings and see if changing some of the security toggles changes something.

@sebastian-roth I’ll give the vmxnet3 card a try today and report back. Thanks for the suggestion!

One alternative for the setup is to have the fogserver vm have a local adapter on the networks that it will deploy to, however I was hoping to have the firewall between endpoints and the fog server for security (once I tighten up the ruleset).

Hi All,

I’m back with another issue!

So I am trying to get EFI pxe booting through a firewall, and it always cuts back to the setup screen.

Background:

All machines are on ESXi.

PFSense has four networks, one of which is an internal one where the FOGserver resides, and an external one where the endpoints live. NAT is turned off and routing has been configured. Firewall is currently wide open between all attached interfaces. See the following screenshots for how its configured. DHCP forwarding is also configured to point at the FOGserver

Both the FOG server and the Target have E1000E network cards configured. If the Target endpoint VM is set to boot BIOS instead of UEFI, things work as expected:

If the Target is set to EFI boot, it gets stuck in a loop between the following screens:

However, If I put the target on the same network segment as the fogserver, EFI boot works normally

I’ve attached a packet capture of the exchange between target and fog server as seen by firewall:

pfsensepacketcapture.pcap
pfsense-extnet-packetcapture.pcap

Target01 is 00:50:56:87:9E:E8
FOGserver is 00:50:56:87:21:61

Does anyone have any idea what I could be doing wrong, or something I need to configure to allow FOG EFI to traverse the firewall (BIOS is working fine).

Thank you for any and all assistance!

@sebastian-roth said in UEFI boot loop issue:

@jonhwood360 said in UEFI boot loop issue:

Updating the refind x64 binary

To which version did you update? This might be helpful for other users as well.

refind-bin-0.13.0.zip

@sebastian-roth said in UEFI boot loop issue:

@jonhwood360 Some hardware/firmware is just not playing nicely when chainload the OS from disk. While rEFInd is your best call for UEFI based systems we do see this making problems on come hardware.

Take a look at this post: https://forums.fogproject.org/post/141390

Thanks for the references. Updating the refind x64 binary and making the following changes to the conf file seemed to work for my vmware vm:

timeout 5
scanfor internal,external,manual,netboot
scan_delay 5

@sebastian-roth said in Fog client installation error - Cannot install CA certificate:

@jonhwood360 said in Fog client installation error - Cannot install CA certificate:

Got it. Install worked fine. Is the superwebsocket defined in the fogservice.exe.config supposed to listen on localhost?

Ok, fine. Communication also working I suppose?

Yes the SuperWebSocket stuff is only used for inter process communication.

Yes communication is working.

@sebastian-roth said in Fog client installation error - Cannot install CA certificate:

@jonhwood360 With this one your need to manually distribute it to the hosts or use some other way of distributed install. The next version will then auto update.

I would suggest you test this on one of your machines - just to make sure I didn’t miss anything. After installation you want to keep an eye on the fog-client log (C:\fog.log by default) to see if communication works all fine. Then go ahead and install to the other machines.

Got it. Install worked fine. Is the superwebsocket defined in the fogservice.exe.config supposed to listen on localhost?

Hi All,

I am having an issue where if no task is set the machine will boot loop indefinitely from the EFI PXE menu. I’ve tried changing the EXIT TYPE to all of the options. The only one that works is rEFInd, and then that menu doesn’t exit to the OS on disk. Am I missing something here?

Machine is UEFI, Windows 10. Will boot from hard drive normally if selected through hardware boot menu.

Thanks!

@sebastian-roth said in Fog client installation error - Cannot install CA certificate:

he next future release will work seamlessly.

Is there a specific place on the fog server I should place these, or just manually distribute them to endpoint clients?