Problem with HTTPS upgrade

[sbenson]

@sebastian-roth said in Problem with HTTPS upgrade:

Can you access this URL from a browser?

Yes

#!ipxe
set fog-ip 10.63.76.44
set fog-webroot fog
set boot-url http://${fog-ip}/${fog-webroot}
cpuid --ext 29 && set arch x86_64 || set arch i386
goto get_console
:console_set
colour --rgb 0x00567a 1 ||
colour --rgb 0x00567a 2 ||
colour --rgb 0x00567a 4 ||
cpair --foreground 7 --background 2 2 ||
goto MENU
:alt_console
cpair --background 0 1 ||
cpair --background 1 2 ||
goto MENU
:get_console
console --picture http://10.63.76.44/fog/service/ipxe/lbs-fog-bg.png --left 100 --right 80 && goto console_set || goto alt_console
:MENU
menu
colour --rgb 0xff0000 0 ||
cpair --foreground 1 1 ||
cpair --foreground 0 3 ||
cpair --foreground 4 4 ||
item --gap Host is NOT registered!
item --gap -- -------------------------------------
item fog.local Boot from hard disk
item fog.memtest Run Memtest86+
item fog.reginput Perform Full Host Registration and Inventory
item fog.reg Quick Registration and Inventory
item fog.deployimage Deploy Image
item fog.multijoin Join Multicast Session
item fog.sysinfo Client System Information (Compatibility)
choose --default fog.local --timeout 10000 target && goto ${target}
:fog.local
sanboot --no-describe --drive 0x80 || goto MENU
:fog.memtest
kernel memdisk initrd=memtest.bin iso raw
initrd memtest.bin
boot || goto MENU
:fog.reginput
kernel bzImage32 loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=127000 web=10.63.76.44/fog/ consoleblank=0 rootfstype=ext4 storage=10.63.76.44:/images/ storageip=10.63.76.44 loglevel=4 mode=manreg
imgfetch init_32.xz
boot || goto MENU
:fog.reg
kernel bzImage32 loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=127000 web=10.63.76.44/fog/ consoleblank=0 rootfstype=ext4 storage=10.63.76.44:/images/ storageip=10.63.76.44 loglevel=4 mode=autoreg
imgfetch init_32.xz
boot || goto MENU
:fog.deployimage
login
params
param mac0 ${net0/mac}
param arch ${arch}
param username ${username}
param password ${password}
param qihost 1
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
param sysuuid ${uuid}
:fog.multijoin
login
params
param mac0 ${net0/mac}
param arch ${arch}
param username ${username}
param password ${password}
param sessionJoin 1
isset ${net1/mac} && param mac1 ${net1/mac} || goto bootme
isset ${net2/mac} && param mac2 ${net2/mac} || goto bootme
param sysuuid ${uuid}
:fog.sysinfo
kernel bzImage32 loglevel=4 initrd=init_32.xz root=/dev/ram0 rw ramdisk_size=127000 web=10.63.76.44/fog/ consoleblank=0 rootfstype=ext4 storage=10.63.76.44:/images/ storageip=10.63.76.44 loglevel=4 mode=sysinfo
imgfetch init_32.xz
boot || goto MENU
:bootme
chain -ar http://10.63.76.44/fog/service/ipxe/boot.php##params ||
goto MENU
autoboot

[Sebastian Roth]

@sbenson Ok, we still have some URLs in non HTTPS style (and another one here) in the bootmenu generation PHP code. I am sure @Tom-Elliott will fix this soon.

EDIT: Tom just fixed it, you might want to upgrade to the latest working…

[Tom Elliott]

THis isn’t a very common ask, but wouuld you mind switching the git branch to working with: git checkout working?

I’ve pushed up a hopeful fix to route the httpproto in use for the tftp call to also call the node it’s going to reference.

I do plan, in the near future, to add the ability to check/uncheck if a node is expecting to be used in https mode, though how to build the binaries with this in mind is a not simple.

[sbenson]

@tom-elliott said in Problem with HTTPS upgrade:

git checkout working

Switched, Pulled, errored about changed, Deleted the packages dir, pulled.

Should I run buildIpxe?

[Tom Elliott]

@sbenson if you would like, please.

Though I’m going to imagine the pull failed to switch.

Try:

git reset --hard
git checkout working
git pull
/root/buildIpxe TRUST=/var/www/fog/management/other/ca.cert.pem
cd bin
./installfog.sh -y

[sbenson]

@tom-elliott the whole problem is the -S, so i am guessing you want a -S in there too?

EDIT: Did not work still, and is listed as http:// not https:// so default.ipxe didn’t get changed
EDIT2: Hardcoded https in /tftpboot/default.ipxe. now it booting says https but still file not found

[Sebastian Roth]

@sbenson Open https://10.63.76.44/fog/service/ipxe/boot.php in your browser again and see if there are still http:// URLs or https:// now! Post the full output here if you would like us to have a look as well.

[sbenson]

@sebastian-roth it’s all https now

[Sebastian Roth]

@sbenson Please post the output and a picture of the error on screen as well. I am pretty sure there is something that we all overlook but might notice when we see listing and picture of the error.

[sbenson]

@sebastian-roth
https://youtu.be/Cw7VgV4s3j8

[bmcalister]

@sbenson
I just went through this exact process to get HTTPS working for my fog deployment. One of the more helpful things, especially for where you are at right now, was to enable debugging on the ipxe build as the file not found error was due to a couple of other issues related to the certificate. Additionally, although it shouldn’t have been necessary, building the CA certificate into the ipxe build solved another issue I had with it. To do the above, using Tom’s script, you would pass CERT and DEBUG to it like so (I used my own CA cert and not the FOG one, so the paths were different for me, but should be correct for you. Additionally, tls is likely the only debug option that you might need, but just in case I included the others):

/root/buildIPXE CERT=/var/www/fog/management/other/ca.cert.pem TRUST=/var/www/fog/management/other/ca.cert.pem DEBUG=tls,x509,certstore

Once built, you’ll of course need to reinstall FOG with the new build. When the client boots, it should give you much more insight into what is happening and why it is reporting the file cannot be found (assuming building in the cert doesn’t solve the issue you’re having). Once it booted, I ran into an issue with the check-in script not understanding the HTTPS redirect, but this is resolved in the working version now (I’m personally on the current release version, with some custom changes that are already in the working version).

Hope it helps.

EDIT: Oh, should probably also point out that if you do try this and eventually get it working, removing the debugging after would be a good idea so your systems don’t spew out debug text on boot.

[Tom Elliott]

@bmcalister Thank you, I’m not familiar with the process to build in certificates to this helps me out just as much as the community as a whole. I didn’t even know there was a CERT argument that could be passed in.

@sbenson Hopefully this helps you too and sorry I didn’t know about this sooner.

[sbenson]

@bmcalister Well guess what, it’s working now. Something in the

/root/buildIPXE CERT=/var/www/fog/management/other/ca.cert.pem TRUST=/var/www/fog/management/other/ca.cert.pem DEBUG=tls,x509,certstore

fixed it. So, turning debugging off might not work. Deploying an image now to see if that works, which seems to be working.

[sbenson]

@tom-elliott said in Problem with HTTPS upgrade:

CERT argument

Oh I am so over this issue i am blindly copying and pasting commands that look correct at first glance. I totally missed the CERT= argument

[sbenson]

Here’s something else I have found now. Deploying an image is going at 127MB/min, where previously it was going at 8GB/min

[Tom Elliott]

@sbenson Maybe a fluke? Mind restarting the fog server see if it helps out a little?

I doubt there’s anything else causing it as the init’s have nothing to do with the network boot process (after iPXE releases the init’s and kernels) which hasn’t changed beyond the change to actually follow the redirects back to the fog server in https mode as required.

[sbenson]

@tom-elliott I will reboot it(its a VM), but I also attempted to register a host and it didn’t work 3 times.
http://www.youtube.com/watch?v=sCV7x1zKBwE

[Tom Elliott]

@sbenson No Viable mac’s to use… hmmmm. I’ll take a quick look for that.

[Tom Elliott]

So the No Viable mac’s to use comes from the host page when it’s attempting to add the primary mac to the host. I know this portion works as I have deregistered and registered multiple hosts even in the latest working tree. Do you have MAC Filters setup that maybe this machine’s mac address is being filtered out before it gets a chance to use it?

[sbenson]

@tom-elliott No, no mac filters setup, this is a new laptop that I have been testing only with fog. I just rebooted the server, and now its going at ~1GB/min