Problem with HTTPS upgrade



  • Server
    • FOG Version: 1.5.0-RC-4
    • OS: Ubuntu 16.04
    Description

    Getting an error saying
    http://10.x.x.x/fog/service/ipxe/boot.php…No such file or directory.

    I have SSL enabled on fog so it should be contacting https://. I am not sure if the ipxe boot agent honors HTTP 302 responses. Is this something that should be hard coded in the boot image since it comes from an https installation?

    I have confirmed that the file does exist in https, and http provides a 302.


  • Developer

    Ok lets try to get this sorted…

    @sbenson said:

    Let me just give you a rundown of everything that happened.
    Yesterday
    Fog 1.5.0 RC7 With SSL with ipxe built with CERT and DEBUG
    Image deploy: 1GB/min(Slow)
    Host registration fails

    This is not nice but I can’t see the connection between either RC version or SSL iPXE to a slower deploy speed (which is simply a point of NFS or udp-cast if you do multicast). I’ve just done capture and deploy test (1.5.0 RC8) in my VM environment (SSL iPXE) and I get around 4-5 GB/min which I think is ok as all the VMs are running on a lenovo laptop - no server hardware or anything.

    This morning
    Fog 1.5.0 RC8 without SSL with ipxe built with CERT
    Host registration succeded(PC-45)
    Image deploy: 8.25GB/min

    Fine, that’s good.

    Fog 1.5.0 RC8 With SSL with ipxe built with CERT
    registration detection fails(doesn’t see that it is PC-45)
    Host registration fails
    attempting to deploy and it takes the username/pass and just drops me back to fog

    Yeah, sorry for that, we are still missing the HTTPS URL scheme in some places. But we’re getting there. You can download a patched version of the init.xz file here for now (SSL only!!!). But keep in mind that you need to put in that init.xz again after re-running the installer. Sorry for the inconvenience!

    I’ll talk to @Tom-Elliott to add the fixes in the near future. The curl following redirects does not work as intended. See the manpage:

                  When  curl follows a redirect and the request is not a plain GET
                  (for example POST or PUT), it will do the following request with
                  a GET if the HTTP response was 301, 302, or 303. If the response
                  code was any other 3xx code, curl  will  re-send  the  following
                  request using the same unmodified method.
    
                  You  can  tell  curl to not change the non-GET request method to
                  GET after a 30x response by  using  the  dedicated  options  for
                  that: --post301, --post302 and -post303.
    

    Sure we could add the --post302 option as well but I’d think we better make all those URLs https:// instead of relying on the redirect (which costs an extra round trip anyway!).


  • Developer

    @sbenson I am sorry for taking so long to answer. Other things kept popping up and so I didn’t get to sit down and rework this from scratch. I’ve done it now and pushed a couple of fixes that make the installer do a proper job setting FOG up with SSL. I also added a script that does the iPXE compiling for you. Tom and I are still working on nicely integrating this with the installer.

    Not much news for you so far I am afraid. So now I can actually get to what’s bugging you. The speed issue. I’ll look into this fairly soon! Promise!



  • @tom-elliott any update?



  • Updates, the fix for enabling https works

    ./buildIpxe CERT=/var/www/fog/management/other/ca.cert.pem TRUST=/var/www/fog/management/other/ca.cert.pem DEBUG=tls,x509,certstore
    

    Let me just give you a rundown of everything that happened.
    Yesterday
    Fog 1.5.0 RC7 With SSL with ipxe built with CERT and DEBUG
    Image deploy: 1GB/min(Slow)
    Host registration fails

    This morning
    Fog 1.5.0 RC8 without SSL with ipxe built with CERT
    Host registration succeded(PC-45)
    Image deploy: 8.25GB/min

    Fog 1.5.0 RC8 With SSL with ipxe built with CERT
    registration detection fails(doesn’t see that it is PC-45)
    Host registration fails
    attempting to deploy and it takes the username/pass and just drops me back to fog

    If you want any more information I can test this all



  • @sbenson Nope still slow

    EDIT: New patch, 3 different ports. Could it be because the content wasn’t actually over https previously… Registering still doesn’t work. Though I am still working off of the “Working” trunk. I will try 1.5.0 RC8 tomorrow with the debugging turned off.



  • @tom-elliott I’ll swap it out for another, it was working fine the last time I imaged a machine. I’ll let you know


  • Senior Developer

    @sbenson Maybe patch cable is iffy? There’s really loads of variables that can cause speed issues and nothing has been changed in regards to partclone with the exception of hfsplus to patch a 32 bit integer overflow potential. So if it’s going so slow I would say start by checking the network cable, switch it’s connected to, etc…

    I don’t know anything else on the server beyond image replication, but that only occurs if you have multiple storage nodes (which I’m just guessing you don’t at this point).

    The no suitable mac’s issue might be due to the https switchover, though everything else appears to be working as expected. I know the mac address is “usable” so I’ll see what I can do to try to replicate it.



  • @tom-elliott No, no mac filters setup, this is a new laptop that I have been testing only with fog. I just rebooted the server, and now its going at ~1GB/min


  • Senior Developer

    So the No Viable mac’s to use comes from the host page when it’s attempting to add the primary mac to the host. I know this portion works as I have deregistered and registered multiple hosts even in the latest working tree. Do you have MAC Filters setup that maybe this machine’s mac address is being filtered out before it gets a chance to use it?


  • Senior Developer

    @sbenson No Viable mac’s to use… hmmmm. I’ll take a quick look for that.



  • @tom-elliott I will reboot it(its a VM), but I also attempted to register a host and it didn’t work 3 times.
    http://www.youtube.com/watch?v=sCV7x1zKBwE


  • Senior Developer

    @sbenson Maybe a fluke? Mind restarting the fog server see if it helps out a little?

    I doubt there’s anything else causing it as the init’s have nothing to do with the network boot process (after iPXE releases the init’s and kernels) which hasn’t changed beyond the change to actually follow the redirects back to the fog server in https mode as required.



  • Here’s something else I have found now. Deploying an image is going at 127MB/min, where previously it was going at 8GB/min



  • @tom-elliott said in Problem with HTTPS upgrade:

    CERT argument

    Oh I am so over this issue i am blindly copying and pasting commands that look correct at first glance. I totally missed the CERT= argument



  • @bmcalister Well guess what, it’s working now. Something in the

    /root/buildIPXE CERT=/var/www/fog/management/other/ca.cert.pem TRUST=/var/www/fog/management/other/ca.cert.pem DEBUG=tls,x509,certstore
    

    fixed it. So, turning debugging off might not work. Deploying an image now to see if that works, which seems to be working.


  • Senior Developer

    @bmcalister Thank you, I’m not familiar with the process to build in certificates to this helps me out just as much as the community as a whole. I didn’t even know there was a CERT argument that could be passed in.

    @sbenson Hopefully this helps you too and sorry I didn’t know about this sooner.



  • @sbenson
    I just went through this exact process to get HTTPS working for my fog deployment. One of the more helpful things, especially for where you are at right now, was to enable debugging on the ipxe build as the file not found error was due to a couple of other issues related to the certificate. Additionally, although it shouldn’t have been necessary, building the CA certificate into the ipxe build solved another issue I had with it. To do the above, using Tom’s script, you would pass CERT and DEBUG to it like so (I used my own CA cert and not the FOG one, so the paths were different for me, but should be correct for you. Additionally, tls is likely the only debug option that you might need, but just in case I included the others):

    /root/buildIPXE CERT=/var/www/fog/management/other/ca.cert.pem TRUST=/var/www/fog/management/other/ca.cert.pem DEBUG=tls,x509,certstore
    

    Once built, you’ll of course need to reinstall FOG with the new build. When the client boots, it should give you much more insight into what is happening and why it is reporting the file cannot be found (assuming building in the cert doesn’t solve the issue you’re having). Once it booted, I ran into an issue with the check-in script not understanding the HTTPS redirect, but this is resolved in the working version now (I’m personally on the current release version, with some custom changes that are already in the working version).

    Hope it helps.

    EDIT: Oh, should probably also point out that if you do try this and eventually get it working, removing the debugging after would be a good idea so your systems don’t spew out debug text on boot.




  • Developer

    @sbenson Please post the output and a picture of the error on screen as well. I am pretty sure there is something that we all overlook but might notice when we see listing and picture of the error.


Log in to reply
 

362
Online

38982
Users

10712
Topics

101678
Posts

Looks like your connection to FOG Project was lost, please wait while we try to reconnect.