Not able to TFTP boot. Invalid Argument Error

george1421 · Dec 7, 2018, 10:24 PM

@hancocza OK that tells us that boot.php code is working on the fog server. That is the complete iPXE boot menu. So we know its not the fog server causing you pain.

So you haven’t changed the fog default.ipxe, boot.php is working because the menu is created when you call it from a browser.

so the question is what is being passed that is causing boot.php to return an invalid ipxe menu.

The next step is to usb boot again and get the error. Then inspect the apache access log, error log, and php-fpm error log files. These are typically in /var/log directory.

george1421 · Dec 7, 2018, 10:25 PM

@george1421 Just wondering is this fog client computer you are trying to USB boot registered in FOG? If so delete the registration and see if it throws the same error.

Sebastian Roth · Dec 7, 2018, 11:30 PM

@hancocza Your picture shows the iPXE error http://ipxe.org/err/1c0de8 - see this is definitely talking about TLS being the issue. Now having a closer look at the iPXE menu output you posted from “boot.php” I see that i does a chain https://... right at the end as well as console --picture https://... and others which all will likely fail if you don’t have iPXE binaries compiled with your certificate in it.

Those URLs are generated and the http(s) part is derived from the client request itself. So if a client asks for http:// all the URLs in the bootmenu will also be http:// - but we generate an apache config to redirect HTTP to HTTPS and therefore you end up with https:// URLs in the iPXE boot menu. Either you disable the forced HTTPS redirect or you look into compiling correct iPXE binaries. The later should be real easy using the script as I suggested.

hancocza · Dec 10, 2018, 11:54 AM

@Sebastian-Roth I ran the recompiling script last friday. At the end it says it needs to go through the FOG installation again, so i did. After that, it still gave the same error. Should I be allowing it to go through the FOG installation again or does that basically reset the iPXE binaries again?

hancocza · Dec 10, 2018, 12:04 PM

@george1421 said in Not able to TFTP boot. Invalid Argument Error:

@george1421 Just wondering is this fog client computer you are trying to USB boot registered in FOG? If so delete the registration and see if it throws the same error.

This happened on both laptops that were registered as well as a few that were not.

I also checked all of the logs after attempting to boot, and it didn’t update any of those logs. I assume that means it’s in line with what Sebastian said about the TLS error.

Sebastian Roth · Dec 10, 2018, 2:51 PM

@hancocza said in Not able to TFTP boot. Invalid Argument Error:

Should I be allowing it to go through the FOG installation again or does that basically reset the iPXE binaries again?

When you run the buidlipxe.sh Script it compiles new binaries including the correct SSL cert and puts them in the “installer directory”. From now on you can run the FOG installer as often as you want, it should always install those HTTPS-enabled iPXE binaries!

hancocza · Dec 10, 2018, 2:53 PM

@Sebastian-Roth Hmm… I did that and still get the same error after. Same error code as well, pointing at the TLS handshake.

Sebastian Roth · Dec 10, 2018, 3:06 PM

@hancocza Sure you don’t get any error from the iPXE build script?

Do you use a custom SSL certificate for the FOG web server?

hancocza · Dec 10, 2018, 3:08 PM

@Sebastian-Roth I don’t think i get one from the script. How could i check?

I do use a custom SSL cert for the web server. In the apache2 site config, it points to the correct cert and key. That’s also why I get the password prompts during installation.

Sebastian Roth · Dec 10, 2018, 3:19 PM

@hancocza said in Not able to TFTP boot. Invalid Argument Error:

I do use a custom SSL cert for the web server.

Ah sorry, I missed that. Then you need to adjust the utils/FOGiPXE/buildipxe.sh Script to point to those custom certs as well. Edit that file and put in the correct paths for CA cert and server cert:

#!/bin/bash
BUILDOPTS="TRUST=/var/www/fog/management/other/ca.cert.pem CERT=/var/www/fog/management/other/ca.cert.pem"
IPXEGIT="https://git.ipxe.org/ipxe.git"
...

This is the default. Change to where you have your custom cert files and compile again! Then rerun the installer as well.

hancocza · Dec 10, 2018, 3:52 PM

@Sebastian-Roth Tried that with our cert that we use. I replaced both the path for both TRUST= and CERT= with the certificate path, compiled and ran the installer. I tried it with the default.ipxe unchanged, as well as with the IP in place of the FQDN in the chain line.

My buildipxe.sh looks like:

#!/bin/bash
BUILDOPTS=“TRUST=/home/fogserver/Documents/fogcert.crt CERT=/home/fogserver/Documents/fogcert.crt”
IPXEGIT=“https://git.ipxe.org/ipxe.git”

The chain line of the default.ipxe looks like this:

chain https://xxx.xxx.xxx.xxx/fog/service/ipxe/boot.php##params

Am I messing up the placement of the certificate? Apache asks for three separate certificate components (certificate, key, chain), am I supposed to have two different certificates specified in the buildipxe.sh?
…

Sebastian Roth · Dec 10, 2018, 4:30 PM

@hancocza said in Not able to TFTP boot. Invalid Argument Error:

am I supposed to have two different certificates specified in the buildipxe.sh?

Yes! The one going in TRUST is the CA cert or possibly what you use in Chain parameter in the apache config. If it is a real chain of two or more certs in the chain I am not sure if iPXE can handle it this way. Give it a try. The CERT is just that, the webserver certificate.

By the way, keeping the certs in the home directory might not be a wise idea. Someone comes along and cleans up the home dir one day and boom all is gone.

hancocza · Dec 11, 2018, 12:25 PM

@Sebastian-Roth I’m kind of at a loss on this. I have tried many different combinations of the certs that I have, have tried using the one fog uses, and have tried the ones provided by iPXE. They all continue to give me invalid argument error. I added the DEBUG=tls line to the buildipxe.sh file, and tried booting. It said my cert was added to the certstore, and then ran through what I’m guessing are handshakes. In the end I still get the invalid argument error.

Is there a way to use my ssl certs for just the web server? Then all of the fog functions would use the supplied fog certs?

Sebastian Roth · Dec 11, 2018, 3:36 PM

@hancocza said in Not able to TFTP boot. Invalid Argument Error:

Is there a way to use my ssl certs for just the web server?

Sorry, but no. iPXE requests information through HTTP(S) and you can’t separate that from the web interface (certs). I suppose you could setup another webserver (like nginx) on another port and run the FOG web UI on that. But that’s not a great way - don’t go there!

I am fairly sure we can get this to work for you. Let’s start by talking about the certificate. Where did you get it from. What CN is it dedicated to and does it have any extensions enabled? Please run openssl x509 -in /path/to/custom/certificate.crt -text -noout and post output here. A certificate is the public part of a public/private key crypto system and hence there should be no issue posting this content here. If you’d prefer you could also just send me a private message with the information.

hancocza · Dec 11, 2018, 4:00 PM

@Sebastian-Roth
Darn.

The certificate was purchase through Go Daddy. The CN is associated with our main web server. I added the domain name of the fog server to the “other names” section in order to cover that as well. I don’t believe it has any extensions enabled.

Here is the output:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1807900440220026086 (0x1916f3df289644e6)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, ST = Arizona, L = Scottsdale, O = "GoDaddy.com, Inc.", OU = http://certs.godaddy.com/repository/, CN = Go Daddy Secure Certificate Authority - G2
        Validity
            Not Before: Mar 15 18:52:13 2018 GMT
            Not After : Jul 13 16:53:00 2019 GMT
        Subject: OU = Domain Control Validated, CN = csims.clas.gvsu.edu
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:e2:79:a6:3f:ac:83:5a:ec:97:ab:2c:74:95:a5:
                    2e:cc:30:41:f6:32:f0:4f:e5:2a:fb:c7:dd:7d:52:
                    9b:a0:8c:20:4e:1d:c2:7f:d5:99:ca:3b:b7:f5:ca:
                    05:dc:f6:85:a8:e5:99:03:95:77:6b:49:67:fe:b9:
                    cb:78:8c:da:9f:3b:89:db:46:7a:c7:e2:ed:22:04:
                    84:f5:61:2f:58:9d:0a:ee:66:9e:26:40:fe:54:8f:
                    a8:44:fd:75:16:dd:1a:24:d5:77:28:8f:f5:79:76:
                    ab:9f:92:f2:fe:e1:f5:1e:17:e6:7f:d3:b2:07:52:
                    8f:60:94:28:3a:48:e6:8a:3b:57:0c:6d:4d:30:d3:
                    23:de:76:07:3a:f3:bf:60:ef:26:47:c4:17:45:54:
                    71:d7:ce:c0:e8:ef:c3:f8:42:d5:3c:47:1b:5d:97:
                    96:a6:2a:3d:dd:ac:d7:4e:38:03:68:f4:29:eb:80:
                    fb:48:04:40:f6:f7:4d:19:34:a5:d8:6e:ec:5b:15:
                    e1:97:42:17:4c:bc:c2:55:cf:44:80:ca:0d:5f:20:
                    fb:98:c6:25:e3:12:19:a0:bb:b2:e8:b1:cf:fc:2e:
                    00:10:ab:e6:7b:da:85:01:6d:0b:d7:ed:53:c8:ae:
                    d4:18:e4:52:ab:86:aa:c1:14:e3:6c:47:fe:0b:a2:
                    9c:db
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 CRL Distribution Points: 

                Full Name:
                  URI:http://crl.godaddy.com/gdig2s1-815.crl

            X509v3 Certificate Policies: 
                Policy: 2.16.840.1.114413.1.7.23.1
                  CPS: http://certificates.godaddy.com/repository/
                Policy: 2.23.140.1.2.1

            Authority Information Access: 
                OCSP - URI:http://ocsp.godaddy.com/
                CA Issuers - URI:http://certificates.godaddy.com/repository/gdig2.crt

            X509v3 Authority Key Identifier: 
                keyid:40:C2:BD:27:8E:CC:34:83:30:A2:33:D7:FB:6C:B3:F0:B4:2C:80:CE

            X509v3 Subject Alternative Name: 
                DNS:csims.clas.gvsu.edu, DNS:www.csims.clas.gvsu.edu, DNS:fogserver.gvsu.edu, DNS:csimsweb.clas.gvsu.edu, DNS:viewlinc.clas.gvsu.edu
            X509v3 Subject Key Identifier: 
                AB:E1:0F:46:89:C4:69:F7:D0:6E:C6:A1:40:E4:C5:70:7A:EA:C4:74
    Signature Algorithm: sha256WithRSAEncryption
         2a:1a:42:5a:21:ba:ac:81:cd:6e:7b:6f:73:55:92:5b:cc:d5:
         93:de:32:7c:b7:56:53:d5:8c:7c:4c:5d:4b:6e:cd:d9:2c:8e:
         a4:87:39:9d:85:05:3f:c8:12:fc:c0:d2:b3:c8:de:67:15:02:
         b9:22:5a:6d:f1:6a:f7:12:a0:28:b8:e6:69:c6:82:c5:61:ce:
         ff:cf:a5:9c:f3:6c:08:51:04:c8:4f:8a:28:08:be:a4:06:d7:
         54:26:91:9f:3b:76:7f:cb:7c:71:63:e3:54:f0:d4:8a:f9:a2:
         06:cb:11:dd:a4:4c:5d:c1:9a:5d:bb:96:6f:13:90:56:e4:e2:
         bd:11:b2:83:67:c2:9f:99:9b:60:10:40:c8:8b:56:5c:3d:95:
         2d:24:d8:7d:53:2d:2f:eb:fe:73:c4:54:ff:fc:73:12:51:b4:
         86:16:64:56:bf:4c:99:96:d3:2e:0e:d5:33:58:84:09:6b:ce:
         16:f1:b8:91:2f:cd:8b:35:52:e7:3d:d1:83:b8:5e:d9:9b:ce:
         8c:f8:0f:80:5c:23:60:5a:91:07:45:e2:fc:8d:0c:ee:c6:56:
         8c:76:c4:23:33:14:e6:80:56:33:d1:ef:30:b1:26:be:9f:34:
         ac:b9:74:ae:9c:89:d5:b7:76:f0:cb:88:bb:7c:41:fc:d5:70:
         16:98:0a:a7

Sebastian Roth · Dec 11, 2018, 4:31 PM

@hancocza said in Not able to TFTP boot. Invalid Argument Error:

Subject: OU = Domain Control Validated, CN = csims.clas.gvsu.edu

So this is the main subject it was created for. All the other DNS names we find in the alternative subject extension further down. I am not exactly sure about this from the top of my head but possibly iPXE does not support using the alternative subjects from a cert. I will check this out and get back to you.

Sebastian Roth · Dec 11, 2018, 4:44 PM

@hancocza Oh man/woman, I have overlooked the most obvious problem with this. iPXE is connecting to the IP address instead of hostname and therefore cannot match the names in the certificate. I just had a quick look at the manual pages and seems like iPXE can handle alternative subjects, so I was wrong in my last post.

Edit /tftpboot/default.ipxe on your FOG server and make it use the DNS name instead of the IP.

hancocza · Dec 11, 2018, 5:08 PM

@Sebastian-Roth the default.ipxe file already specifies the fqdn of fogserver.gvsu.edu. on 1.5.0, I would just change it to the IP and change https to http and not recompile, and it all worked fine. Since we’ve been doing this, I’ve left it as https://fogserver.gvsu.edu/…

If it could still be an issue of the two separate certs (ca and server) in the buildipxe.sh, I’m not sure what to use as the ca one. Go Daddy uses an intermediate cert instead of a CA, but that was the chain one that we weren’t sure if it’d work.

Sebastian Roth · Dec 11, 2018, 5:49 PM

@hancocza Please check to see if you have also set the FQDN in FOG configuration -> FOG settings -> Web Server -> WEB HOST! Writing my last messages in the train where I don’t have good access to the code I forgot about this.

From the information you posted I would expect you’d need to use the GoDaddy Secure Server Certificate (Intermediate Certificate) - G2 as TRUST. This (or more precisely its private key) should be exactly the one that was used to sign your certificate.

hancocza · Dec 11, 2018, 6:42 PM

@Sebastian-Roth said in Not able to TFTP boot. Invalid Argument Error:

@hancocza Please check to see if you have also set the FQDN in FOG configuration -> FOG settings -> Web Server -> WEB HOST! Writing my last messages in the train where I don’t have good access to the code I forgot about this.

From the information you posted I would expect you’d need to use the GoDaddy Secure Server Certificate (Intermediate Certificate) - G2 as TRUST. This (or more precisely its private key) should be exactly the one that was used to sign your certificate.

I had the IP set in the WEB HOST field. I switched that now. Also updated the TRUST parameter in buildipxe.sh to have that cert. I’ll test tomorrow and see what happens!

Not able to TFTP boot. Invalid Argument Error

287

12.0k

17.3k

155.2k