Not able to TFTP boot. Invalid Argument Error
-
@Sebastian-Roth I’m kind of at a loss on this. I have tried many different combinations of the certs that I have, have tried using the one fog uses, and have tried the ones provided by iPXE. They all continue to give me invalid argument error. I added the DEBUG=tls line to the buildipxe.sh file, and tried booting. It said my cert was added to the certstore, and then ran through what I’m guessing are handshakes. In the end I still get the invalid argument error.
Is there a way to use my ssl certs for just the web server? Then all of the fog functions would use the supplied fog certs?
-
@hancocza said in Not able to TFTP boot. Invalid Argument Error:
Is there a way to use my ssl certs for just the web server?
Sorry, but no. iPXE requests information through HTTP(S) and you can’t separate that from the web interface (certs). I suppose you could setup another webserver (like nginx) on another port and run the FOG web UI on that. But that’s not a great way - don’t go there!
I am fairly sure we can get this to work for you. Let’s start by talking about the certificate. Where did you get it from. What CN is it dedicated to and does it have any extensions enabled? Please run
openssl x509 -in /path/to/custom/certificate.crt -text -nooutand post output here. A certificate is the public part of a public/private key crypto system and hence there should be no issue posting this content here. If you’d prefer you could also just send me a private message with the information. -
@Sebastian-Roth
Darn.The certificate was purchase through Go Daddy. The CN is associated with our main web server. I added the domain name of the fog server to the “other names” section in order to cover that as well. I don’t believe it has any extensions enabled.
Here is the output:
Certificate: Data: Version: 3 (0x2) Serial Number: 1807900440220026086 (0x1916f3df289644e6) Signature Algorithm: sha256WithRSAEncryption Issuer: C = US, ST = Arizona, L = Scottsdale, O = "GoDaddy.com, Inc.", OU = http://certs.godaddy.com/repository/, CN = Go Daddy Secure Certificate Authority - G2 Validity Not Before: Mar 15 18:52:13 2018 GMT Not After : Jul 13 16:53:00 2019 GMT Subject: OU = Domain Control Validated, CN = csims.clas.gvsu.edu Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public-Key: (2048 bit) Modulus: 00:e2:79:a6:3f:ac:83:5a:ec:97:ab:2c:74:95:a5: 2e:cc:30:41:f6:32:f0:4f:e5:2a:fb:c7:dd:7d:52: 9b:a0:8c:20:4e:1d:c2:7f:d5:99:ca:3b:b7:f5:ca: 05:dc:f6:85:a8:e5:99:03:95:77:6b:49:67:fe:b9: cb:78:8c:da:9f:3b:89:db:46:7a:c7:e2:ed:22:04: 84:f5:61:2f:58:9d:0a:ee:66:9e:26:40:fe:54:8f: a8:44:fd:75:16:dd:1a:24:d5:77:28:8f:f5:79:76: ab:9f:92:f2:fe:e1:f5:1e:17:e6:7f:d3:b2:07:52: 8f:60:94:28:3a:48:e6:8a:3b:57:0c:6d:4d:30:d3: 23:de:76:07:3a:f3:bf:60:ef:26:47:c4:17:45:54: 71:d7:ce:c0:e8:ef:c3:f8:42:d5:3c:47:1b:5d:97: 96:a6:2a:3d:dd:ac:d7:4e:38:03:68:f4:29:eb:80: fb:48:04:40:f6:f7:4d:19:34:a5:d8:6e:ec:5b:15: e1:97:42:17:4c:bc:c2:55:cf:44:80:ca:0d:5f:20: fb:98:c6:25:e3:12:19:a0:bb:b2:e8:b1:cf:fc:2e: 00:10:ab:e6:7b:da:85:01:6d:0b:d7:ed:53:c8:ae: d4:18:e4:52:ab:86:aa:c1:14:e3:6c:47:fe:0b:a2: 9c:db Exponent: 65537 (0x10001) X509v3 extensions: X509v3 Basic Constraints: critical CA:FALSE X509v3 Extended Key Usage: TLS Web Server Authentication, TLS Web Client Authentication X509v3 Key Usage: critical Digital Signature, Key Encipherment X509v3 CRL Distribution Points: Full Name: URI:http://crl.godaddy.com/gdig2s1-815.crl X509v3 Certificate Policies: Policy: 2.16.840.1.114413.1.7.23.1 CPS: http://certificates.godaddy.com/repository/ Policy: 2.23.140.1.2.1 Authority Information Access: OCSP - URI:http://ocsp.godaddy.com/ CA Issuers - URI:http://certificates.godaddy.com/repository/gdig2.crt X509v3 Authority Key Identifier: keyid:40:C2:BD:27:8E:CC:34:83:30:A2:33:D7:FB:6C:B3:F0:B4:2C:80:CE X509v3 Subject Alternative Name: DNS:csims.clas.gvsu.edu, DNS:www.csims.clas.gvsu.edu, DNS:fogserver.gvsu.edu, DNS:csimsweb.clas.gvsu.edu, DNS:viewlinc.clas.gvsu.edu X509v3 Subject Key Identifier: AB:E1:0F:46:89:C4:69:F7:D0:6E:C6:A1:40:E4:C5:70:7A:EA:C4:74 Signature Algorithm: sha256WithRSAEncryption 2a:1a:42:5a:21:ba:ac:81:cd:6e:7b:6f:73:55:92:5b:cc:d5: 93:de:32:7c:b7:56:53:d5:8c:7c:4c:5d:4b:6e:cd:d9:2c:8e: a4:87:39:9d:85:05:3f:c8:12:fc:c0:d2:b3:c8:de:67:15:02: b9:22:5a:6d:f1:6a:f7:12:a0:28:b8:e6:69:c6:82:c5:61:ce: ff:cf:a5:9c:f3:6c:08:51:04:c8:4f:8a:28:08:be:a4:06:d7: 54:26:91:9f:3b:76:7f:cb:7c:71:63:e3:54:f0:d4:8a:f9:a2: 06:cb:11:dd:a4:4c:5d:c1:9a:5d:bb:96:6f:13:90:56:e4:e2: bd:11:b2:83:67:c2:9f:99:9b:60:10:40:c8:8b:56:5c:3d:95: 2d:24:d8:7d:53:2d:2f:eb:fe:73:c4:54:ff:fc:73:12:51:b4: 86:16:64:56:bf:4c:99:96:d3:2e:0e:d5:33:58:84:09:6b:ce: 16:f1:b8:91:2f:cd:8b:35:52:e7:3d:d1:83:b8:5e:d9:9b:ce: 8c:f8:0f:80:5c:23:60:5a:91:07:45:e2:fc:8d:0c:ee:c6:56: 8c:76:c4:23:33:14:e6:80:56:33:d1:ef:30:b1:26:be:9f:34: ac:b9:74:ae:9c:89:d5:b7:76:f0:cb:88:bb:7c:41:fc:d5:70: 16:98:0a:a7 -
@hancocza said in Not able to TFTP boot. Invalid Argument Error:
Subject: OU = Domain Control Validated, CN = csims.clas.gvsu.edu
So this is the main subject it was created for. All the other DNS names we find in the alternative subject extension further down. I am not exactly sure about this from the top of my head but possibly iPXE does not support using the alternative subjects from a cert. I will check this out and get back to you.
-
@hancocza Oh man/woman, I have overlooked the most obvious problem with this. iPXE is connecting to the IP address instead of hostname and therefore cannot match the names in the certificate. I just had a quick look at the manual pages and seems like iPXE can handle alternative subjects, so I was wrong in my last post.
Edit /tftpboot/default.ipxe on your FOG server and make it use the DNS name instead of the IP.
-
@Sebastian-Roth the default.ipxe file already specifies the fqdn of fogserver.gvsu.edu. on 1.5.0, I would just change it to the IP and change https to http and not recompile, and it all worked fine. Since we’ve been doing this, I’ve left it as https://fogserver.gvsu.edu/…
If it could still be an issue of the two separate certs (ca and server) in the buildipxe.sh, I’m not sure what to use as the ca one. Go Daddy uses an intermediate cert instead of a CA, but that was the chain one that we weren’t sure if it’d work.
-
@hancocza Please check to see if you have also set the FQDN in FOG configuration -> FOG settings -> Web Server -> WEB HOST! Writing my last messages in the train where I don’t have good access to the code I forgot about this.
From the information you posted I would expect you’d need to use the GoDaddy Secure Server Certificate (Intermediate Certificate) - G2 as TRUST. This (or more precisely its private key) should be exactly the one that was used to sign your certificate.
-
@Sebastian-Roth said in Not able to TFTP boot. Invalid Argument Error:
@hancocza Please check to see if you have also set the FQDN in FOG configuration -> FOG settings -> Web Server -> WEB HOST! Writing my last messages in the train where I don’t have good access to the code I forgot about this.
From the information you posted I would expect you’d need to use the GoDaddy Secure Server Certificate (Intermediate Certificate) - G2 as TRUST. This (or more precisely its private key) should be exactly the one that was used to sign your certificate.
I had the IP set in the WEB HOST field. I switched that now. Also updated the TRUST parameter in buildipxe.sh to have that cert. I’ll test tomorrow and see what happens!
-
@Sebastian-Roth Just tested, still a no go. Trust line looks like this with the intermediate cert:
BUILDOPTS=“CERT=/home/fogserver/Documents/csims.clas.gvsu.edu/fogcert.pem TRUST=/home/fogserver/Documents/csims.clas.gvsu.edu/GoDaddyCA.pem”
-
@hancocza said in Not able to TFTP boot. Invalid Argument Error:
GoDaddyCA.pem
Sure this is the same one that I pointed out? Please run the following commands to check:
cd /tmp/ wget https://certs.godaddy.com/repository/gdig2.crt.pem sha256sum gdig2.crt.pem /home/fogserver/Documents/csims.clas.gvsu.edu/GoDaddyCA.pem ...See if you get the same hash for both files.
You might need to use different ones for iPXE versus the ones used for Apache. Apache needs the full trust chain I reckon. You have some major root CA certificates installed on your system already and point Apache to some chain certificates (possibly two intermediate ones) so the chain is fully trusted. For iPXE I guess (sorry, don’t have enough time to test or look it up right now) you only need to trust the one single certificate/key that was used to issue your web server certificate and I think that is https://certs.godaddy.com/repository/gdig2.crt.pem in your case.
-
@Sebastian-Roth It wasn’t the one you mentioned. I downloaded the correct one and put it in place, still got an invalid argument error. Do i need to append my certificate to the trust as well, so that it looks like:
TRUST=/home/fogserver/Desktop/gdig2.crt.pem,/home/fogserver/Documents/csims.clas.gvsu.edu/fogcert.crt
I was looking at this website when I came across that: https://ipxe.org/crypto
-
@hancocza said in Not able to TFTP boot. Invalid Argument Error:
Do i need to append my certificate to the trust as well
Don’t think you have to. But reading more about how (open)ssl certificate verification works I figured that I was wrong with one of my assumptions. The intermediate certificate is not enough as it is not selfiestick signed (only root certs are). So you’d need to specify intermediate and root cert as TRUST.
-
@hancocza Try using
TRUST=gdroot-g2.crt.pem,gdig2.crt.pemhttps://certs.godaddy.com/repository/gdroot-g2.crt
https://certs.godaddy.com/repository/gdig2.crt.pemI think you need to convert the gdroot-g2.crt to PEM format. I read that iPXE can only handle PEM but not DER cert format. Convert using
openssl x509 -inform DER -outform PEM -text -in gdroot-g2.crt -out gdroot-g2.crt.pemYou might want to check the whole chain using openssl as well:
openssl verify -CAfile gdroot-g2.crt.pem -untrusted gdig2.crt.pem fogcert.pemall untested…
EDIT: Posted on the iPXE developers mailinglist: http://lists.ipxe.org/pipermail/ipxe-devel/2018-December/006395.html
-
@Sebastian-Roth Looks like i didn’t need to convert the root certificate, it was already in pem format. I ran the openssl verify command, and it returned an ‘OK’. I’ll insert that into the trust and recompile. I wont be able to test until tomorrow morning, so I’ll update you then. Thanks for all the help!
-
@Sebastian-Roth Just tested, still the same. I ran the certstat command in the ipxe shell and it only listed the fogcert.crt certificate. Should it also have the other ones listed?
-
@hancocza I totally forgot to post new information in the forum as well. We were talking about this in chat and I sent an issue post to the iPXE forum because it looks like iPXE cannot handle big certificate TLS handshakes. http://forum.ipxe.org/showthread.php?tid=16998
Now I found the time to look into this again. Unfortunately there was no answer in the forum or the developers mailing list yet. Suppose they are all very busy.
Digging into the source a bit more I might have found what is causing this. The packet containing the TLS handshake with the certificate is probably being fragmented because of it’s size and iPXE is not made to handle this yet. Here I read about different kinds of fragmentation. I sure know of TCP fragmentation but I have not heard of TLS fragmentation yet. So I am trying to figure this out but it will probably take a bit more time.
-
@Sebastian-Roth Thanks for checking it out. In the meantime, I ended up migrating the database to another build, and then instead of installing fog with https, I stuck with a normal install, and then reconfigured the apache2 config to redirect to https and use my certificates. Everything is working correctly now, but eventually it’d be nice to run it all in https again.
-
@hancocza Unfortunately we still don’t have an answer from the (main) iPXE developers yet. From what I read between the lines the issue with certificates larger than 4096 KB is kind of known but might need a major rewrite of the code. I will keep an eye on this as I think it’s important for us and iPXE to get this fixed at some point. But I guess it won’t be any time soon. I’d like to work on this myself but have way too little time right now to dig into this part of the iPXE code way more than I have already. Will keep you posted here.
-
@hancocza Just a quick update on this. I have been following the iPXE developer mailing list over the last weeks and it’s very quiet. Well there are questions coming in as well as pull requests but the developers seem to be too busy with other projects and don’t get to even review those pull requests. Probably just the current state of affairs and I guess we will see more work on iPXE at some point again. But right now I don’t see any chance of getting help from them to fix this.
-
@Sebastian-Roth No problem. Like I said I’ve got it working without the certificate for now so no big deal. Thanks for being on top of it!