Not able to TFTP boot. Invalid Argument Error



  • Hello,

    I’m trying to tftp boot into the server to register some new hosts. When trying to boot, i get an “invalid argument” error. This only happened after upgrading to 1.5.5.

    Things to note:
    -I use the HTTPS setup switch for the installer.
    -In order to get around the 066 067 DHCP setup (dont have access to the companies DHCP server), i use a ipxe USB key to point to the fog server.
    -Server is running Ubuntu 16.04 LTS
    ![0_1544208861012_IMG_20181207_135137.jpg](Uploading 100%)



  • @Sebastian-Roth No problem. Like I said I’ve got it working without the certificate for now so no big deal. Thanks for being on top of it!


  • Developer

    @hancocza Just a quick update on this. I have been following the iPXE developer mailing list over the last weeks and it’s very quiet. Well there are questions coming in as well as pull requests but the developers seem to be too busy with other projects and don’t get to even review those pull requests. Probably just the current state of affairs and I guess we will see more work on iPXE at some point again. But right now I don’t see any chance of getting help from them to fix this.


  • Developer

    @hancocza Unfortunately we still don’t have an answer from the (main) iPXE developers yet. From what I read between the lines the issue with certificates larger than 4096 KB is kind of known but might need a major rewrite of the code. I will keep an eye on this as I think it’s important for us and iPXE to get this fixed at some point. But I guess it won’t be any time soon. I’d like to work on this myself but have way too little time right now to dig into this part of the iPXE code way more than I have already. Will keep you posted here.



  • @Sebastian-Roth Thanks for checking it out. In the meantime, I ended up migrating the database to another build, and then instead of installing fog with https, I stuck with a normal install, and then reconfigured the apache2 config to redirect to https and use my certificates. Everything is working correctly now, but eventually it’d be nice to run it all in https again.


  • Developer

    @hancocza I totally forgot to post new information in the forum as well. We were talking about this in chat and I sent an issue post to the iPXE forum because it looks like iPXE cannot handle big certificate TLS handshakes. http://forum.ipxe.org/showthread.php?tid=16998

    Now I found the time to look into this again. Unfortunately there was no answer in the forum or the developers mailing list yet. Suppose they are all very busy.

    Digging into the source a bit more I might have found what is causing this. The packet containing the TLS handshake with the certificate is probably being fragmented because of it’s size and iPXE is not made to handle this yet. Here I read about different kinds of fragmentation. I sure know of TCP fragmentation but I have not heard of TLS fragmentation yet. So I am trying to figure this out but it will probably take a bit more time.



  • @Sebastian-Roth Just tested, still the same. I ran the certstat command in the ipxe shell and it only listed the fogcert.crt certificate. Should it also have the other ones listed?



  • @Sebastian-Roth Looks like i didn’t need to convert the root certificate, it was already in pem format. I ran the openssl verify command, and it returned an ‘OK’. I’ll insert that into the trust and recompile. I wont be able to test until tomorrow morning, so I’ll update you then. Thanks for all the help!


  • Developer

    @hancocza Try using TRUST=gdroot-g2.crt.pem,gdig2.crt.pem

    https://certs.godaddy.com/repository/gdroot-g2.crt
    https://certs.godaddy.com/repository/gdig2.crt.pem

    I think you need to convert the gdroot-g2.crt to PEM format. I read that iPXE can only handle PEM but not DER cert format. Convert using openssl x509 -inform DER -outform PEM -text -in gdroot-g2.crt -out gdroot-g2.crt.pem

    You might want to check the whole chain using openssl as well: openssl verify -CAfile gdroot-g2.crt.pem -untrusted gdig2.crt.pem fogcert.pem

    all untested…


  • Developer

    @hancocza said in Not able to TFTP boot. Invalid Argument Error:

    Do i need to append my certificate to the trust as well

    Don’t think you have to. But reading more about how (open)ssl certificate verification works I figured that I was wrong with one of my assumptions. The intermediate certificate is not enough as it is not selfiestick signed (only root certs are). So you’d need to specify intermediate and root cert as TRUST.



  • @Sebastian-Roth It wasn’t the one you mentioned. I downloaded the correct one and put it in place, still got an invalid argument error. Do i need to append my certificate to the trust as well, so that it looks like:

    TRUST=/home/fogserver/Desktop/gdig2.crt.pem,/home/fogserver/Documents/csims.clas.gvsu.edu/fogcert.crt

    I was looking at this website when I came across that: https://ipxe.org/crypto


  • Developer

    @hancocza said in Not able to TFTP boot. Invalid Argument Error:

    GoDaddyCA.pem

    Sure this is the same one that I pointed out? Please run the following commands to check:

    cd /tmp/
    wget https://certs.godaddy.com/repository/gdig2.crt.pem
    sha256sum gdig2.crt.pem /home/fogserver/Documents/csims.clas.gvsu.edu/GoDaddyCA.pem
    ...
    

    See if you get the same hash for both files.

    You might need to use different ones for iPXE versus the ones used for Apache. Apache needs the full trust chain I reckon. You have some major root CA certificates installed on your system already and point Apache to some chain certificates (possibly two intermediate ones) so the chain is fully trusted. For iPXE I guess (sorry, don’t have enough time to test or look it up right now) you only need to trust the one single certificate/key that was used to issue your web server certificate and I think that is https://certs.godaddy.com/repository/gdig2.crt.pem in your case.



  • @Sebastian-Roth Just tested, still a no go. Trust line looks like this with the intermediate cert:

    BUILDOPTS=“CERT=/home/fogserver/Documents/csims.clas.gvsu.edu/fogcert.pem TRUST=/home/fogserver/Documents/csims.clas.gvsu.edu/GoDaddyCA.pem”



  • @Sebastian-Roth said in Not able to TFTP boot. Invalid Argument Error:

    @hancocza Please check to see if you have also set the FQDN in FOG configuration -> FOG settings -> Web Server -> WEB HOST! Writing my last messages in the train where I don’t have good access to the code I forgot about this.

    From the information you posted I would expect you’d need to use the GoDaddy Secure Server Certificate (Intermediate Certificate) - G2 as TRUST. This (or more precisely its private key) should be exactly the one that was used to sign your certificate.

    I had the IP set in the WEB HOST field. I switched that now. Also updated the TRUST parameter in buildipxe.sh to have that cert. I’ll test tomorrow and see what happens!


  • Developer

    @hancocza Please check to see if you have also set the FQDN in FOG configuration -> FOG settings -> Web Server -> WEB HOST! Writing my last messages in the train where I don’t have good access to the code I forgot about this.

    From the information you posted I would expect you’d need to use the GoDaddy Secure Server Certificate (Intermediate Certificate) - G2 as TRUST. This (or more precisely its private key) should be exactly the one that was used to sign your certificate.



  • @Sebastian-Roth the default.ipxe file already specifies the fqdn of fogserver.gvsu.edu. on 1.5.0, I would just change it to the IP and change https to http and not recompile, and it all worked fine. Since we’ve been doing this, I’ve left it as https://fogserver.gvsu.edu/

    If it could still be an issue of the two separate certs (ca and server) in the buildipxe.sh, I’m not sure what to use as the ca one. Go Daddy uses an intermediate cert instead of a CA, but that was the chain one that we weren’t sure if it’d work.


  • Developer

    @hancocza Oh man/woman, I have overlooked the most obvious problem with this. iPXE is connecting to the IP address instead of hostname and therefore cannot match the names in the certificate. I just had a quick look at the manual pages and seems like iPXE can handle alternative subjects, so I was wrong in my last post.

    Edit /tftpboot/default.ipxe on your FOG server and make it use the DNS name instead of the IP.


  • Developer

    @hancocza said in Not able to TFTP boot. Invalid Argument Error:

    Subject: OU = Domain Control Validated, CN = csims.clas.gvsu.edu

    So this is the main subject it was created for. All the other DNS names we find in the alternative subject extension further down. I am not exactly sure about this from the top of my head but possibly iPXE does not support using the alternative subjects from a cert. I will check this out and get back to you.



  • @Sebastian-Roth
    Darn.

    The certificate was purchase through Go Daddy. The CN is associated with our main web server. I added the domain name of the fog server to the “other names” section in order to cover that as well. I don’t believe it has any extensions enabled.

    Here is the output:

    Certificate:
        Data:
            Version: 3 (0x2)
            Serial Number: 1807900440220026086 (0x1916f3df289644e6)
            Signature Algorithm: sha256WithRSAEncryption
            Issuer: C = US, ST = Arizona, L = Scottsdale, O = "GoDaddy.com, Inc.", OU = http://certs.godaddy.com/repository/, CN = Go Daddy Secure Certificate Authority - G2
            Validity
                Not Before: Mar 15 18:52:13 2018 GMT
                Not After : Jul 13 16:53:00 2019 GMT
            Subject: OU = Domain Control Validated, CN = csims.clas.gvsu.edu
            Subject Public Key Info:
                Public Key Algorithm: rsaEncryption
                    RSA Public-Key: (2048 bit)
                    Modulus:
                        00:e2:79:a6:3f:ac:83:5a:ec:97:ab:2c:74:95:a5:
                        2e:cc:30:41:f6:32:f0:4f:e5:2a:fb:c7:dd:7d:52:
                        9b:a0:8c:20:4e:1d:c2:7f:d5:99:ca:3b:b7:f5:ca:
                        05:dc:f6:85:a8:e5:99:03:95:77:6b:49:67:fe:b9:
                        cb:78:8c:da:9f:3b:89:db:46:7a:c7:e2:ed:22:04:
                        84:f5:61:2f:58:9d:0a:ee:66:9e:26:40:fe:54:8f:
                        a8:44:fd:75:16:dd:1a:24:d5:77:28:8f:f5:79:76:
                        ab:9f:92:f2:fe:e1:f5:1e:17:e6:7f:d3:b2:07:52:
                        8f:60:94:28:3a:48:e6:8a:3b:57:0c:6d:4d:30:d3:
                        23:de:76:07:3a:f3:bf:60:ef:26:47:c4:17:45:54:
                        71:d7:ce:c0:e8:ef:c3:f8:42:d5:3c:47:1b:5d:97:
                        96:a6:2a:3d:dd:ac:d7:4e:38:03:68:f4:29:eb:80:
                        fb:48:04:40:f6:f7:4d:19:34:a5:d8:6e:ec:5b:15:
                        e1:97:42:17:4c:bc:c2:55:cf:44:80:ca:0d:5f:20:
                        fb:98:c6:25:e3:12:19:a0:bb:b2:e8:b1:cf:fc:2e:
                        00:10:ab:e6:7b:da:85:01:6d:0b:d7:ed:53:c8:ae:
                        d4:18:e4:52:ab:86:aa:c1:14:e3:6c:47:fe:0b:a2:
                        9c:db
                    Exponent: 65537 (0x10001)
            X509v3 extensions:
                X509v3 Basic Constraints: critical
                    CA:FALSE
                X509v3 Extended Key Usage: 
                    TLS Web Server Authentication, TLS Web Client Authentication
                X509v3 Key Usage: critical
                    Digital Signature, Key Encipherment
                X509v3 CRL Distribution Points: 
    
                    Full Name:
                      URI:http://crl.godaddy.com/gdig2s1-815.crl
    
                X509v3 Certificate Policies: 
                    Policy: 2.16.840.1.114413.1.7.23.1
                      CPS: http://certificates.godaddy.com/repository/
                    Policy: 2.23.140.1.2.1
    
                Authority Information Access: 
                    OCSP - URI:http://ocsp.godaddy.com/
                    CA Issuers - URI:http://certificates.godaddy.com/repository/gdig2.crt
    
                X509v3 Authority Key Identifier: 
                    keyid:40:C2:BD:27:8E:CC:34:83:30:A2:33:D7:FB:6C:B3:F0:B4:2C:80:CE
    
                X509v3 Subject Alternative Name: 
                    DNS:csims.clas.gvsu.edu, DNS:www.csims.clas.gvsu.edu, DNS:fogserver.gvsu.edu, DNS:csimsweb.clas.gvsu.edu, DNS:viewlinc.clas.gvsu.edu
                X509v3 Subject Key Identifier: 
                    AB:E1:0F:46:89:C4:69:F7:D0:6E:C6:A1:40:E4:C5:70:7A:EA:C4:74
        Signature Algorithm: sha256WithRSAEncryption
             2a:1a:42:5a:21:ba:ac:81:cd:6e:7b:6f:73:55:92:5b:cc:d5:
             93:de:32:7c:b7:56:53:d5:8c:7c:4c:5d:4b:6e:cd:d9:2c:8e:
             a4:87:39:9d:85:05:3f:c8:12:fc:c0:d2:b3:c8:de:67:15:02:
             b9:22:5a:6d:f1:6a:f7:12:a0:28:b8:e6:69:c6:82:c5:61:ce:
             ff:cf:a5:9c:f3:6c:08:51:04:c8:4f:8a:28:08:be:a4:06:d7:
             54:26:91:9f:3b:76:7f:cb:7c:71:63:e3:54:f0:d4:8a:f9:a2:
             06:cb:11:dd:a4:4c:5d:c1:9a:5d:bb:96:6f:13:90:56:e4:e2:
             bd:11:b2:83:67:c2:9f:99:9b:60:10:40:c8:8b:56:5c:3d:95:
             2d:24:d8:7d:53:2d:2f:eb:fe:73:c4:54:ff:fc:73:12:51:b4:
             86:16:64:56:bf:4c:99:96:d3:2e:0e:d5:33:58:84:09:6b:ce:
             16:f1:b8:91:2f:cd:8b:35:52:e7:3d:d1:83:b8:5e:d9:9b:ce:
             8c:f8:0f:80:5c:23:60:5a:91:07:45:e2:fc:8d:0c:ee:c6:56:
             8c:76:c4:23:33:14:e6:80:56:33:d1:ef:30:b1:26:be:9f:34:
             ac:b9:74:ae:9c:89:d5:b7:76:f0:cb:88:bb:7c:41:fc:d5:70:
             16:98:0a:a7
    

  • Developer

    @hancocza said in Not able to TFTP boot. Invalid Argument Error:

    Is there a way to use my ssl certs for just the web server?

    Sorry, but no. iPXE requests information through HTTP(S) and you can’t separate that from the web interface (certs). I suppose you could setup another webserver (like nginx) on another port and run the FOG web UI on that. But that’s not a great way - don’t go there!

    I am fairly sure we can get this to work for you. Let’s start by talking about the certificate. Where did you get it from. What CN is it dedicated to and does it have any extensions enabled? Please run openssl x509 -in /path/to/custom/certificate.crt -text -noout and post output here. A certificate is the public part of a public/private key crypto system and hence there should be no issue posting this content here. If you’d prefer you could also just send me a private message with the information.


Log in to reply
 

469
Online

6.2k
Users

13.6k
Topics

128.0k
Posts