PXE Boot not working properly from Storage Node after Upgrade to 1.5.8
-
@Sebastian-Roth
Command 1:drwxr-xr-x 6 fogproject root 4096 Feb 20 11:34 . drwxr-xr-x 26 root root 4096 Feb 21 11:01 .. drwxr-xr-x 4 fogproject root 4096 Feb 20 11:34 10secdelay drwxr-xr-x 2 fogproject root 4096 Feb 21 08:23 arm64-efi -rw-r-xr-x 1 fogproject root 868 Feb 21 11:02 boot.txt -rw-r-xr-x 1 fogproject root 457 Feb 21 11:02 default.ipxe drwxr-xr-x 2 fogproject root 4096 Feb 20 11:34 i386-efi -rw-r-xr-x 1 fogproject root 227424 Feb 21 11:02 intel.efi -rw-r-xr-x 1 fogproject root 99123 Feb 21 11:02 intel.kkpxe -rw-r-xr-x 1 fogproject root 99171 Feb 21 11:02 intel.kpxe -rw-r-xr-x 1 fogproject root 99146 Feb 21 11:02 intel.pxe -rw-r-xr-x 1 fogproject root 1007360 Feb 21 11:02 ipxe.efi -rw-r-xr-x 1 fogproject root 876544 Feb 21 11:02 ipxe.iso -rw-r-xr-x 1 fogproject root 358066 Feb 21 11:02 ipxe.kkpxe -rw-r-xr-x 1 fogproject root 358114 Feb 21 11:02 ipxe.kpxe -rw-r-xr-x 1 fogproject root 357700 Feb 21 11:02 ipxe.krn -rw-r-xr-x 1 fogproject root 357700 Feb 21 11:02 ipxe.lkrn -rw-r-xr-x 1 fogproject root 358328 Feb 21 11:02 ipxe.pxe -rw-r-xr-x 1 fogproject root 1409024 Feb 21 11:02 ipxe.usb -rw-r-xr-x 1 fogproject root 123448 Feb 20 13:24 ldlinux.c32 -rw-r-xr-x 1 fogproject root 187820 Feb 20 13:24 libcom32.c32 -rw-r-xr-x 1 fogproject root 26468 Feb 20 13:24 libutil.c32 -rw-r-xr-x 1 fogproject root 26140 Feb 21 11:02 memdisk -rw-r-xr-x 1 fogproject root 29208 Feb 20 13:24 menu.c32 -rw-r-xr-x 1 fogproject root 252768 Feb 21 11:02 ncm--ecm--axge.efi -rw-r-xr-x 1 fogproject root 43210 Feb 20 13:24 pxelinux.0.old drwxr-xr-x 2 fogproject root 4096 Feb 18 08:17 pxelinux.cfg -rw-r-xr-x 1 fogproject root 226272 Feb 21 11:02 realtek.efi -rw-r-xr-x 1 fogproject root 99950 Feb 21 11:02 realtek.kkpxe -rw-r-xr-x 1 fogproject root 99998 Feb 21 11:02 realtek.kpxe -rw-r-xr-x 1 fogproject root 99968 Feb 21 11:02 realtek.pxe -rw-r-xr-x 1 fogproject root 225696 Feb 21 11:02 snp.efi -rw-r-xr-x 1 fogproject root 225952 Feb 21 11:02 snponly.efi -rw-r-xr-x 1 fogproject root 98645 Feb 21 11:02 undionly.kkpxe -rw-r-xr-x 1 fogproject root 98693 Feb 21 11:02 undionly.kpxe -rw-r-xr-x 1 fogproject root 98696 Feb 21 11:02 undionly.pxe -rw-r-xr-x 1 fogproject root 29728 Feb 20 13:24 vesamenu.c32Command 2:
SHA1 Fingerprint=52:79:6A:2A:DB:DB:B2:97:93:0E:81:45:84:1B:92:D8:BB:6D:2B:6FCommand 3:
SHA1 Fingerprint=52:79:6A:2A:DB:DB:B2:97:93:0E:81:45:84:1B:92:D8:BB:6D:2B:6F -
@Silv4n Still all good. We shall find it soon I am sure. Try these commands:
openssl x509 -noout -fingerprint -sha1 -in /var/www/html/fog/management/other/ssl/srvpublic.crt echo -n | openssl s_client -CAfile /var/www/html/fog/management/other/ca.cert.pem -connect 10.144.1.22:443 | head -
@Silv4n And here is one more command:
echo -n | openssl s_client -CAfile /var/www/html/fog/management/other/ca.cert.pem -connect 10.144.1.22:443 | openssl x509 -noout -fingerprint -
fogadmin@v-fogsrv02:~$ openssl x509 -noout -fingerprint -sha1 -in /var/www/html/fog/management/other/ssl/srvpublic.crt SHA1 Fingerprint=83:7B:9D:57:E9:11:51:83:46:20:7F:81:04:A2:23:44:A7:68:34:93 fogadmin@v-fogsrv02:~$ echo -n | openssl s_client -CAfile /var/www/html/fog/management/other/ca.cert.pem -connect 10.144.1.22:443 | head depth=1 CN = FOG Server CA verify return:1 depth=0 CN = 10.144.1.22 verify return:1 DONE CONNECTED(00000005) --- Certificate chain 0 s:CN = 10.144.1.22 i:CN = FOG Server CA 1 s:CN = FOG Server CA i:CN = FOG Server CA --- Server certificate -----BEGIN CERTIFICATE----- fogadmin@v-fogsrv02:~$ echo -n | openssl s_client -CAfile /var/www/html/fog/management/other/ca.cert.pem -connect 10.144.1.22:443 | openssl x509 -noout -fingerprint depth=1 CN = FOG Server CA verify return:1 depth=0 CN = 10.144.1.22 verify return:1 DONE SHA1 Fingerprint=83:7B:9D:57:E9:11:51:83:46:20:7F:81:04:A2:23:44:A7:68:34:93 -
@Silv4n This is really strange. All the certificates seem perfectly fine and match the fingerprints we see in the picture you posted initially. I just did a fresh clean install here and it worked out of the box. Though this is a master server only. Let me try adding a storage node and see if that makes a difference.
-
@Sebastian-Roth That’s the storage node
-
@Silv4n Let’s switch over to chat (chat bubble in the top right corner).
-
@Sebastian-Roth unfortunatly the can’t display all of it: https://imgur.com/a/OKdQzwh
-
@Silv4n Ok, unfortunately not of much help yet. Please recompile but leave out the
tls,in DEBUG parameter…make EMBED=ipxescript DEBUG=x509,validator bin/undionly.kpxe CERT=/opt/fog/snapins/ssl/CA/.fogCA.pem TRUST=/opt/fog/snapins/ssl/CA/.fogCA.pem cp bin/undionly.kpxe /tftpboot -
-
After some extended research I figured out this was caused by the build script not re-generating the trusted root part of the code compiled into the iPXE binaries. It’s really easy to fix and I pushed a fix to both
dev-branchandworking-1.6so we hopefully never run into this again.cd path/to/fogproject/bin/ touch ../../ipxe/src/crypto/rootcert.c rm /tftpboot/undionly.kkpxe ./installfog.shOn that way I learned a couple of things about iPXE booting over HTTPS and so I hope we can find most upcoming issues more quickly from now on.