Error decrypting LUKS partition prior to capture/imaging
-
@george1421 thanks for uploading this! I just got a chance to try it out. It fails but a different error this time, so I think we’re getting closer.
[Mon Oct 21 root@fogclient /]# cryptsetup luksOpen /dev/md126p3 crypt Enter passphrase for /dev/md126p3: device-mapper: table: 251:0: crypt: Error allocating crypto tfm reload ioctl on failed: No such file or directory Failed to setup dm-crypt key mapping for device /dev/md126p3 Check that the kernel supports aes-xts-plain64 cipher (check syslog for more info).From /var/log/messages:
Oct 21 21:03:16 fogclient user.err kernel: device-mapper: table: 251:0: crypt: Error allocating crypto tfm Oct 21 21:03:16 fogclient user.warn kernel: device-mapper: ioctl: error adding target to tableDoing some research, it looks like a module may be missing (“No such file or directory”). Could it be that the module for the cipher is missing? I’m currently using cryptsetup default options (which uses aes-xts-plain64 as cipher). When I try
cat crypto | grep aeson this FOS build, I only seeaes-genericwhereas I see ~20 options when doing the same on my Ubuntu server (includingxts-aes-aesni).Let me know if I can do anything to help debug further.
Some links referencing similar error messages out there:
https://www.raspberrypi.org/forums/viewtopic.php?t=60278
https://bugs.centos.org/view.php?id=9311 -
@humoss233 OK I do see some crypto parameters not enabled in the kernel.
CONFIG_CRYPTO_AES=y # CONFIG_CRYPTO_AES_TI is not set # CONFIG_CRYPTO_AES_X86_64 is not set # CONFIG_CRYPTO_AES_NI_INTEL is not set # CONFIG_CRYPTO_CAMELLIA_AESNI_AVX_X86_64 is not set # CONFIG_CRYPTO_CAMELLIA_AESNI_AVX2_X86_64 is not setif you could run
cat /proc/crypto | grep aeson both fos linux and the system where the it works. Or is that where you posted above thecat crypto | grep aesabove? -
@humoss233 As well run
cryptsetup --helpand check the last couple of lines for cipher information (from https://superuser.com/questions/1039487/check-that-kernel-supports-aes-xts-plain64-cipher). -
@humoss233 I added in aes ni and recompiled it here: https://drive.google.com/open?id=1N6q6Oqmi7W7WkdtNPK1H0O8B1f-a4RFU
--- kernelx64.config 2019-08-29 12:46:58.222184653 -0400 +++ .config 2019-10-20 00:20:29.579817034 -0400 @@ -1273,12 +1273,17 @@ # CONFIG_BCACHE is not set CONFIG_BLK_DEV_DM_BUILTIN=y CONFIG_BLK_DEV_DM=y -# CONFIG_DM_MQ_DEFAULT is not set +CONFIG_DM_MQ_DEFAULT=y # CONFIG_DM_DEBUG is not set -# CONFIG_DM_UNSTRIPED is not set -# CONFIG_DM_CRYPT is not set -# CONFIG_DM_SNAPSHOT is not set -# CONFIG_DM_THIN_PROVISIONING is not set +CONFIG_DM_BUFIO=y +CONFIG_DM_DEBUG_BLOCK_MANAGER_LOCKING=y +CONFIG_DM_DEBUG_BLOCK_STACK_TRACING=y +CONFIG_DM_BIO_PRISON=y +CONFIG_DM_PERSISTENT_DATA=y +CONFIG_DM_UNSTRIPED=y +CONFIG_DM_CRYPT=y +CONFIG_DM_SNAPSHOT=y +CONFIG_DM_THIN_PROVISIONING=y # CONFIG_DM_CACHE is not set # CONFIG_DM_WRITECACHE is not set # CONFIG_DM_ERA is not set @@ -3135,10 +3140,12 @@ CONFIG_CRYPTO_NULL2=y # CONFIG_CRYPTO_PCRYPT is not set CONFIG_CRYPTO_WORKQUEUE=y -# CONFIG_CRYPTO_CRYPTD is not set +CONFIG_CRYPTO_CRYPTD=y # CONFIG_CRYPTO_MCRYPTD is not set CONFIG_CRYPTO_AUTHENC=y # CONFIG_CRYPTO_TEST is not set +CONFIG_CRYPTO_SIMD=y +CONFIG_CRYPTO_GLUE_HELPER_X86=y # # Authenticated Encryption with Associated Data @@ -3220,8 +3227,8 @@ # CONFIG_CRYPTO_AES=y # CONFIG_CRYPTO_AES_TI is not set -# CONFIG_CRYPTO_AES_X86_64 is not set -# CONFIG_CRYPTO_AES_NI_INTEL is not set +CONFIG_CRYPTO_AES_X86_64=y +CONFIG_CRYPTO_AES_NI_INTEL=y # CONFIG_CRYPTO_ANUBIS is not set CONFIG_CRYPTO_ARC4=y # CONFIG_CRYPTO_BLOWFISH is not set @@ -3424,8 +3431,6 @@ CONFIG_HAVE_ARCH_KASAN=y # CONFIG_KASAN is not set CONFIG_ARCH_HAS_KCOV=y -CONFIG_CC_HAS_SANCOV_TRACE_PC=y -# CONFIG_KCOV is not set # CONFIG_DEBUG_SHIRQ is not set # @@ -3460,7 +3465,7 @@ # CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set # CONFIG_LOCK_TORTURE_TEST is not set # CONFIG_WW_MUTEX_SELFTEST is not set -# CONFIG_STACKTRACE is not set +CONFIG_STACKTRACE=y # CONFIG_WARN_ALL_UNSEEDED_RANDOM is not set # CONFIG_DEBUG_KOBJECT is not set # CONFIG_DEBUG_BUGVERBOSE is not setPlease help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
-
@george1421 As well there should be
CONFIG_CRYPTO_XTS(see https://cateee.net/lkddb/web-lkddb/CRYPTO_XTS.html) - but you need to enableCONFIG_EXPERIMENTALfor that option to show up. -
@george1421 With XTS kernel module too: https://drive.google.com/open?id=1N6q6Oqmi7W7WkdtNPK1H0O8B1f-a4RFU
Edit: We may not be done yet depending on the password hash you used ref: https://lists.gt.net/gentoo/user/300718
-
@george1421 I tried the version with the XTS kernel module and it works! luksOpen works without errors and I can also mount the decrypted partition and see my files.
Now I just need to setup a post init script.@george1421 @Sebastian-Roth
Debugging info on FOS (latest bzImageCrypt):
cryptsetup --helploop-AES: aes, Key 256 bits plain: aes-cbc-essiv:sha256, Key: 256 bits, Password hashing: ripemd160 LUKS1: aes-xts-plain64, Key: 256 bits, LUKS header hashing: sha256, RNG: /dev/urandom
on Ubuntu LTS 18.04:
cryptsetup --helpDefault compiled-in device cipher parameters: loop-AES: aes, Key 256 bits plain: aes-cbc-essiv:sha256, Key: 256 bits, Password hashing: ripemd160 LUKS1: aes-xts-plain64, Key: 256 bits, LUKS header hashing: sha256, RNG: /dev/urandomuser@server:~$ cat /proc/crypto | grep aes name : cmac(aes) driver : cmac(aes-aesni) name : __xts(aes) driver : cryptd(__xts-aes-aesni) name : pcbc(aes) driver : pcbc-aes-aesni module : aesni_intel name : fpu(pcbc(__aes)) driver : fpu(pcbc(__aes-aesni)) module : aesni_intel name : pcbc(__aes) driver : pcbc(__aes-aesni) name : xts(aes) driver : xts-aes-aesni module : aesni_intel name : ctr(aes) driver : ctr-aes-aesni module : aesni_intel name : cbc(aes) driver : cbc-aes-aesni module : aesni_intel name : ecb(aes) driver : ecb-aes-aesni module : aesni_intel name : gcm(aes) driver : generic-gcm-aesni module : aesni_intel name : __generic-gcm-aes-aesni driver : __driver-generic-gcm-aes-aesni module : aesni_intel name : rfc4106(gcm(aes)) driver : rfc4106-gcm-aesni module : aesni_intel name : __gcm-aes-aesni driver : __driver-gcm-aes-aesni module : aesni_intel name : __xts(aes) driver : __xts-aes-aesni module : aesni_intel name : __ctr(aes) driver : __ctr-aes-aesni module : aesni_intel name : __cbc(aes) driver : __cbc-aes-aesni module : aesni_intel name : __ecb(aes) driver : __ecb-aes-aesni module : aesni_intel name : __aes driver : __aes-aesni module : aesni_intel name : aes driver : aes-aesni module : aesni_intel name : aes driver : aes-asm module : aes_x86_64 driver : drbg_nopr_ctr_aes256 driver : drbg_nopr_ctr_aes192 driver : drbg_nopr_ctr_aes128 driver : drbg_pr_ctr_aes256 driver : drbg_pr_ctr_aes192 driver : drbg_pr_ctr_aes128 name : aes driver : aes-generic -
@humoss233 OK for the post init script, can you document the steps needed to activate that volume?
Maybe something before you create the postinit script is to pxe boot into a debug capture/ or deploy what ever action you want to do. Then manually activate that disk using your commands. And finally launch the imaging script with
fog. You will have to press enter at each step, but this way you can capture any error messages if any. If it captures OK then you can take the steps to activate it and place it in a bash script in the /images/dev/postinit scripts directory. And then finally hook your bash script into the fog.postinit master script. -
@george1421 I followed your instructions, but I keep running into an error after typing in “fog.” Maybe it’s because I set Host Primary Disk to /dev/mapper/crypt (which I confirm exists after using cryptsetup). Error message, commands, and host/image settings below.
mdadm -D /dev/md126 cryptsetup luksOpen /dev/md126p3 crypt fog
-
@humoss233 Well I guess a few things here.
- The kernel parameters are not complete for some reason. There is a variable mode or something (like that) that should be up or down depending on if you are capturing or deploying.
so after running the cryptsetup, what does
lsblkshow? What happens if you manually try to mount that encrypted partition over /mnt can you read the partition contents?Does this command
cryptsetup luksOpen /dev/md126p3 cryptcreate a device called /dev/crypt?If so /dev/crypt should represent an encrypted partition /dev/md126p3 and not the physical disk /dev/md126.
Understand we have not worked with encrypted partitions so we have to rely on your knowledge of the filesystem.
-
@george1421
/dev/mapper/cryptis created, not/dev/crypt. Cryptsetup uses device mapper to create a mapped decrypted partition. I can mount this decrypted partition usingmount /dev/mapper/crypt /mnt/tempand successfully view all the files on the partition. This is why I thought it’d work to use/dev/mapper/cryptin the “Host Primary Disk” field. Could FOS be confused because it expects to find a disk device and not a partition?I’m not sure re: kernel parameters. This is a capture in debug mode. I’ve successfully completed captures of the full encrypted partition without debug mode (using
/dev/md126as “Host Primary Disk”). So, I’m not sure if missing parameters are contributing to the error.See
lsblkoutput below:
-
@humoss233 Great to see George has come up with the correct set of kernel options for your crypto setup.
At this point I think we are hitting kind of a wall. We might find a hole through but I am not sure yet.
FOG is made to capture whole disks, so one of the first things it does is get a list of partitions from the device. This surely fails on
/dev/mapper/crypto. There is an option in FOG that you can use to make it capture only one single partition (in the host’s settings you have Partition - defaults to Everything) but the script code as it exists right now would still try to enumerate the partitions and bail out.So looking at your
lsblkoutput my first idea was to set Host Primary Disk to/dev/md126and create a symbolic link pointing frommd126p3tomapper/crypt. But that doesn’t work because/dev/md126p3device file already exists. Hmmmm, well maybe you can delete it. It’s not an issue in the live FOS Linux because on reboot it will be restored. Try this:mdadm -D /dev/md126 cryptsetup luksOpen /dev/md126p3 crypt rm /dev/md126p3 ln -s /dev/mapper/crypt /dev/md126p3 fog -
It also looks like OP is using mdraid, not sure if specifying a disk will produce the desired results under those circumstances anyway. Though; I don’t know at all how that’s handled behind the screens so it could be no problem at all.
-
@Sebastian-Roth clever hack! there was one more hurdle:
blockdev --rereadptin the runPartprobe function fails due toioctl error on BLKRRPART: Device or resource busybecausecryptsetup luksOpenappears to be locking the device. Luckilypartprobeworks fine, so I just replaced that part of the script. Here’s my final commands (the last line just shows that the line has been replaced successfully). After runningfog, the decrypted partition/disk is successfully captured (with/dev/md126as “Host Primary Disk”). 1 GB instead of 800 GB!
-
@humoss233 I’m not sure this will work, since you are linking the /dev/md126p3_crypt to /dev/md126p3 then deleted it and then recreating it as itself. You are kind of looping back to itself. I can see a circular link here.
I wonder if you can rename /dev/md126p3 right from the start to /dev/md126p3raw and then do your cryptsetup against the renamed raw device and linking.
-
@george1421 That’s a good point and your method is safer, but the one that I’m using (from @Sebastian-Roth) also works - I unzipped and mounted the resulting .img file to make sure it’s good. It’s beyond me but cryptsetup must work in a way that once the decrypted partition is mapped, it’s no longer dependent on the device file representation.
Now I just need to think of a clever way of prompting for and transmitting the password over the network, as I’d rather not put the plaintext pass in the postinit script.
Both of you, thanks very much for your help!!
-
@humoss233 Hmm… pass-o-words…
How about an encrypted password passed as a kernel parameter to FOS Linux bzImage, then in your postinit script decode the password using local seed (same one used to encrypt the password).
-
@george1421 that’s a good idea - I’ve been researching it, but it looks like openssl is not available in FOS. Is there another way available to decrypt a given cipher?
-
@humoss233 I don’t know off the top of my head of base64 is part of fos linux or not. But that would be one option
Update: Base64 is part of fos linux, but I don’t think that is the tool to use looking a bit deeper into it.
-
@humoss233 I’m rebuilding the inits with openssl included. This is only half of the issue if the kernel doesn’t have openssl enabled. We’ll see one step at a time.
Edit: Wait, I just remembered that we built a custom kernel for the LUKS bits, so I can add it if needed since you are already running a custom kernel.
Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
