CST Segfault while Encrypting Large Image

[robertfendricks]

I am attempting to use the Linux 64-bit version of CST 3.1.0 to encrypt various images for boot. I am able to successfully encrypt an SPL and U-Boot image and can boot from both on an i.MX6 with no issues. However when I attempt to encrypt my Kernel Image the CST fails with a segmentation fault. After a little bit of trial and error, it would appear that the CST will happily encrypt an image up to about 0x7FD000 (8,376,620) bytes. I found the precise number to be 0x7FDCB0 (8,379,568) bytes, however according to Appendix G.1 of Application Note 4581 a kernel image must be padded to the nearest 0x1000  boundary; 0x7FD000 is the closest boundary that will still work.

I am using the FIT Image format which in my case contains a zImage, a Device Tree Binary, and an initramfs, so my image is inherently big. However, I can verify that a zImage padded to be 0x7FD000 bytes can be encrypted successfully, but a zImage padded to 0x7FE000 will segfault.

Is this a known issue for the CST or is this a hard limit of the HAB module? If it's an issue with the CST, is there any plan to fix this issue in a future release?

[Yuri]

Hello,

  Do You use signing (AN4581) or encryption (AN12056)?
Can You provide sequence to reproduce the issue?

Regards,

Yuri.

[robertfendricks]

Yuri,

I was following AN4581, I was unaware that AN12056 existed; it looks like it was released fairly recently. The only thing that I'm not doing that AN12056 says to do is to sign the encrypted image after it's been generated (which is a good idea!). This wouldn't solve my immediate segfault issue, however.

Here is the sequence that I've been using to reproduce the issue:

1. Ensure that the CST has been rebuilt for encryption.
2. Ensure that the certificates and keys have been generated.
3. Change to the CST's linux64 directory.
4. Generate a large test image by running:

   # Create an image that is 0x800000 bytes long
   dd if=/dev/urandom of=test_image.bin bs=1 count=$((0x800000))‍‍‍

5. Generate an IVT for the image, naming it 'test_image.ivt'. I used Boundary Device's genIVT program found here: https://boundarydevices.com/high-assurance-boot-hab-dummies/
   1. jump_loc = 0x10800000
   2. self_ptr = 0x11000000
   3. csf_data = 0x11000020
   4. Everything else is left as 0x00
6. Concatenate the IVT to the end of the image. I did not have to pad the image in this case.
   

   cat test_image.bin test_image.ivt > test_image_ivt.bin‍

7. Create the .ini file to generate the CSF, naming it test_image.csf.ini. The full text of the file can be found at the end of this post.
8. Run the CST program

   ./bin/cst --o test_image.csf --i test_image.csf.ini‍

The program should then seg fault. Making the test_image smaller (Such as setting it to 0x700000 bytes) will allow the program to complete successfully.

test_image.csf.ini:

[Header]
Version = 4.1
Security Configuration = Open
Hash Algorithm = sha256
Engine Configuration = 0
Certificate Format = X509
Signature Format = CMS
Engine = CAAM

[Install SRK]
File = "../crts/SRK_1_2_3_4_table.bin"
Source index = 0

[Install CSFK]
File = "../crts/CSF1_1_sha256_4096_65537_v3_usr_crt.pem"

[Authenticate CSF]

[Unlock]
Engine = CAAM
Features = RNG

[Install Key]
# Key slot index used to authenticate the key to be installed
Verification index = 0
# Key to install
Target index = 2
File = "../crts/IMG1_1_sha256_4096_65537_v3_usr_crt.pem"

[Authenticate Data]
Verification index = 2
# Authenticate the IVT
Blocks = 0x11000000 0x800000 0x20 "test_image_ivt.bin"

#Encrypt the boot image and create a DEK
[Install Secret Key]
Verification Index = 0
Target Index = 0
Key = "test_image_dek.bin"
Key Length = 128
# Blob Address is equal to load_addr + image_size + IVT_size + CSF_padding - 0x100
#                        (0x10800000)  (0x800000)    (0x20)     (0x2000)
Blob Address = 0x11001F20

#Provide DEK blob location to decrypt
[Decrypt Data]
Verification Index = 0
Mac Bytes = 16
#Encrypt everything but the IVT
Blocks = 0x10800000 0x00 0x800000 "test_image_ivt.bin"
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[robertfendricks · Accepted solution]

Ok, I figured out what the issue was and I have found a solution to fix it.

TL;DR: The max stack size on my system is set to 8192 KB by default, or 0x800000 bytes. The CST puts an array that is at least the size of the entire image on the stack, which when combined with all of the other items on the stack exceeds the max stack size and causes a segfault. To fix this issue, either increase the max stack size or modify ssl_wrapper.c as described below.

Detailed analysis:

On a whim, I decided to run Valgrind on the program to see if I could get any further insight into what was going on. I recompiled the CST to include debugging symbols and reran the test using Valgrind. Here is the output of this test: https://gist.github.com/rjfendricks-indesign/2ba68e9e39416d536ef92b4d2025cdc2 . According to this log, SIGSEGV is being thrown from line 72 of ssl_wrapper.c. This is one of the back_end source files available to the developer to modify. This specific line looks like it's allocating some space for a struct, but it's not immediately apparent why a segfault would be thrown.

if (!(ctx = EVP_CIPHER_CTX_new())) {‍‍

However, taking another look at the Valgrind output, the following lines are printed out immediately after the segfault is thrown:

==12501==  If you believe this happened as a result of a stack
==12501==  overflow in your program's main thread (unlikely but
==12501==  possible), you can try to increase the size of the
==12501==  main thread stack using the --main-stacksize= flag.
==12501== The main thread stack size used in this run was 8388608.‍‍‍‍‍‍‍‍‍‍

This is very interesting, because a stack size of 8388608 bytes is the same size as the image I passed in. I reran Valgrind passing in --main-stacksize=$((0x900000)), and the program succeeded (There were still a bunch of  Valgrind warnings, but it didn't segfault)! I did a little researching and found the `ulimit` command, which allows the developer to change default stack size.  Running "ulimit -S -a" revealed to me that my stack size was 8192 KB, or 8388608 bytes. Given this information, the following command allowed the CST to run without issue:

ulimit -S -s 9216 && ./bin/cst --o --test_image.csf --i test_image.csf.ini‍‍‍

I set the stack size to 9126 KB, 1024KB bytes larger than my image size, to give the program more than enough space to hold the image.

My best guess for why the segfault happened at line 72 of ssl_wrapper.c is that a few lines earlier, on line 66, an array is created that is the size of the plaintext file + EVP_MAX_BLOCK_LENGTH.

unsigned char ciphertext[plaintext_len + EVP_MAX_BLOCK_LENGTH] ;‍‍‍

Then on line 72 a function is called to allocate memory for a new cipher constructor. So the array, which is at least 0x800000 bytes long, is put onto the stack and overruns the max stacksize, causing a segfault and crashing the program. To fix this, I allocated a buffer that is plaintext_len + EVP_MAX_BLOCK_LENGTH long in place of creating an array:

unsigned char *ciphertext = NULL;
ciphertext = (unsigned char *) malloc(
    sizeof(unsigned char) * plaintext_len + EVP_MAX_BLOCK_LENGTH‍‍‍‍‍);‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Not shown above is error checking and freeing the memory. After recompiling, I can confirm that this fixed the segfault issue. Going forward, I will be using this modified version of the CST to encrypt my images.

[Yuri]

Hello,

Good job!

I will inform the app team about Your research.

Regards,

Yuri.