AnsweredAssumed Answered

CST HSM 3.2.0 compiled against Openssl 1.1.1f leaks memory and segfaults

Question asked by Atte Pellikka on May 22, 2020
Latest reply on May 26, 2020 by Atte Pellikka

Hi,

 

I'm attempting to compile CST against OpenSSL 1.1.1f. To achieve this I've recompiled libfrontend (as the packaged one seems to be compiled against older openssl) and used this binary to compile the complete tool. The tool out of my compile efforts segfaults, under valgrind the execution completes successfully, but valgrind reports memory leaks and an invalid read (resulting in the segmentation fault under normal run). The same behaviour manifests with both pkcs11-proxy and softhsm.

 

OS: Ubuntu 20.04 x64

CST ver: 3.2.0

OpenSSL ver: 1.1.1f

PKCS11 lib: pkcs11-proxy (libpkcs11-proxy)

 

Compilation script: 

#!/bin/bash


export OSTYPE=linux64
export ARCH=64

 

pushd code/cst/
make clean
make build
popd


pushd code/back_end-hsm/src/

 

cp ../../cst/code/obj.linux64/libfrontend.a .
make clean
make
make all
popd

Valgrind output (relevant parts):

=9601== Invalid read of size 8
==9601== at 0x4842AD6: memmove (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==9601== by 0x5174B36: gck_rpc_mechanism_list_purge (in /usr/local/lib/libpkcs11-proxy.so.0.1)
==9601== by 0x516C6F3: rpc_C_GetMechanismList (in /usr/local/lib/libpkcs11-proxy.so.0.1)
==9601== by 0x1209D1: hsm_register_token (in /home/xzr/work/cst/release/code/back_end-hsm/src/cst)
==9601== by 0x121447: pre_init_hsm (in /home/xzr/work/cst/release/code/back_end-hsm/src/cst)
==9601== by 0x121E97: hsm_ctrl (in /home/xzr/work/cst/release/code/back_end-hsm/src/cst)
==9601== by 0x49AE49A: ENGINE_ctrl_cmd_string (in /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1)
==9601== by 0x11DC84: gen_sig_data (in /home/xzr/work/cst/release/code/back_end-hsm/src/cst)
==9601== by 0x110B60: create_sig_file (cst.c:1389)
==9601== by 0x119300: cmd_handler_authenticatecsf (csf_cmd_aut_dat.c:410)
==9601== by 0x11041B: handle_command (cst.c:928)
==9601== by 0x114310: yyparse (cst_parser.y:109)
==9601== Address 0x4dce440 is 0 bytes after a block of size 544 alloc'd
==9601== at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==9601== by 0x12092A: hsm_register_token (in /home/xzr/work/cst/release/code/back_end-hsm/src/cst)
==9601== by 0x121447: pre_init_hsm (in /home/xzr/work/cst/release/code/back_end-hsm/src/cst)
==9601== by 0x121E97: hsm_ctrl (in /home/xzr/work/cst/release/code/back_end-hsm/src/cst)
==9601== by 0x49AE49A: ENGINE_ctrl_cmd_string (in /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1)
==9601== by 0x11DC84: gen_sig_data (in /home/xzr/work/cst/release/code/back_end-hsm/src/cst)
==9601== by 0x110B60: create_sig_file (cst.c:1389)
==9601== by 0x119300: cmd_handler_authenticatecsf (csf_cmd_aut_dat.c:410)
==9601== by 0x11041B: handle_command (cst.c:928)
==9601== by 0x114310: yyparse (cst_parser.y:109)
==9601== by 0x1111AB: main (cst.c:1676)
==9601==

 

<snip--> ... <--snip>

 

CSF Processed successfully and signed data available in out.bin
==9601==
==9601== HEAP SUMMARY:
==9601== in use at exit: 101,165 bytes in 321 blocks
==9601== total heap usage: 4,690 allocs, 4,369 frees, 583,740 bytes allocated
==9601==
==9601== LEAK SUMMARY:
==9601== definitely lost: 59,185 bytes in 127 blocks
==9601== indirectly lost: 11,255 bytes in 103 blocks
==9601== possibly lost: 0 bytes in 0 blocks
==9601== still reachable: 30,725 bytes in 91 blocks
==9601== suppressed: 0 bytes in 0 blocks
==9601== Rerun with --leak-check=full to see details of leaked memory
==9601==
==9601== For lists of detected and suppressed errors, rerun with: -s
==9601== ERROR SUMMARY: 12 errors from 1 contexts (suppressed: 0 from 0)

 

Any ideas to sort this out? Should I just compile against 1.1.0 or so? I did try to compile against Openssl 1.0.2 but ran into some issues there too (though might have been pebkac).

Outcomes