CST HSM - Segmentation fault (core dumped)

[tomasklein]

Hello,

I have implemented solution to sign the u-boot and zImage with CST tool. Now I would like to implement CST tool to work with HSM module. I already have PKI tree generated on the HSM. I created the CST tool with back_end-hsm, create hsm.cfg file with path to the certificate and run CST tool. After a moment I get error Segmentation fault (core dumped). With cst tool located in /linux64/bin/cst there is no problem to sign the image, but with back_end-hsm I get segmentation fault. (CST 3.1.0 and also with CST 3.2.0)

EDIT:

I am using module libsc-hsm-pkcs11.so, error is on the line 589 (ENGINE_ctrl_cmd_string (eng, "MODULE_PATH", hsm_conf.module_path, 0);)

Thank you for any idea.

Best regards

Tomas Klein

[pfaust60]

The solution is correct by there are some details missing.  The function that is referenced in the newly created header file "engine_hsm" is referenced in backend.c.  Since backend.c does not see the definition, it assumes that it is a function that returns an int.  So it truncates 32 bits off of the pointer and assigns it to the return of engine_hsm.  When it passes that value to the next libcrypto call, it will fail since the pointer has been corrupted.

If you put a reference to the newly created header file in backend.c so that it sees a proper prototype, it will no longer truncate the pointer and when it is passed to the next libcrypto function, it will succed.

The original developer of backend-hsm must have test this with a 32bit compiler.

[tomasklein · Accepted solution]

I solve the problem by moving the Engine constructor (ENGINE *engine_hsm (void); - e_hsm.c, line: 124), from C file to to e_hsm.h (files path: cst-3.1.0/release/code/back_end-hsm/src)

[sudhakervincent]

I did the same. But this doesn't work.

Does it have anything to do with version of openssl ? I am using the openssl 1.1.1 with Ubuntu18.04

[tomasklein]

In our case, we also using openssl version 1.1.1, cst 3.1.0 and library to comunicate with HSM (GitHub - CardContact/sc-hsm-embedded: PKCS#11 and CSP-Minidriver library for the SmartCard-HSM and S... )

[sudhakervincent]

Do u have any steps u had already followed. ?

This is the error i got after using cst-hsm compilation

 cst[20371]: segfault at ffffffff8cdac980 ip 00007fa5b8c56fbc sp 00007ffc19f0f320 error 5 in libcrypto.so.1.1[7fa5b8b04000+29b000]

this is the configuration I am using. Even i tried static compilation of those.

ldd /usr/bin/openssl
    linux-vdso.so.1 (0x00007ffe39bd5000)
    libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007fdeefe05000)
    libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007fdeef93a000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fdeef71b000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fdeef32a000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fdeef126000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fdef0345000)

I solve the problem by moving the Engine constructor (ENGINE *engine_hsm (void); - e_hsm.c, line: 124), from C file to to e_hsm.h (files path: cst-3.1.0/release/code/back_end-hsm/src)

Where exactly you moved the line ?

[Yuri]

Hello,

  Would You please provide more details how Your CST has been build;

what toolchain and OS were used? 

Regards,

Yuri.