cst-3.3.1 back_end-hsm linker error

cancel
Showing results for 
Search instead for 
Did you mean: 

cst-3.3.1 back_end-hsm linker error

Jump to solution
3,134 Views
cheuschkel
Contributor III

I am receiving the following error when following directions to build the back end HSM source code from the recently released NXP CST 3.3.1 package (note in the directory listing that I had copied in the libfrontend.a which was built successfully):

 

osboxes@osboxes:~/cst-3.3.1/code/back_end-hsm/src$ make clean
$ARCH is []
rm -f backend.o config.o e_hsm.o e_hsm_err.o openssl_helper.o *~ core tags *.bak Makefile.bak libbackend.*
osboxes@osboxes:~/cst-3.3.1/code/back_end-hsm/src$ ls -al
total 680
drwxr-xr-x 3 osboxes osboxes   4096 Sep 20 22:46 .
drwxr-xr-x 4 osboxes osboxes   4096 Aug 14 08:53 ..
-rwxr-xr-x 1 osboxes osboxes  38775 Aug 14 08:56 backend.c
-rwxr-xr-x 1 osboxes osboxes   3874 Aug 14 08:56 config.c
-rwxr-xr-x 1 osboxes osboxes  51093 Aug 14 08:56 e_hsm.c
-rwxr-xr-x 1 osboxes osboxes   8670 Aug 14 08:56 e_hsm_err.c
drwxr-xr-x 2 osboxes osboxes   4096 Aug 14 08:53 include
-rwxr-xr-x 1 osboxes osboxes  96914 Sep 20 22:46 lib
-rw-rw-r-- 1 osboxes osboxes 437150 Sep 20 13:25 libfrontend.a
-rwxr-xr-x 1 osboxes osboxes   1937 Sep 20 22:22 Makefile
-rw-r--r-- 1 osboxes osboxes  12288 Sep 20 22:22 .Makefile.swp
-rwxr-xr-x 1 osboxes osboxes   4197 Aug 14 08:56 openssl_helper.c
-rw------- 1 osboxes osboxes  12288 Sep 20 21:34 .swp
osboxes@osboxes:~/cst-3.3.1/code/back_end-hsm/src$ make 
$ARCH is []
[Compile] backend.c
[Compile] config.c
[Compile] e_hsm.c
[Compile] e_hsm_err.c
[Compile] openssl_helper.c
[Link (Static) libbackend]
osboxes@osboxes:~/cst-3.3.1/code/back_end-hsm/src$ make all
$ARCH is []
gcc   -std=c99  -D_POSIX_C_SOURCE=200809L -Wall -Werror -g -Wall -o cst  libfrontend.a libbackend.a -L./lib  -I/../openssl/include -mno-ms-bitfields -L/../openssl/lib -lssl -lcrypto -ldl -lpthread -lconfig -fno-builtin -fno-strict-aliasing -fno-common -DREMOVE_ENCRYPTION -Wl,--allow-multiple-definition
/usr/bin/ld: libfrontend.a(csf_cmd_ins_key.o): in function `cmd_handler_installcsfk':
/home/osboxes/cst-3.3.1/code/cst/code/obj.linux64/../../code/front_end/src/csf_cmd_ins_key.c:741: undefined reference to `get_der_encoded_certificate_data'
/usr/bin/ld: libfrontend.a(csf_cmd_ins_key.o): in function `cmd_handler_installnocak':
/home/osboxes/cst-3.3.1/code/cst/code/obj.linux64/../../code/front_end/src/csf_cmd_ins_key.c:847: undefined reference to `get_der_encoded_certificate_data'
/usr/bin/ld: libfrontend.a(csf_cmd_ins_key.o): in function `cmd_handler_installkey':
/home/osboxes/cst-3.3.1/code/cst/code/obj.linux64/../../code/front_end/src/csf_cmd_ins_key.c:992: undefined reference to `get_der_encoded_certificate_data'
collect2: error: ld returned 1 exit status
make: *** [Makefile:77: all] Error 1

 

  

I'm not sure how get_der_encoded_certificate_data is supposed to be linked in in the first place. In the back_end-engine source, this function exists in backend.c, but in back_end-hsm backend.c, it does not. Can you please tell me if I'm missing a step?

0 Kudos
1 Solution
2,938 Views
cheuschkel
Contributor III

In addition to your recommendations, I also had to add the following files to back_end-hsm/src (and associated header files for each source file to back_end-hsm/src/include) and the local copies to the back_end-hsm/Makefile:

cst/code/back_end/src/adapt_layer_openssl.c

cst/code/back_end/src/pkey.c

After that, I was able to clean, re-run make, and run make all and build successfully. The cst executable now works, but I cannot yet attest to it working with my HSM as I'd expect. I suppose that means this can be closed since I solved the original linker problem. Thanks for all of the help.

View solution in original post

22 Replies
2,583 Views
marouene_boubakri
NXP Employee
NXP Employee

Hi @cheuschkel,

 

In order to use CST with your HSM please build using back_end-engine. The back_end-hsm is deprecated and will be removed in next release.

use AN12812 (Using Code-Signing Tool with Hardware Security Module) as advised by @Yuri .

 

Regards

Maro

0 Kudos
2,578 Views
cheuschkel
Contributor III

@marouene_boubakri @Yuri Is the hsm.cfg not needed for back_end-engine?

Thanks,

Cory

0 Kudos
2,571 Views
marouene_boubakri
NXP Employee
NXP Employee

@cheuschkel  Not needed  

2,288 Views
Terry3
Contributor II

How do we specify the pkcs11 URI for AHAB ?

I put the pkcs11 URI in the "Source =" key of the [Install SRK] section but the CST returns an error "Unable to read".

 

0 Kudos
2,272 Views
YairBA
Contributor II

Looks like a problem in the object=<cert_label> part in the pkcs11 URI.

If you are using pkcs15-tool -D  to list all objects from the HSM, the label of the object is what inside the squared brackets e.g.  x.509 certificate [some_label]

0 Kudos
2,257 Views
Terry3
Contributor II

Not sure of what you mean, I tried to put the object as you suggested but still "unable to read".

It looks like the parser does not recognize the PKCS11 URI for the "Source" key for AHAB

0 Kudos
2,198 Views
Terry3
Contributor II

It looks like the back_end-engine (not the hsm one) only creates a sig_req.txt file but does not actually sign with the HSM.

What is the sig_req.txt for (and its associated container.bin) for ?

0 Kudos
2,190 Views
marouene_boubakri
NXP Employee
NXP Employee

Hi @Terry3,

 

Since this thread has been marked solved, I recommend opening a new one for better tracking of the issue.

 

Regards

Maro

0 Kudos
2,822 Views
richardgottscha
Contributor I

I have written a script that makes it work with CentOS 7, so that all dependencies are included within libfrontend.a:

https://justpaste.it/nxp_cst

3,007 Views
cheuschkel
Contributor III

For what it is worth, I just tested the cst-3.1.0 package and was able to link that successfully. The libfrontend.a was pre-compiled in the lib/ directory, and worked just fine for me. This appears to be an issue with the custom build of the libfrontend.a package in cst-3.3.0.

 

osboxes@osboxes:~/cst-3.1.0/release/linux64/lib$ cp libfrontend.a ../../code/back_end-hsm/src/
osboxes@osboxes:~/cst-3.1.0/release/linux64/lib$ cd ../../code/back_end-hsm/src/
osboxes@osboxes:~/cst-3.1.0/release/code/back_end-hsm/src$ make all
$ARCH is []
gcc   -std=c99  -D_POSIX_C_SOURCE=200809L -Wall -Werror -g -Wall -o cst  libfrontend.a libbackend.a -L./lib  -I/../openssl/include -mno-ms-bitfields -L/../openssl/lib -lssl -lcrypto -ldl -lpthread -lconfig -fno-builtin -fno-strict-aliasing -fno-common -DREMOVE_ENCRYPTION -Wl,--allow-multiple-definition
osboxes@osboxes:~/cst-3.1.0/release/code/back_end-hsm/src$ ls -al
total 700
drwxr-x--- 3 osboxes root      4096 Sep 21 10:35 .
drwxr-x--- 4 osboxes root      4096 Sep 21 10:23 ..
-rwxr-x--- 1 osboxes root     37338 Sep 21 10:23 backend.c
-rw-rw-r-- 1 osboxes osboxes  27664 Sep 21 10:34 backend.o
-rwxr-x--- 1 osboxes root      2705 Sep 21 10:23 config.c
-rw-rw-r-- 1 osboxes osboxes   4152 Sep 21 10:34 config.o
-rwxrwxr-x 1 osboxes osboxes 187520 Sep 21 10:35 cst
-rwxr-x--- 1 osboxes root     50555 Sep 21 10:23 e_hsm.c
-rwxr-x--- 1 osboxes root      7501 Sep 21 10:23 e_hsm_err.c
-rw-rw-r-- 1 osboxes osboxes   8504 Sep 21 10:34 e_hsm_err.o
-rw-rw-r-- 1 osboxes osboxes  54064 Sep 21 10:34 e_hsm.o
drwxr-x--- 2 osboxes root      4096 Sep 21 10:23 include
-rw-rw-r-- 1 osboxes osboxes  96962 Sep 21 10:34 libbackend.a
-rwxr-x--- 1 osboxes osboxes 190424 Sep 21 10:35 libfrontend.a
-rwxr-x--- 1 osboxes root      2216 Sep 21 10:23 Makefile
-rwxr-x--- 1 osboxes root      3028 Sep 21 10:23 openssl_helper.c
-rw-rw-r-- 1 osboxes osboxes   1376 Sep 21 10:34 openssl_helper.o
2,975 Views
cheuschkel
Contributor III

It's worth noting that I'm using an Ubuntu 20.04 VM with the default system OpenSSL v1.1.1f installed. Now that I see all of the NXP rep posts saying to use v1.0.2, I went ahead and downloaded and installed it locally. But now when I set my OPENSSL_PATH and try to run make on the cst to build libfrontend.a, I get the following errors:

 

 

osboxes@osboxes:~/cst-3.3.1/code/cst$ OSTYPE=linux64 OPENSSL_PATH=/home/osboxes/openssl-1.0.2l make 
Compile srktool.o
Compile openssl_helper.o
Compile srk_helper.o
Compile err.o
Link srktool
/usr/bin/ld: srktool.o: in function `generate_srk_data':
/home/osboxes/cst-3.3.1/code/cst/code/obj.linux64/../../code/srktool/src/srktool.c:720: undefined reference to `X509_get_pubkey'
...
(lot of linker errors)
...
/usr/bin/ld: cryptlib.c:(.text+0x79b): undefined reference to `sk_value'
collect2: error: ld returned 1 exit status
make[1]: *** [../build/make/rules.mk:24: srktool] Error 1
make: *** [Makefile:59: rel_bin] Error 2

 

 

@Yuri you seem to be the most active NXP rep on these posts. Can you please look over these problems with the HSM back end?

2,850 Views
Yuri
NXP TechSupport
NXP TechSupport

Hello,

    use AN12812 (Using Code-Signing Tool with Hardware Security Module).

The document is valid for CST versions starting from 3.3.1.
OpenSSL should be 1.0.2g .
Ubuntu 16.04 and as root user.

 

Regards,
Yuri.

2,910 Views
YairBA
Contributor II

Assuming your host/VM runs a different version of OpenSSL, you can build it locally:

 

git clone https://github.com/openssl/openssl.git $HOME/openssl
pushd $HOME/openssl
    git checkout OpenSSL_1_0_2
    ./Configure linux-x86_64 --prefix=/home/$USER/ssl
    make
    make install
popd

 

For linking cst:

  1. Clone OpenSSL and checkout to OpenSSL_1_0_2
  2. Change into ~/cst-3.3.1/code/cst
  3. Open Makefile and add the following two changes:

 

line #48:
OPENSSL_CONFIG += no-deprecated no-threads no-shared no-dso no-engine no-hw \
+                  no-idea --prefix=$(HOME)

line 75:
        make clean                    && \
        make                          && \
+        make install                  && \
        cp ms/applink.c include/openssl/​​

4. Make:

 

 

OSTYPE=linux64 OPENSSL_PATH=$HOME/openssl make openssl
OSTYPE=linux64 OPENSSL_PATH=$HOME/openssl make

 

2,890 Views
cheuschkel
Contributor III

Thank you @YairBA! This got past my compilation problems for the CST. But now, using the OpenSSL on the 1_0_2 branch, I am having new errors when building the back_end-hsm project:

sboxes@osboxes:~/nxp/cst-3.3.1/code/back_end-hsm/src$ OSTYPE=linux64 OPENSSL_PATH=$HOME/openssl make
$ARCH is []
[Compile] backend.c
In file included from ./include/e_hsm.h:46,
                 from backend.c:48:
/home/osboxes/openssl/include/openssl/engine.h:71:4: error: #error ENGINE is disabled.
   71 | #  error ENGINE is disabled.
      |    ^~~~~
make: *** [Makefile:67: backend.o] Error 1

 

For what it's worth, if I just run `make` (which would in turn use my OpenSSL 1.0.2l version I have in a different directory that installed to /usr/bin/ssl) I compile the back end without problems. Perhaps I need to get rid of my separate OpenSSL v1.0.2l install. 

Do you know why this ENGINE error is presented?

Thanks in advance.

0 Kudos
2,879 Views
YairBA
Contributor II

Please try:

  1. Remove the current cst-3.3.1 dir and re-extract cst-3.3.1.tgz .
  2. Change into ~/cst-3.3.1/code/cst/ and do the above changes to ~/cst-3.3.1/code/cst/Makefile .
  3. Run the following make commands:

 

OSTYPE=linux64 OPENSSL_PATH=$HOME/openssl make openssl
OSTYPE=linux64 OPENSSL_PATH=$HOME/ssl make​

4. Change into ~/cst-3.3.1/code/back_end-hsm/src/ and run:

 

 

OSTYPE=linux64 OPENSSL_PATH=$HOME/ssl make
OSTYPE=linux64 OPENSSL_PATH=$HOME/ssl make all​

 

You will get (again):

 

cst-3.3.1/code/cst/code/obj.linux64/../../code/front_end/src/csf_cmd_ins_key.c:741: undefined reference to `get_der_encoded_certificate_data'
/usr/bin/ld: cst-3.3.1/code/cst/code/obj.linux64/libfrontend.a(csf_cmd_ins_key.o): in function `cmd_handler_installnocak':
...​

 

To overcome the above linking issues, I build adapt_layer_openssl.c from within ~/cst-3.3.1/code/back_end-hsm/src/ , basically copied the relevant (for adapt_layer_openssl.c  e.g. headers files) files from ~/cst-3.3.1/code/cst/code/back_end/src/ to ~/cst-3.3.1/code/back_end-hsm/src/ and from ~/cst-3.3.1/code/cst/code/front_end/hdr/ to ~/cst-3.3.1/code/back_end-hsm/src/include/ and modify ~/cst-3.3.1/code/back_end-hsm/src/Makefile to build adapt_layer_openssl.c .

Not the most sophisticated solution, but now I can build the cst file.

I'll try to post some more complete solution later today.

Please take into account that I not yet tested the created cst file.

0 Kudos
2,819 Views
cheuschkel
Contributor III

@YairBA After being informed that the official supported Linux distribution was Ubuntu 16.04 LTS that includes OpenSSL v1.0.2g for the system install, I followed your directions to move all of the associated headers for adapt_layer_openssl.c and was able to build. I will test the CST now and see if it works as expected. 

Thanks,

Cory

2,939 Views
cheuschkel
Contributor III

In addition to your recommendations, I also had to add the following files to back_end-hsm/src (and associated header files for each source file to back_end-hsm/src/include) and the local copies to the back_end-hsm/Makefile:

cst/code/back_end/src/adapt_layer_openssl.c

cst/code/back_end/src/pkey.c

After that, I was able to clean, re-run make, and run make all and build successfully. The cst executable now works, but I cannot yet attest to it working with my HSM as I'd expect. I suppose that means this can be closed since I solved the original linker problem. Thanks for all of the help.

2,716 Views
YairBA
Contributor II

Hi,

I got CST version 3.3.1 to compile and work with HSM:

pushd ~/cst-3.3.1/code/cst
    OSTYPE=linux64 make rel_bin
popd

pushd ~/cst-3.3.1/code/back_end-engine/src
    sed -i 's#^ROOT :=.*#ROOT := ../../cst/code#g' ./Makefile
    sed -i 's#^FRONTEND :=.*#FRONTEND := $(ROOT)/obj.linux$(BITNESS)/libfrontend.a#g' ./Makefile
    OSTYPE=linux64 make
popd

Use  p11tool --list-tokens  to get the HSM URL.

The  File  field inside the .csf file should looks like:

File = "pkcs11:token=some-token%20%28UserPIN%29;object=some_file;type=cert;pin-value=123456"

Then run the  cst  command with the  -i and -o  parameters.

If you run into "segmentation fault" make sure that SoftHSM is not installed, or some other race-condition (use valgrind).

I got it to run with the default package for Debian 10 (OpenSSL 1.1.1d, etc.)

2,704 Views
cheuschkel
Contributor III

@YairBA,

Have you successfully signed some binary file with this CST you built on Debian 10? What HSM are you using if you don't mind me asking?

 

Thanks,

Cory

0 Kudos
2,700 Views
YairBA
Contributor II