OpenSSL Engine SE050 ECKey authentication Issue with OpenSSL 3.0

[adli]

Hello,

We are using the SE050 primarily for authentication and remote access needs, using certificates with private keys stored inside the SE050. Our setup includes OpenVPN for remote access with OpenSSL and the NXP OpenSSL engine.

We authenticate to the SE050 using the ECKey method (formerly known as FastSCP). The ECKey authentication has been integrated into the engine during the initialization. Our current versions are:

• OpenSSL: 3.0.15
• SE050 Middleware: 04.05.01

Provisioning Steps:

• Generate a private key inside the secure element (EC Key NISTP256).
• Generate a reference key pointing to the private key.
• Generate a CSR using OpenSSL with the reference key.
• Sign the CSR with the relevant CA for remote access.
• Upload the signed certificate to the device.

Issue:

When attempting to connect to the OpenVPN server, we encounter the following unexpected error:

digital envelope routines::command not supported
OpenSSL: error:03000093:digital envelope routines::command not supported

We identified the root cause in the SE050 Middleware code. Specifically, the issue occurs during ECKey authentication, in the function sss_openssl_asymmetric_sign_digest within the file fsl_sss_openssl_apis.c: 

EVP_PKEY_CTX_set_rsa_pss_saltlen(pKey_Ctx, RSA_PSS_SALTLEN_DIGEST);

This function call fails because the key in use is an ECKey, not an RSA key. In OpenSSL 3.0, this error propagates to the upper layer (SSL in libssl), resulting in the observed error.
This issue does not occur in OpenSSL 1.1.1.

Commenting out this line resolves the issue for us, as we are using only ECKeys.

Could you please review this issue and confirm if we are implementing things correctly? We would appreciate knowing if a fix or update will be released to address this issue if confirmed.

Thank you for your time and support.

[Kan_Li]

Hi @adli ,

As far as I know, OpenSSL engine can only work with OpenSSL 1.1.1, but not with OpenSSL 3.0, as OpenSSL provider is used instead.  Please kindly refer to simw-top/doc/sss/plugin/openssl_provider/scripts/readme.html for details.

Have a great day,
Kan

-------------------------------------------------------------------------------
Note:
- If this post answers your question, please click the "Mark Correct" button. Thank you!
- We are following threads for 7 weeks after the last post, later replies are ignored
Please open a new thread and refer to the closed one, if you have a related question at a later point in time.
-------------------------------------------------------------------------------