wolfSSL_connect returns error (wolfSSL library for RTCS)

[nickerb]

Environment:
KDS, KSDK 1.3, MQX for KSDK, wolfSSL

I'm porting the new AWS IoT library to MQX using the RTCS and, more predominantly, wolfSSL.

The setup:  I have created the RTCS properly and have bound my local ethernet hardware to a DHCP instance.  I can ping google.ca and return valid results from the shell.

In general, I call the following to securely connect:

int iot_tls_connect(Network *pNetwork, TLSConnectParams params) {
  IoT_Error_t ret_val = NONE_ERROR;
  server_TCPSocket = Create_TCPSocket();
  if(-1 == server_TCPSocket){
    ret_val = TCP_SETUP_ERROR;
    return ret_val;
  }
  if (!wolfSSL_CTX_load_verify_locations(pSSLContext, params.pRootCALocation, NULL)) {
    ret_val = SSL_CERT_ERROR;
  }
  if (!wolfSSL_CTX_use_certificate_file(pSSLContext, params.pDeviceCertLocation, SSL_FILETYPE_PEM)) {
    ret_val = SSL_CERT_ERROR;
  }
  if(1 != wolfSSL_CTX_use_PrivateKey_file(pSSLContext, params.pDevicePrivateKeyLocation, SSL_FILETYPE_PEM)){
    ret_val = SSL_CERT_ERROR;
  }
  wolfSSL_CTX_set_verify(pSSLContext, SSL_VERIFY_PEER, NULL);
  pSSLHandle = wolfSSL_new(pSSLContext);
  pDestinationURL = params.pDestinationURL;
  ret_val = Connect_TCPSocket(server_TCPSocket, params.pDestinationURL, params.DestinationPort);
  if(NONE_ERROR != ret_val){
    return ret_val;
  }
  wolfSSL_set_fd(pSSLHandle, server_TCPSocket);
  if(NONE_ERROR == ret_val){
  ret_val = setSocketToNonBlocking(server_TCPSocket);
  }
  if(NONE_ERROR == ret_val){
    ret_val = ConnectOrTimeoutOrExitOnError(pSSLHandle, params.timeout_ms);
  // ensure you have a valid certificate returned, otherwise no certificate exchange happened
    if(NULL == wolfSSL_get_peer_certificate(pSSLHandle)){
      ret_val = SSL_CONNECT_ERROR; 
    }
  }
  return ret_val;
}
IoT_Error_t ConnectOrTimeoutOrExitOnError(WOLFSSL *pSSL, int timeout_ms){
  enum{
    SSL_CONNECTED = 1,
    SELECT_TIMEOUT = 0,
    SELECT_ERROR = -1
  };
  IoT_Error_t ret_val = NONE_ERROR;
  int rc = 0;
  fd_set readFds;
  fd_set writeFds;
  struct timeval timeout = {0, timeout_ms*1000};
  int errorCode = 0;
  int select_retCode = SELECT_TIMEOUT;
  do{
    rc = wolfSSL_connect(pSSL);
    if(SSL_CONNECTED == rc){
      ret_val = NONE_ERROR;
      break;
    }
    errorCode = wolfSSL_get_error(pSSL, rc);
    ... // some error code detection here
    else{
      ret_val = SSL_CONNECT_ERROR;
    }
  }while(SSL_CONNECT_ERROR != ret_val && SSL_CONNECT_TIMEOUT_ERROR != ret_val);
  return ret_val;
}

That is, I create a socket (this succeeds), I load my root CA, client cert and client key (this succeeds), I set verification of the context (succeeds), I create a new wolfSSL context (succeeds), set my destination URL, connects to the destination URL on the desired port (succeeds!), set fd (succeeds -- returning 0x1, a successful return), sets to non-block (succeeds) and then enters ConnectOrTimeoutOrExitOnError (a function name provided by the AWS IoT) on above line 41 which fails. The failure point in this function is at line 69 above, the wolfSSL_connect(pSSL) call.

I'd debug with an appropriate JLink connection, but (according to known issues in the readme & my experience) turning debug on in the wolfSSL library exhausts memory, so that's pretty much out.  I checked the wolfSSL library and beside the connect function there's a note:

/* please see note at top of README if you get an error from connect */

Well, nice!  Except the readme with wolfSSL for MQX doesn't actually have a note regarding failed connect calls in it.  So I look up the README on the github (which ISN'T the MQX library version, but actually has the note about potentially failing auth), which can be found here: wolfssl/README at master · wolfSSL/wolfssl · GitHub

Except there seems to be no internal program logic that does anything with WOLFSSL_STATIC_DH or WOLFSSL_STATIC_RSA or WOLFSSL_STATIC_PSK.  None of them show up with a text search on the wolfSSL for MQX codebase.  How do I get this wolfSSL_connect to not fail.  I went through the error code parse but I just get a generic SSL_CONNECT_ERROR. wolfSSL_connect returns -1.

NB: I am using TLS1.2

edit: I can confirm wolfSSL_connect fails on case CONNECT_BEGIN

edit2: to provide further information: we enter SendBuffered (in SSL_source/WolfSSL_source/internal.c) appropriately and attempt a CBIOSend.  this send returns < 0 and ends up in the default switch case.

edit3: even more information: in EmbedSend, SEND_FUNCTION maps to send and returns a 'general error' when TranslateReturnCode is called.

edit4: indeed, the send is returning -1--is this to do with the non-blocking setup?

TL;DR: wolfSSL_connect fails (returns -1). Can't debug. My socket and DHCP and internet access and CA/cert/key are all fine.  Code used is heavily based upon existing openSSL code (that apparently works without fault).  Am I missing a crucial step?

[ohadbenjamin]

Hi nickerb​,

I don't have an answer to your question, but rather another question.

Why did you choose to use the WolfSSL and not the mbedTLS library that AWS IOT is already written for?

Thanks