i.MX6UL high TCP latency

[nadja_staerk]

We recently run some tests over TCP and UDP sockets to get a maximum amount of messages per second.

Over TCP we got an unexpected high RTT and a small amount of messages per second, see attached screenshot.

We also pinged from a Linux machine to the imx6, the RTT is unexpected high too.

<p>sudo ping -i 0 -c 10000 172.20.230.123 -q</p>
<p>PING 172.20.230.123 (172.20.230.123) 56(84) bytes of data.</p>

<p></p>

<p>--- 172.20.230.123 ping statistics ---</p>
<p>10000 packets transmitted, 10000 received, 0% packet loss, time 15269ms</p>
<p>rtt min/avg/max/mdev = 1.377/1.454/1.712/0.066 ms, ipg/ewma 1.527/1.568 ms</p>

Our receiver is the i.MX6UL and the sender is an Ubuntu machine. The imx6 is connected through a local network with only this two participants.

Kernel 5.4 is running on the imx6, we tried it with and without preemption.

Someone has already encountered this kind of problem?

Is there any way to improve the latency?

[nadja_staerk]

Hello Diego,

thanks for your answer. I tried it with your latest BSP and the imxul-evk, still using my local network.

The ping shows the same high RTT.

PING 172.20.230.123 (172.20.230.123) 56(84) bytes of data.

--- 172.20.230.123 ping statistics ---
10000 packets transmitted, 10000 received, 0% packet loss, time 15343ms
rtt min/avg/max/mdev = 1.337/1.468/14.059/0.137 ms, pipe 2, ipg/ewma 1.534/1.468 ms

Over TCP I got the same behavior. 

I tried running a TCP receiver and sender on the EVK. Both using localhost.

I measured the time from receiving the first package until receiving 10 000 small packages and it needs over 1 second to receive them.

Any way to improve this?

Many thanks,

Nadja

[diegoadrian]

Hello,

Unfortunately, reviewing. This problem seems to be more related to the Ethernet PHY and the topology of the connection that you are using. The PHY of the board could take some time to perform the connection. 

Best regards,

Diego.

[nadja_staerk]

Hi Diego,

I also tried it with the sender and the receiver on the same EVK, using local host.

I don't think the Ethernet Phy is used in this case. I measured the time from receiving the first package until receiving 10 000 small packages and it needs over 1 second to receive them.

Attached you find the sender and the receiver.

sender:

<p>

#include <stdio.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <string.h>
#include <sys/time.h>
#include <stdlib.h>

#define PORT 6180

  
int main(int argc, char const *argv[])
{
    int sock = 0, valread;
    struct sockaddr_in serv_addr;
    char *hello = "Hello from client";
    char buffer[1024] = {0};
    int i = atoi(argv[1]);
    int delay = atoi(argv[2]);
    printf("messages to send: %d with delay of %dus\n",i,delay);
    struct timeval tval_before, tval_after, tval_result;   
    
    if ((sock = socket(AF_INET, SOCK_STREAM, 0)) < 0)
    {
        printf("\n Socket creation error \n");
        return -1;
    }
   
    serv_addr.sin_family = AF_INET;
    serv_addr.sin_port = htons(PORT);
       
    // Convert IPv4 and IPv6 addresses from text to binary form
    if(inet_pton(AF_INET, "127.0.0.1", &serv_addr.sin_addr)<=0)  
    {
        printf("\nInvalid address/ Address not supported \n");
        return -1;
    }
   
    if (connect(sock, (struct sockaddr *)&serv_addr, sizeof(serv_addr)) < 0)
    {
        printf("\nConnection Failed \n");
        return -1;
    }
    gettimeofday(&tval_before, NULL);  
    while(i > 0){
        send(sock , hello , 17 , 0 );
        i--;
        if( delay > 0){
            usleep(delay);
        }
    }
    gettimeofday(&tval_after, NULL);
    timersub(&tval_after, &tval_before, &tval_result);
    printf("Time elapsed: %ld.%06lds\n", (long int)tval_result.tv_sec, (long int)tval_result.tv_usec);

    return 0;
}

</p>

receiver:

<p>

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <netinet/in.h>
#include <sys/time.h>
#include <errno.h>
#include <netinet/tcp.h>

#define BUFSZ 1024
int port = 6180;             /* no significance */

int main(int argc, char *argv[]) {

  char buf[BUFSZ];
  int rc;
  int amountOfMessages=0;
  struct timeval tval_before, tval_after, tval_result;   

  /**********************************************************
   * create an IPv4/TCP socket, not yet bound to any address
   *********************************************************/
  int fd = socket(AF_INET, SOCK_STREAM, 0);
  if (fd == -1) {
    printf("socket: %s\n", strerror(errno));
    exit(-1);
  }

  /**********************************************************
   * internet socket address structure: our address and port
   *********************************************************/
  struct sockaddr_in sin;
  sin.sin_family = AF_INET;
  sin.sin_addr.s_addr = htonl(INADDR_ANY);
  sin.sin_port = htons(port);

  /**********************************************************
   * bind socket to address and port we'd like to receive on
   *********************************************************/
  if (bind(fd, (struct sockaddr*)&sin, sizeof(sin)) == -1) {
    printf("bind: %s\n", strerror(errno));
    exit(-1);
  }

  /**********************************************************
   * put socket into listening state
   *********************************************************/
  if (listen(fd,1) == -1) {
    printf("listen: %s\n", strerror(errno));
    exit(-1);
  }

  /**********************************************************
   * accept a connection, read til it closes, repeat
   *********************************************************/
  while (1) {
    struct sockaddr_in cin;
    socklen_t cin_sz = sizeof(cin);
    int i = 1;
    int fa = accept(fd, (struct sockaddr*)&cin, &cin_sz);
    if (fa == -1) {
      printf("accept: %s\n", strerror(errno));
      continue;
    }

    if (sizeof(cin)==cin_sz){
        printf("connection from %s:%d\n",
      inet_ntoa(cin.sin_addr), (int)ntohs(cin.sin_port));
      amountOfMessages = 0;
      gettimeofday(&tval_before, NULL);
    }
    do {
      rc = read(fa,buf,BUFSZ);
      if (rc==-1) printf("read: %s\n", strerror(errno));
      else if (rc==0){
           gettimeofday(&tval_after, NULL);
           printf("connection closed after %d messages\n", amountOfMessages);
           timersub(&tval_after, &tval_before, &tval_result);
           printf("Time from connect to close: %ld.%06lds\n", (long int)tval_result.tv_sec, (long int)tval_result.tv_usec);
       }
      else{
          amountOfMessages++;
      }
    } while (rc > 0);
    close(fa);
  }
}

</p>

Many thanks,

Nadja

[diegoadrian]

Hello, 

Unfortunately, you are using a Linux kernel version that is not supported by us. I suggest you test with the BSP that we already provide. Please find them on the following web-page.

Embedded Linux for i.MX Applications Processors | NXP 

Best regards,

Diego.