FRDM-K64F KSDK 1.3.0 tcp_echo demo bugs?

[nicolabongiovan]

I tried to use the tcp echo demo project with rtos MQX supplied with KSDK_1.3.0, but I found some problems:

when I have tested the tcp_echo application using a PC Tool that performs echo request on TCP port 7, the FRDM-K64F responds for a little time but after the application stops in the assert function "void sys_assert( const char *msg )" file sys_arch.c.

I have captured the two msg below:

Name : msg

  Details:0x1def8 "tcpip_thread: invalid message"

  Default:0x1def8 "tcpip_thread: invalid message"

  Decimal:122616

  Hex:0x1def8

  Binary:11101111011111000

  Octal:0357370

Name : msg

  Details:0x1ca80 "pbuf_free: p->ref > 0"

  Default:0x1ca80 "pbuf_free: p->ref > 0"

  Decimal:117376

  Hex:0x1ca80

  Binary:11100101010000000

  Octal:0345200   

No changes I made to the project "lwip_tcpecho_demo_mqx_frdmk64f" from KSDK_1.3.0.

There are some bugs on the LWIP stack or there are some errata configuration parameters?

Thank for any help!

Nicola

[DavidS]

Hi Nicola,

I noticed that the linker script is really only using the lower SRAM block which is 0x10000.  The upper SRAM block has 0x30000 mostly free.  So for this example the heap is mostly used (>90%).

So I have moved the ucHeap to the second block and now it is <12% used.

The ZIP attached has text file with edit locations and also the source files if you want to try having more available heap.

Let us know if this helps or not.

Regards,

David

[manfredschnell]

Hi David,

my application uses the big heap. So I think, there is no heap problem.

Best regards

Manfred

[manfredschnell]

Hi Nicola,

perhaps it is the same problem as in KSDK2.0 focused in Thread "FreeRTOS LwIP SDK v2.0.0 tpcipecho demo Crash".

You should have a sharp look at  ethernetif_input().

Best regards

Manfred

[nicolabongiovan]

Thanks for the fast support.

I have checked the value of PBUF_POOL_BUFSIZE in lwipopts.h file: the value is correct (1518).

I have changed also the PBUF_POOL_SIZE and other paramters but nothing.

In my network the TCP frames are fragmented (total data len 65500 ) every 20/50 msec.

My network traffic is enough important.

After a little time the FRDM-K64F goes to assert function above descripted.

Is there some patch for lwip stack?

In this tutorial §Possible Problems

https://mcuoneclipse.com/2015/10/28/tutorial-lwip-with-the-freertos-and-the-freescale-frdm-k64f-boar...

I have found this sentence:

"The problem is if the received frames are handled in an interrupt context and passed up the lwip stack.

This causes problems with reentrance and can cause an assertion."

The toturial suggests to change the define ENET_RECEIVE_ALL_INTERRUPT with this:

     #define ENET_RECEIVE_ALL_INTERRUPT  0

Has NXP identified a solution/patch?

Best regards

Nicola

[manfredschnell]

Hi Nicola,

you're welcome.

My K64 project is setup with FreeRTOS and KSDK2.0.

I changed the ethernet_callback() to notify a low priority task about the new packet. The handling is done in this task.

With MQX you can do something equivalent.

You can have a look on my source.

The callback with IRQ context.

void ethernet_callback(ENET_Type *base, enet_handle_t *handle, enet_event_t event, void *param)
{
#if !NO_SYS
    portBASE_TYPE xWake = pdFALSE;
#endif
    switch (event)
    {
        case kENET_RxEvent:
        // post info from IRQ context to task
        // instead of handling ist direct!!!! 
        // by MS
          //
          // A RTOS is being used.  Signal the Ethernet interrupt task.
          //
          /* Notify the task that the transmission is complete. */
          vTaskNotifyGiveFromISR( xTaskToNotifyETH, &xWake );
          // don't call this in IRQ context (long working time)
          // ethernetif_input(handle, param);
        break;
        default:
        break;
    }
    //
    // Potentially task switch as a result of the above notify write.
    //
    portEND_SWITCHING_ISR(xWake);
}

The simple, low priority Task.

static portTASK_FUNCTION( lwIPInterruptTask, pvParameters )
{
    /* The parameters are not used. */
    ( void ) pvParameters;
    //
    // Loop forever.
    //
    while(1)
    {
        //
        // Wait until the notify has been signalled.
        //
        
        // xTaskToNotifyETH
        ulTaskNotifyTake( pdTRUE, portMAX_DELAY );
        ethernetif_input(&g_handle, &fsl_netif0);
    }
}

Have a nice weekend.

Best regards

Manfred

[nicolabongiovan]

Hi Manfred,

I deepened the lwip problem.

Now I have downloaded the KSDK 2.0 and I use the LWIP with freertos.

I have imported the lwip_tcpecho_freertos_frdmk64f but the lwip problem is still present!

Also I have changed the PBUF_POOL_BUFSIZE to 1518 and PBUF_POOL_SIZE to 50.

By sending consecutive 512 bytes echo requests on TCP port 7 and simultaneously (to stimulate the problem)

requesting a dummy UDP echo to the FRDM-K64F IP address,

after little time the lwip_tcpecho_freertos_frdmk64f application goes to the assert function as indicated above.

It seems there are performance problems to handle memory allocation.

Allowing that the FRDM-K64F can lose Ethernet packets with high IP traffic,

how can I avoid that the program stops on the assert and it runs without losing TCP echo functionality ?

Tanks a lot!

Best regards

Nicola

[manfredschnell]

Hi Nicola,

can you see the call-stack of your system in debugger, when sys_assert() is called?

Perhaps there is a hint to the root cause...

You can have a look at FreeRTOS LwIP SDK v2.0.0 tpcipecho demo crash again.

I posted my reviewed ethernetif_input() function.

The updated LWIP statistics can give you a deeper view of the communication.

Best regards

Manfred

[nicolabongiovan]

Hi Manfred,

I have tried the solution that you have suggested but the lwip problem is still present!

Infact the patch analyses the packet lenght (PBUF_POOL_BUFSIZE issue) but my issue is different.

The ethernetif driver is not robust at Ethernet packets storm from the network.

When a lot of packets came from Ethernet, the ethernetif_input function allocate a lot of pbuf buffer

that come back free only when the upper Ethernet stack handles them.

It' is correct?

So, how can I avoid memory assert when packets storm appears?

Thank you for your support!

PS

From the Debugger I see that the assert function is called from:

  - tcpip_thread(void *arg) in tcpip.c when "tcpip_thread: invalid message:" happens. The msg->type is a random number.

  - pbuf_alloc function in pbuf.c

There seems to be a memory fragmentation problem due to the use of malloc and memory availability issue during a ethernet packets storm.

[manfredschnell]

Hi Nicola,

I've tested my target with a flood of ethernet packets. ( port 7, size 1k, every 10ms).

I get the message:

p != NULL

Fail to allocate new memory space.

out of ethernetif_input().

And then my target is no longer available...

That seems to be an other effect.

--> I have to look deeper on it. But that will be after my days off.

I'll be back, when I have news.

Best regards

Manfred

[nicolabongiovan]

Hi Manfred,

thanks a lot for your test!

Also to me results the error that you have indicated more the assert messages (of the first message)!

The problems are on memory allocation and memory alligment!

I have tested also the RTCS TCP/IP stack for MQX RTOS available only for KSDK up to the version 1.3.0.

With this RTCS TCPIP stack seems that there are not problems.

The application (rtcs example httpsrv + my tcpecho implementation) doesn't crash!

Now I will try to use this with my application and I'll let you know the results of my depth tests!

Still await the output of your analysis to know if there is anyway a solution to LWIP bugs.

Thanks a lot!

Best regards

Nicola

[manfredschnell]

Hi Nicola,

the processing of incomming packets is in tcpip_thread(). And there also happens the freeing of buffers.

In my application the lwIPInterruptTask() and the tcpip_thread() had the same priority. (in my case: 1)

So lwIPInterruptTask() gets fed with packets from the ethernet_callback(). In a message storm condition the  tcpip_thread() isn't active anymore because of equal priority.

I did following changes:

in file "lwipopts.h"

#define TCPIP_THREAD_PRIO       2

in file "lwiplib_MS.c"

#define eth0LWIP_PRIORITY               (TCPIP_THREAD_PRIO-1)

With this change I give tcpip_thread() a higher priority than lwIPInterruptTask(). So the processing of incomming packets works regardless of the incomming packets. --> the packet flood (with: port 7, size 1k, every 10ms) is no problem anymore.

The stack stays responsive.

Please let me know, if you can confirm this.

Best regards

Manfred

[andresmartinez]

Hello Nicola and Manfred, I have the same problem in KSDK 2.0 and KDS 3.0 in one multithread server tcp program.

I try all tips in this post and others but the program continue bug in a few minutes so i try this:

-change de FreeRTOS 8.2.3 that comes to default in KSDK 2.0 and lastest to FreeRTOS 9.0 (downloadable in oficial page).

-The real fix is when you change :

#ifndef PBUF_DEBUG

#define PBUF_DEBUG                      LWIP_DBG_OFF

#endif

to:

#ifndef PBUF_DEBUG

#define PBUF_DEBUG                      LWIP_DBG_ON

#endif

In my case this makes the program run correctly with 5 client connected simultaneously sending every 0.6 sec for more than an hour. I dont sure what is the real problem but this works. :smileyhappy: