Memory Leak problem in LPC1788 with FreeRTOS and LWIP

lpcware · ‎06-15-2016

Content originally posted in LPCWare by amarpol1305 on Wed Apr 09 05:46:13 MST 2014
Hello all,

I am implementing LWIP based web server on LPC1788 with FreeRTOS and its working but after few hits (loading page from browser) LWIP get down.

I think this is the case of memory leake, can any one help me out to solve this problem

Thank you.

mariob · ‎05-29-2017

Hello everybody,

I habe the same problem with the webserver demo application.

Do I need to change the chip manufacturer, or is there a solution?

Best Regards,

Mario

Carlos_Mendoza · ‎05-31-2017

Hi greg and Mario,

We found a workaround to resolve this issue in the webserver project, the suggested workaround is this:

In the tcp_impl.h file change the definition of TCP_MSL to the following:

#define TCP_MSL 100 //1000 //60000UL /* The maximum segment lifetime in milliseconds */

It means the memory is freed more quickly when closing connection.

I tried the value 100 and 1000 on lpcopen webserver example on lpc1788 and it can work.

We analyze the cause should be that when TCP connection is closed, the PCB is not released in time due to the long (120s) time waiting. This cause the PCB created more and more so that memory leaks due to the limited memory of MCU (SRAM).

Hope it helps!

Best Regards,
Carlos Mendoza
Technical Support Engineer

mariob · ‎06-06-2017

Hi Carlos,

thank you for the workaround. First I want to say, I'm working on a LPCpresso 1769 Board.

I changed the value of TCP_MSL to 100, as recommended. I put

Hard_fault_Handler.

The "vecthpc" Register shows the following status:

Faults = IBusErr

Fault PC = 0x10008000

= No symbol matches 0x10008000.

= No line number information available for address 0x10008000

Fault status registers:

IPSR = 0x1000003: 3 (HardFault)

CFSR = 00000100 (Configurable Fault Status Register)

HFSR = 40000000 (Hard Fault Status Register)

DFSR = 00000000 (Debug Fault Status Register)

MMAR = e000edf8 [INVALID] (MemManage Fault Address Register)

BFAR = e000edf8 [INVALID] (Bus Fault Address Register)

AFSR = 00000000 (Auxiliary Fault Status Register)

Stacked registers:

R0 = 00000004

R1 = 0000007b

R2 = 00000004

R3 = 10008000

R12 = 10001df4

LR = 0000ced9

PC = 10008000

PSR = 00000000

SP = 10001638

Could you please give me an advice to resolve this issue?

Best Regards

Mario

Carlos_Mendoza · ‎06-15-2017

Hi Mario,

Are you using the unmodified webserver project? You could extend your hardFault handler to get more information about the MCU state before it happens:

https://community.nxp.com/message/630605

Best Regards!
Carlos Mendoza
Technical Support Engineer

robertschrem · ‎07-18-2017

Hi Carlos,

I have the same issue: The completely unchanged weberver example code stops after ca. 100 requests with a hard fault. This is also true for the webserver_freetos example. Please try it yourself!

Your proposed "fix" above does not change anything for me.

To build the exampels I used your latest and greatest MCU Expresso IDE and the OM13085 Dev Kit from embedded artists as atervtised on your web site.

Frustrating if even the most basic examples code crashes right out of the box after light usage. :smileysad:

Some examples do not even compile in a Mac environment as they use in #include statements the non standard folder delimiter \ (instead of /) that only seems to work on PC environments.

So any solution or any activity / willingness to fix your obviously broken example code that comes along with your dev kits?

Best regards,

Robert

gborowik · ‎05-07-2017

Hi everyone,

Is there any update for this thread?

The problem still occurs and has not been fixed for more than a year.
I'm able to reproduce the last issue, even with the latest lwip 2.0.2 and LPCOpen v2.10.

I know that there is an option such Keil, but I am glued to LPCXpresso using macOS because Keil is generally for MS Windows (virtualization is not an option for me).

Has anyone tried to port FreeRTOS-Plus network services, especially a web server, to LPC1788?
I would be very grateful if any of you would give me an access to such resources.

Thank you,
Greg

lpcware · ‎06-15-2016

Content originally posted in LPCWare by medake on Tue Apr 29 03:21:41 MST 2014
Dear Gideon,

Thank you for the reply.

Yeah. I read about Ethernet DMA and I checked that my Keil project also uses pbufs in
peripheral RAM to keep the Rx and Tx descriptors.

Whenever we try with different IDEs (LPCXpresso, Keil or IAR) it is toolchain and C libraries that differs.
It is acceptable that behavior differs because of back end entities like toolchain and libraries.

Since LPCXpresso uses GCC, I wish there should be a stable port of LWIP + FreeROTS for GCC should be released
so as to develop and run complex TCP and/or threaded applications those can handle bunch of data transactions
without memory failure.

Although NXP has provided a stable port and the examples also execute fine; but it is frequently a question on the list when
it comes to applications like webserver.

If these malloc bugs gets solved by some way, then LWIP + FreeRTOS can be used with LPCXpresso and GCC
to develop more sophisticated applications using NXP platforms.

Best Regards,
Mangesh Edake

lpcware · ‎06-15-2016

Content originally posted in LPCWare by Gideon196 on Mon Apr 28 07:46:47 MST 2014
I also wish to comment on why the Ethernet pbufs need to be in peripheral SRAM. Careful reading of the LPC178x (and LPC40xx) user manual(s) reveal that the Ethernet DMA is connected to the peripheral SRAM, but not connected to the main SRAM.

lpcware · ‎06-15-2016

Content originally posted in LPCWare by Gideon196 on Mon Apr 28 07:42:00 MST 2014
Hello Mangesh,

I don't have an answer for you. I just wanted to mention that I have observed similar things when working on other projects.

For example, I have a particular lwip/FreeRTOS project that works OK with KEIL. But when recompiled with IAR it doesn't work. I performed enough troubleshooting to determine that the heap memory in my project becomes internally fragmented in the IAR version such that the application's malloc() calls fail--there isn't a free block available that's large enough to satisfy the malloc() request. It seems that the KEIL heap management strategies are compatible with the application's heap usage patterns, but the IAR heap management strategies are not.

I note that in my application I also have the following defines set in the lwipopts.h file:

#define MEM_LIBC_MALLOC 1
#define MEMP_MEM_MALLOC 1

This selection causes the lwip stack to perform a lot of heap activity.

I haven't decided on the best way to fix my problem, but at least I know what's happening in my case.

lpcware · ‎06-15-2016

Content originally posted in LPCWare by medake on Mon Apr 28 03:14:01 MST 2014
Hello Gideon, Richard and other folks on forum,

Why the combination of FreeRTOS and lwIP is having unstable behavior for 1788?

I have created a project in LPCXpresso v6.1.4_194 that acts as a httpd (webserver)
on top of Lwip and FreeRTOS.

For this project, the default memory configuration in LD script in LPCXpresso is,
text + RO data => On chip Flash (starting at 0x00; Size=512K)
RW data + BSS + Heap + Stack   => Peripheral RAM (starting at 0x20000000; Size=32K)
And as per code Nothing goes to => Internal RAM (starting at 0x10000000; Size=64K)

This project is configured to use a standard C library malloc provided by Code Red C library.
#define MEM_LIBC_MALLOC                   1
#define MEMP_MEM_MALLOC                 1

The webserver uses netconn API (RAW and SOCKET are disabled in lwipopts.h)

RESULT: webserver doesn't respond to the page hit count more than 50.

I tried shifting the c library based malloc to LWIP Heap (ram_heap) based malloc.

First of all program didn't compile due to space limitation, since all RAM used by the program
was the peripheral RAM (32K) and it was not sufficient for the BSS (of course to LWIP buffers, mem pools etc)

So I tried shifting the BSS (LWIP bufs) to Internal RAM (62K), this time BSS got fit into the available RAM;
but the Ethernet driver disliked the internal RAM, since it needs all the pbufs in peripheral RAM only.
So program got compiled but executed with the ASSERT about the pbufs boundary not as per requirement etc.

I tried few more combinations,
pbufs in peripheral RAM, rest everything in IRAM ==> RESULT nothing (no o/p on hypertermial)
pbufs + stack + heap in peripheral ram,   rest everything in IRAM ==> RESULT nothing (no o/p on hypertermial)

Now,
if we consider a case of inadequate memory, I have a port of Lwip-1.3.2 on uCos-II
that works perfectly on same LPC1788 board. The project is created and compiled in the Keil.
Same webserver (application) code using netconn API, executes seemlessly for
any number of page hits without hanging.

I have cross-checked the lwipopts.h file for the different configuration options. They are almost same.
I checked with memory configurations in Keil, I see a it used internal RAM (64K) for everything
except the Tx and Rx descriptors are located in peripheral RAM (32K).

Can anyone guide to achieve the same result with LPCXpresso ?

Best Regards,
Mangesh Edake

lpcware · ‎06-15-2016

Content originally posted in LPCWare by rgledhill on Fri Apr 11 00:34:41 MST 2014
Hi Gideon,

That is extremely helpful, thank you!! It seems the combination of FreeRTOS and lwIP throws up more problems than anything else on platforms, and it's mostly to do with bugs like this or inadequate stack.

Cheers
Richard

lpcware · ‎06-15-2016

Content originally posted in LPCWare by Gideon196 on Thu Apr 10 17:40:00 MST 2014
I was able to reproduce this with the LPCOpen v2.10 webserver_freertos example.

I noticed three problems.

1) The memory allocation routines are not thread safe. This will cause the example to lock up in the HardFault exception handler.
2) The vApplicationMallocFailedHook function should not stop, it should return to allow the lwip stack to recover from out of memory conditions.
3) There is not enough heap memory available for the example to keep up with a long sustained flood of hits. It will run out of memory and begin dropping packets. It will recover, however, when the traffic is reduced.

A fix for the first problem is to add the following lines to the end of the lwipopts.h file. (./applications/lpc17xx_40xx/examples/lwip/webserver_freertos/configs/lwipopts.h). I've attached the modified file.

#include <stdint.h>
#include <string.h>
#include "lpc_types.h"
#include "FreeRTOS.h"

#define mem_free vPortFree

#define mem_malloc  pvPortMalloc

STATIC INLINE void *pvPortCalloc(size_t nmemb, size_t size)
{
void *x = mem_malloc(nmemb * size);
if (x != NULL)
memset(x, 0, nmemb * size);
return x;
}

#define mem_calloc pvPortCalloc

A fix for the second is to modify vApplicationMallocFailedHook() from FreeRTOSCommonHooks.c (./software/freertos/freertoslpc/FreeRTOSCommonHooks.c) as follows. The modified file is attached.

/* FreeRTOS malloc fail hook */
void vApplicationMallocFailedHook(void)
{
DEBUGOUT("DIE:ERROR:FreeRTOS: Malloc Failure!\r\n");
//FIXME: Comment out the following two statements so this function will 
//       return.  This allows the lwip stack to recover from out of heap
//       errors.
//taskDISABLE_INTERRUPTS();
//for (;; ) {}
}

I don't have a fix right now for the third problem. I'll post an update if I'm able to come up with something.