BUG: malloc overruns heap, returns invalid pointer, and corrupts memory

[davenadler]

A simple example (created by pruning down an NXP blinky example) shows malloc over-running heap and returning an invalid pointer. Obviously this corrupts memory in any non-trivial program.
KDS 3.2, Freedom K64F board.
Two bugs are illustrated:

1. malloc is supposed to return NULL when no memory is available.
   Please see: http://www.cplusplus.com/reference/cstdlib/malloc/
   For embedded applications, a trap on out-of-memory would be better!
   In no case is returning a pointer outside the heap acceptable.
2. As shown in this example, the default heap size of 1kB (for a processor with 256kB RAM!)
   in K64F example is too small for even a trivial RTL use, even before the application tries to use malloc.
   NXP examples should set sensible default heap size!

Here's the relevant test code (complete project is attached):

// Quick-and-Dirty Blinky7 example for Freedom K64F
// Shows malloc memory over-run and return of invalid (non-NULL) pointer

#include <stdint.h>
// Debug only:
#include <stdio.h> // printf
#include <stdlib.h> // malloc
extern uint32_t __HeapBase;
extern uint32_t __HeapLimit;
// ....
int main(void)
{
    BOARD_InitPins();
    BOARD_BootClockRUN();
    printf("Hello there; this tests semi-hosting (and printf internally calls _malloc_r, and over-runs heap)\n");
    void* pHeapBase = ((void*)&__HeapBase);
    void* pHeapLimit= ((void*)&__HeapLimit);
    void* pMalloc = malloc(16); // does malloc return something within expected heap area? Ooops: 0x200005c0
    printf("Heap: base 0x%x, limit 0x%x, very first malloc 0x%x\n",pHeapBase, pHeapLimit, pMalloc);
    // Prints: Heap: base 0x20000000, limit 0x20000400, very first malloc 0x200005c0
    assert (pMalloc>=pHeapBase && pMalloc<pHeapLimit);
}

Original Attachment has been moved to: 20161228_malloc_overrun_demo.zip

[davenadler · Accepted solution]

Numerous examples from Freescale/NXP set up heap incorrectly for newlib.
Examples for FreeRTOS also do not configure FreeRTOS correctly (if you are using anything requiring heap, for example printf using floating point).

Please see the following for detailed explanation, plus code and link file that actually work for FreeRTOS:
How to safely use newlib with FreeRTOS

Hope that helps!
Best Regards, Dave

[bobpaddock]

"NXP examples should set sensible default heap size!"

According to safety standards such as MISRA the correct heap size is always zero.

It is saying the heap shall never be used in Embedded projects, due to such things as fragmentation that can appear only with time making systems impossible to test.

https://www.misra.org.uk/ The Motor Industry Software Reliability Association

The point is what is 'reasonable' in one project is dangerous in an other.

In my view the correct default behavior for any tool set is always start with heap size of zero and malloc and friends return NULL.  This forces the user to determine and set an appropriate heap size for their project then adjust the linker script as required.

Mastering stack and heap for system reliability: Part 1 – Calculating stack size | Embedded 

Mastering stack and heap for system reliability: Part 2 - Properly allocating stacks | Embedded 

Mastering stack and heap for system reliability: Part 3 - Avoiding heap errors | Embedded 

How to Prevent and Detect Stack Overflow | Barr Group 

[Alice_Yang]

Hello Bob,

When develop on bare mental environment , it seems we need implement an _sbrk function

by yourself, I seared some example , you bring them as reference :

c - malloc() doesn't return null when overruns the HEAP (bare-metal) - Stack Overflow 

malloc doesn't return NULL when run out of heap · Issue #76 · SuperHouse/esp-open-rtos · GitHub    

BR

Alice

[bobpaddock]

See page 498 here:

http://pubs.opengroup.org/onlinepubs/9695969499/toc.pdf 

A simplistic incomplete and dangerous version:

/* ------------------------------------------------------------------------------------
* _sbrk(len) -- Allocate space on the heap

* Return -1 if there is no heap or no heap space.
*/
extern char __heap_start[]; /* Defined by the linker script */
static char *heap_end = __heap_start; /* Start with empty heap */

char *_sbrk(int const incr); /* Amount to increases heap size */
char *_sbrk(int const incr)
{
heap_end += incr; /* \todo check for collisions with the stack [See below how to read SP in GCC]*/

return( (heap_end - incr) );
}

*/
static __inline__ void *sp_get(void)/* Return the current stack pointer */

{
void *sp;

__asm__ __volatile__ ("mrs %0, msp" : "=r"(sp));

return( sp );
}

[Alice_Yang]

Hello Dave,

I'm so sorry reply you so late.

Below is the explanation KDS compiler expert give :

"

Yes you are right, malloc doesn't return NULL pointer when it overrun the heap size. I would not expect it, but after analyzing different situations it showed up, that malloc() returns NULL pointer only if there is not enough space neither in HEAP nor in STACK. I found this discussion on StackOverflow forum:
http://stackoverflow.com/questions/39113658/when-does-malloc-return-null-in-a-bare-metal-environment
They are talking about the HEAP and STACK collision, what could be related with this problem. Although, i am not sure if we can talk about collision even if both (STACK and HEAP) address ranges are managed. For example if i define an local array with 10 integers it will occupy cca 40-48 bytes at the top of the STACK. It means that this part of stack is not available for dynamic allocation and malloc() returns NULL if you try to allocate address space bigger than HEAP+STACK-48bytes. In my project 0x400 + 0x500 - 48bytes(mentioned array) - other local variables.
This is the reason why i think that malloc function works safely and it cannot happen that the dynamic allocation will overwrite some variables in stack.

"

Have a great day,
Alice Yang

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------