Using FreeRTOS with newlib and newlib-nano

Posted on by

For reliable applications, I avoid using functions of the standard libraries. They are banned for most safety related applications anyway. I do not use or avoid malloc(), printf() and all the other variants, for many reasons including the ones listed in “Why I don’t like printf()“. Instead, I’m using smaller variants (see “XFormat“). Or I’m using only the thread-safe FreeRTOS heap memory allocation which exist for many good reasons.

Things get problematic if malloc() still is pulled in, either because it is used by a middleware (e.g. TCP/IP stack) or if using C++. Dave Nadler posted a detailed article (http://www.nadler.com/embedded/newlibAndFreeRTOS.html) about how to use newlib and newlib-nano with FreeRTOS.

FreeRTOS Newlib Memory Allocation Scheme

FreeRTOS Newlib Memory Allocation Scheme

The problem with malloc() is not only that it can lead to heap fragmentation and a failure of the program at runtime. In many libraries, it is not reentrant or thread-safe (see https://stackoverflow.com/questions/855763/is-malloc-thread-safe). Using malloc() from a thread or from an interrupt is most likely a very dangerous thing!

Heap_3

Avoid using malloc() is a good approach. But things get difficult if malloc() is used in libraries or middleware stacks. Many USB and TCP/IP stacks use malloc().

Using two different heap implementations (one for FreeRTOS, the other for the middleware) does not make much sense. FreeRTOS offers a thread-safe wrapper to malloc()(wrapper to the newlib malloc() (heap_4.c):

FreeRTOS Thread Safe Malloc Wrapper

FreeRTOS Thread Safe Malloc Wrapper

But that wrapper is effective if the application uses the FreeRTOS API functions pvPortMalloc() and vPortFree(). So what I ended up is changing the middleware sources to use  pvPortMalloc() and vPortFree() instead.

But what if it is not possible to change the library sources? The GNU gcov (coverage) library is one example. This usually is only available as binary library, and rebuilding the GNU library is a challenge on its own (see “GNU Libs with Debug Information: Rebuilding the GNU ARM Libraries“). Even worse: if using C++, then very likely it is using malloc() and variants in the background.

configUSE_NEWLIB_REENTRANT

FreeRTOS offers the configUSE_NEWLIB_REENTRANT setting:

configUSE_NEWLIB_REENTRANT

configUSE_NEWLIB_REENTRANT

With this turned on, FreeRTOS uses a newlib reentrant structure (see this FreeRTOS discussion). This requires several hooks implemented, for details please read Dave’s article.

Newlib FreeRTOS Heap Implementation

The solution is a new FreeRTOS heap allocation scheme:

FreeRTOS Newlib Memory Allocation Scheme

💡 At the time of writing this article, the settings above are available in the implementation on GitHub, but not in the 06-May-2017 SourceForge release.

Heap Base, Limit and Size

With using the new Scheme 6, obviously the normal FreeRTOS heap size setting is not used (configTOTAL_HEAP_SIZE).

The Heap 6 implementation with using newlib needs three symbols defined by the linker:

  • Heap Base (default __HeapBase): start address of heap
  • Heap Limit (default __HeapLimit): end address of heap
  • Heap size (default __heap_size): size of the heap memory in bytes

The symbol settings are available in the FreeRTOS Processor Expert component:

Linker Heap Symbols

Linker Heap Symbols

The real symbol names depend on your system and linker file. Processor Expert projects for example use __heap_size:

__heap_size = 0x00; /* required amount of heap */
__heap_size

__heap_size

The heap size is configured in the CPU settings:

Heap Size in Processor Expert

Heap Size in Processor Expert

Freescale/NXP Kinetis SDK projects use HEAP_SIZE:

HEAP_SIZE = DEFINED(__heap_size__) ? __heap_size__ : 0x0400;
HEAP_SIZE

HEAP_SIZE

The two other symbols are for the start and the end of the heap, by default __HeapBase and __HeapLimit. Typically they are like this in the linker file (example from Kinetis Design Studio):

 /* User_heap_stack section, used to check that there is enough RAM left */
 ._user_heap_stack :
 {
   . = ALIGN(4);
   PROVIDE ( end = . );
   PROVIDE ( _end = . );
   PROVIDE ( __end__ = . );
   __heap_addr = .;
   __HeapBase = .;
   . = . + __heap_size;
   __HeapLimit = .;
   . = . + __stack_size;
   . = ALIGN(4);
 } > m_data

MCUXpresso IDE does this:

 .heap : ALIGN(4)
 {
 _pvHeapStart = .;
 . += 0xa000;
 . = ALIGN(4);
 _pvHeapLimit = .;
 } > SRAM_UPPER

As shown above, there is no defined symbol for the HEAP_SIZE. This can be added to the MCUXpresso IDE:

  1. Have a folder named ‘linkscripts’ in the project root
  2. Add a file named ‘user.ldt’
  3. Add the following content to the file:
    HEAP_SIZE = ${HEAP_SIZE};
HEAP_SIZE Symbol with Managed Linker Scripts in MCUXpresso IDE

HEAP_SIZE Symbol with Managed Linker Scripts in MCUXpresso IDE

Otherwise, check your linker file for the correct symbols. If not, you have to add them.

With this, you can use FreeRTOS with newlib memory allocation in a safe and reentrant way :-).

Summary

Using newlib and newlib-nano and its memory allocation routines are problematic, especially to use it in a reentrant and thread-safe way. Malloc() can get used as a side effect with calling functions like printf(), strtok() and others in the ‘black box’ of newlib. If you have to use things of newlib or newlib-nano, then make sure you use configUSE_NEWLIB_REENTRANT with the heap_useNewlib.c from Dave Nadler.

I have integrated Dave’s work into FreeRTOS, including a port to Processor Expert. If you want to check out the implementation, see https://github.com/ErichStyger/McuOnEclipse_PEx/tree/master/Drivers/freeRTOS.

💡 Many thanks to Dave Nadler for investigating and searching for a solution and give it back to the community!

Happy Mallocing 🙂

Links

16 thoughts on “Using FreeRTOS with newlib and newlib-nano”

  1. Hello, Erich

    This article is very interesting. Congratulations! A question: How can I implement heap 6 and newlib nano in a SDK project without Processor Expert in MCUXpresso? I noticed that MCUXpresso adopts heap 4, by default.

    Like

    Reply ↓

      • Thanks for the tips, Erich

        I just installed the FreeRTOS version you recommended. What is the version of this FreeRTOS? That is not explicit in the directory name. Is it a custom version?

        KDS SDK 2.0 FreeRTOS version is 8.2.3 and MCUXpresso SDK 2.2 FreeRTOS version is 9.0.0. In none of them, there is a heap_6.c file.

        Also, I haven’t found heap_6.c file in your FreeRTOS. I suppose I have to take heap_useNewlib.c file and rename it as heap_6.c and so add it to the project and remove heap_4.c, which is usually the default heap file used by SDK. Please correct me if I’m wrong…

        Thanks,

        Marco Coelho

        Like

        Reply ↓

        • Hi Marco,
          it is the version 9.0.0 with some of the upcoming features present on the ‘bleeding edge’ version from on https://sourceforge.net/p/freertos/code/HEAD/tree/, plus my extensions to support Segger SystemView and Percepio Tracealizer.
          And yes, what you are looking for is not heap_6.c, but heap_useNewlib.c. The name of the file does not matter at all. All what you need to make sure is to link only one version of the heap, I’m doing this with the configUSE_HEAP_SCHEME macro.

          I hope this helps,
          Erich

          Like

  2. Erich, thanks again for your efforts! It took me all evening but I got my CodeWarrior 10 project moved from EWL to newlib-nano and set up for newlib use in FreeRTOS. In case anyone else is attempting this, the EWL to newlib instructions in the CW docs are a good start, but you’ll need to not include -lrdimon and newlib expects __stack to define the top of stack or it’ll trash whatever your setup code set SP to when it uses its default.

    As for the FreeRTOS part, the big caveat would seem to be that the _reent struct adds more than 1 kB to the RAM overhead for each task. That’s pretty huge, especially since it’s mostly useless stuff like atexit() data. Defining _REENT_SMALL knocks that down by over 800 bytes. I’ll see later what I can do to reduce it more.

    I also had to set setbuf(stdout, NULL) for each task. Seems wasteful to use stdio when I’m just wrapping my own output functions anyway, but I’ve done it in this project for the sake of consistency.

    I discovered that I somehow had _DEBUG set to 1 in the Freescale bare metal USB stack and it was doing printf()s. No idea how long it’s been that way, but for some reason they weren’t causing trouble with EWL but immediately crashed with newlib.

    Switching to newlib increased my code size by around 10 kB but I haven’t had time yet to check where the increase was and if it was just a matter of configuration options. Don’t really care at this point, I’m just trying to get off of EWL and on to something still being supported and documented.

    Now what I want is a second heap, so I can separate the non-critical stuff (mostly interactive tasks that can safely fail) with more important stuff. There’s very little important stuff using dynamic memory allocation, but it’d be nice to reserve some memory for things like the task that occasionally checks a server for firmware updates, without the possibility that a user script will have used up all of the available heap.

    Like

    Reply ↓

    • Hi Scott,
      sorry for my delayed answer, cracy times :-(.
      Thanks for sharing all your steps and experience! You mention printf(), and I know that newlib and newlib-nano might use the heap memory for internal data structures.
      And placing the heap heap might be sensitive to the _sbrk() function, as this one tries to check for a stack overflow, but in case of FreeRTOS it would trigger a false alarm (see https://mcuoneclipse.com/2016/08/06/semihosting-again-with-nxp-kinetis-sdk-v2-0/ or google McuOnEclipse about _sbrk()). That might have been your crash with newlib().

      Like

      Reply ↓

      • While I was doing my initial testing I set a breakpoint in malloc() and didn’t see any calls from printf(), but I’ll need to double check that I was looking in the right place.

        I’ve moved the heap up to the top of SRAM_U, and the stack is down in SRAM_L. No trouble there so far. My printf() crash might have been because the USB stack only prints messages on errors and I hadn’t come across those errors until I screwed something else up during the change. I’ll pay close attention when I migrate the next project.

        The only other newlib quirk I’ve run into is that the ctype.h functions are implemented as macros with a lookup table, which throws some new GCC warnings about char subscripts because apparently I never paid attention to the fact that those functions take int parameters.

        I’m eliminating PEx components one by one and eventually I’ll try moving this to MCUXpresso. Probably not till after Christmas, though – can’t spare that much time for anything not needed for getting this project finished.

        Like

        Reply ↓

      • Found another difference – looks like maybe newlib doesn’t support the sscanf() hh format specifier. Serves me right for switching to that newfangled C99 stuff a few months ago. I’ve gone years without using any variety of scanf() at all, but I got lazy and used it in this project for parsing a VT100 cursor position message.

        Like

        Reply ↓

  3. Hi Erich,

    In freertos_tasks_c_additions.h I ran into this:

    #if ((configUSE_HEAP_SCHEME > 5) || (configUSE_HEAP_SCHEME < 1))
    #error "Invalid configUSE_HEAP_SCHEME setting!"
    #endif

    Of course, I"m using scheme 6. I'm certain I ran into this before and fixed it, but I don't remember how. I can get it to compile by changing the 5 to 6, but I'm sure that's not what I did before. I also don't know what I changed to cause this to pop up again. What am I missing?

    Thanks,

    Scott

    Like

    Reply ↓

  4. Erich any suggestions how to define the heap base, limit and size symbols when using FreeRTOS v10.0.1 with newlibnano on MCUXpresso (SDK v2, not PE) ?

    Like

    Reply ↓

  5. Hello Erich,

    Very interesting article!
    Was pondering how to solve this situation with two heaps as well.

    Hereby my ten cents…

    It is possible to make the newlib malloc() and free() functions threadsafe by wrapping them using the –wrap function of gcc.

    By adding “-Wl,–wrap=malloc,–wrap=free” to the linker command line

    The original malloc() and free() function are renamed to __real_malloc() and __real_free() and all reference in the code to malloc() and free() are renamed to __wrap_malloc() and __wrap_free().

    Allowing us to implement __wrap_malloc() and __wrap_free() our selves and call the original __real_malloc() and __real_free() after making sure no context switches occur while in the __real function.

    Hope this helps anybody…

    Like

    Reply ↓

  6. Pingback: New NXP MCUXpresseo IDE V10.3.0 Release | MCU on Eclipse

  7. Pingback: Steps to use FreeRTOS with newlib reentrant Memory Allocation | MCU on Eclipse

What do you think?

This site uses Akismet to reduce spam. Learn how your comment data is processed.