FreeRTOS ARM Thread Debugging with Eclipse and GDB

FreeRTOS is probably the most popular RTOS used and I love it: it is efficient, has a lot of features, simple and easy to use. But despite its popularity, debugging it with open source tools as with Eclipse and GDB is really not that user-friendly: debugging threads/tasks is a pain compared to commercial solutions. For my university classes this semester I was looking for something easy to use by my students. Finally I managed to use a GDB helper which makes things easier now: I can switch task threads in Eclipse with GDB now πŸ™‚

Switched to Shell Task

Switched to Shell Task

The problem is: when I stop my running FreeRTOS application, it is very likely that it stops in the IDLE task, and only that idle task is shown in the Debug view:

Stopped FreeRTOS Application

Stopped FreeRTOS Application

What I need is an easy way to inspect the other tasks/stacks.

FreeRTOS Eclipse GDB Plugins

There are two plugins I’m aware of which ease the debugging with FreeRTOS in Eclipse:

  1. Wittenstein/High Integrity System Plugins (see DIY Free Toolchain for Kinetis: Part 5 – FreeRTOS Eclipse Kernel Awareness with GDB): free-of-charge, but not open source, need to register/download from Wittenstein. They show tasks/queues and timers. But do not allow to switch to a given task stack.

    Task Table in High Integrity System Plugin

    Task Table in High Integrity System Plugin

  2. Plugin from Code Confidence (http://www.codeconfidence.com/freertos-tools.shtml). This is not open source and not free-of-charge (100 GBP per developer seat, so out of reach for our classes). Additionally, it requires a special launch configuration, so the GNU ARM Eclipse launches are not supported. There is a demo mode which allows 2 threads, and debugging is terminated after 2 minutes.

    Multiple Threads with Code Confidence Plugin

    Multiple Threads with Code Confidence Plugin

FreeRTOS GDB Task Backtrace Switcher

I searched the internet for something what would help me to see the task stacks/threads, and have found a cool contribution in the FreeRTOS community under contributed ports (http://interactive.freertos.org/entries/23468301-Tasks-backtrace-switcher-viewer-snippet-for-debugger-gcc-gdb-ARM-Cortex-M3-MPU-port-Eclipse-support-): it is an excellent example how GDB commands can be used to customize debugging :-).

The approach has two parts:

  1. A custom FreeRTOS vPortPendSVDHandler()
  2. A GDB script with custom commands to show and switch threads

The GDB script is using/setting variables which are then used by the GDB script to switch context. That’s actually a really cool idea, and I got that approach working quickly.

FreeRTOS Port Integration

To make it even easier to use, I have integrated that approach into the FreeRTOS Processor Expert component. So with a simple setting I can turn on that custom handler:

Processor Expert Component to turn on GDB Helper

Processor Expert Component to turn on GDB Helper

That setting turns on a custom FreeRTOS configuration I have added to FreeRTOSConfig.h:

/*-----------------------------------------------------------
 * GDB <span data-mce-bogus="1" class="mceItemHidden"><span class="hiddenSpellError" pre="GDB " data-mce-bogus="1">backtrace</span></span> handler support
 * See http://interactive.freertos.org/entries/23468301-Tasks-<span class="hiddenSpellError" pre="stack " data-mce-bogus="1">backtrace</span>-switcher-viewer-snippet-for-debugger-gcc-gdb-ARM-Cortex-M3-MPU-port-Eclipse-support-
 *----------------------------------------------------------*/
#define configGDB_HELPER (1 &amp;&amp; configCPU_FAMILY_IS_ARM(configCPU_FAMILY) &amp;&amp; (configCOMPILER==configCOMPILER_ARM_GCC)) /* 1: enable special GDB stack backtrace debug helper; 0: disabled */

With that define, I can turn it on/off easily without Processor Expert too. The component copies the GDB script as

Generated_Code\.gdbinit-FreeRTOS-helpers

Usage

Enable the GDB Helper option in the settings. It will create the GDB script plus a help text file (just in case you do not remember this blog article :-)):

Genereated Files with GDB Script

Genereated Files with GDB Script

πŸ’‘ By default, the .* files are not visible in an Eclipse project. I need to customize the view to de-filter the .* resources:

Eclipse Project Filter Settings

Eclipse Project Filter Settings

I need to add the custom gdb commands from the .gdbinit-FreeRTOS-helpers script file.

source .//Generated_Code//.gdbinit-FreeRTOS-helpers

πŸ’‘ With GDB and the GNU ARM Eclipse plugins, the current directory of GDB is the project root folder. Otherwise you need to specify the full path to make sure GDB finds the file.

I can do this with the ‘source’ command in the in the debug/launch configuration so it executed every time I start the debugger in Eclipse:

Debugger Settings and Commands

Debugger Settings and Commands

When I stop the target, I switch to the GDB console (click on arm-none-eabi-gdb or select it in the Console view):

GDB console View

GDB console View

In the console, I use

freertos_show_threads

to show all the tasks with their task handles:

freertos_show_threads

freertos_show_threads

With

freertos_switch_to_task 0x1fff0d30

I switch to the task with that handle:

Switched to Shell Task

Switched to Shell Task

Now I can inspect the call chain of that thread.

IMPORTANT: do not step, only inspect the stack. More about this later.

The same way I can inspect other task stacks or to set breakpoints in tasks.

Before resuming execution, make sure you switch back to the context where weΒ  have stopped execution. For this use

freertos_restore_running_context
Restored Context with freertos_restore_running_context

Restored Context with freertos_restore_running_context

How it works

It is important to understand what happens with the

freertos_switch_to_task

command in .gdbinit-FreeRTOS-helpers:

  1. It writes the task handle to a global variable
  2. Triggers a PendSV exception to call the custom PendSV handler
  3. The custom PendSV handler will stop on a brkt (breakpoint) instruction to stop the debugger
  4. The script will do to instruction steps and the stop to show the thread stack

So it is important to understand that the script actually executes code on the target. That’s why it is important not to continue stepping inside a switched context unless you know that you do! Basically switching context actually really switches the context through the GDB debugger.

Summary

That GDB helper works fine for me, and makes debugging a FreeRTOS application much easier. Best of all: it does not need a plugin, but works with normal GDB commands so can be adopted/changed easily too.

All the sources are available on GitHub. Consult the links at the end of the article. The FreeRTOS component will be available with the next release on SourceForge. If you want to get and try the current .PEupd file before the ‘official’ release, contact me by email (see About), and I make it available.

What could be next?

  • Add support for Cortex-M0+ (I have it tested with Cortex-M4(F))
  • Add a feature that I can specify the task name instead of the task handle address for context switching

I was hoping as well to find a nice GDB front end or something I can add buttons easily to Eclipse without writing a plugin? Anyone has an idea? Maybe we could together build an open source plugin? Looking for volunteers :-).

Happy Tasking πŸ™‚

Links

Advertisements

14 thoughts on “FreeRTOS ARM Thread Debugging with Eclipse and GDB

  1. In the PendSV_Handler for cortex m4 you provided,
    I had to use this:
    ” mov r0, %0 \n”

    instead of this:
    ” mov r0, %%0 \n”

    To make it work.
    Otherwise I would get strange assembler error messages.

    But now it rocks!

    Like

    • Hi Mate,
      ah, yes: that’s because I use this file with Processor Expert, and Processor Expert will read/copy it. To have a ‘%’ in the final source I need to use ‘%%’. So yes, if you want to use that code right away, you have to use only ‘%’.

      Like

  2. Hi!
    Just a theoretical question.
    Would it be possible to use the gdbscript to do the things the modified PendSV handler does?

    It is basically a forced task switch back and then forth.
    All the registers which are popped and then pushed back could be modified by the debugger itself. Or is there something that I missed?
    Thanks!

    Like

  3. This reminds me of a company I used to work for (2007 – 2010).
    They use Nucleos on an embedded processor and 700kB of software divided over 40+ tasks.
    After a few years they stopped paying $$$ every year for their rtos and in the 3 years I worked there nobody was able to figure out how to get a debug interface working. (Spent almost a week on it myself).

    The only available uart was dedicated as an “upload” port for new firmware. All I had were 4 user led’s and a vga monitor on which I could squeeze 1 or 2 text lines of “extra” debug code.

    I have vivid memories of a bug on which I spent a lot of time tracking down. After eliminating more than 95% of the code I had to gave up. One of my colleagues took over and he eventually found the bug wit a logic analyser. It was an overflow in a 16bit pointer in the nucleus kernel which should have been a 32bit pointer. Because the maintenance contract with nucleus was expired we did not even bother to report this back to them.

    In that company they are probably still staring at those 4 leds.

    Like

  4. Pingback: McuOnEclipse Components: 17-Oct-2015 Release | MCU on Eclipse

  5. Thank you once again for this article!

    unfortunately, the freertos_show_threads command is not working for me:

    freertos_show_threads
    Wrong number of arguments for specified format-string

    the command freertos_restore_running_context is known so the script was loaded 😦

    Like

    • sorry for the hassle…

      I did not look at the file carefully and did not notice the “%%” sequence in the init script…

      printf “0x%x\t%s\t\t<—RUNNING\n", $threadid, $thread_name_str
      else
      printf "0x%x\t%s\n", $threadid, $thread_name_str

      Like

      • ah, that’s why! The files get copied by Processor Expert into the project, and ‘%’ starts a special command/variable. To copy ‘%’ I have to specify ‘%%’. Sorry for the problem.

        Like

  6. i went back to this topic as i wanted to have the current state of different tasks and their call stacks shown in eclipse.
    as it turns out, openOCD brings some support for this. I never liked openocd because it was slow and unreliable. but maybe it’s time to give it another try.

    code confidence is offering this for 100 GBP. Plus they introduce a new debugger. Not sure if they keep updating it…

    this shows an example with eCos: http://www.open-etech.com/Tech/OpenOCD/index.php?page=section2
    I will try my luck with recent versions of FreeRTOS and openOCD tomorrow.

    Like

    • after some problems I made to myself, I was able to get the thread view up and running in the eclipse debug perspective. I will comment on the down sides later.

      First of all, here is what i achieved:

      the most important step, which I was to stupid to get right in the first place was the following command line for openOCD.

      -f board/stm32429i_eval_stlink.cfg -c “$_TARGETNAME configure -rtos auto”

      i entered this directly in the “config options” box in the OpenOCD debugger config of GNU ARM eclipse.

      the downsides:
      OpenOCD.
      I quit using OpenOCD because it’s slow and unreliable. And this is still true. gdb is asking for the “info thread” command on each single debugging step you make which makes it really unpleasant to step through the code.
      maybe this could be improved by using J-Link instead of st-link debugger but my experience using this setup was also not nice.

      eclipse also loses the back trace of the shown threads if you to a single step. you need to change the perspective to c/c++ and back to debugging to make the backtrace appear again.
      here is a picture with the disappeared backtrace after a single step:

      unfortunetly, mostly because the slow openocd, i consider this a suboptimal solution 😦

      best regards

      Like

      • Interesting, thanks for sharing! I had bad experience (very slow and buggy9 with OpenOCD 0.8.0 too, so I pretty much stopped using it and using P&E and Segger instead.

        Like

  7. Pingback: Better FreeRTOS Debugging in Eclipse | MCU on Eclipse

What do you think?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s