Tutorial: GNU Coverage with MCUXpresso IDE

If you are developing Linux or desktop applications with GNU tools, you  very likely are familiar with gcov: the GNU coverage tool. It collects data what parts of the code gets executed and represents that in different formats, great to check what is really used in the application code or what has been covered during multiple test runs.

Coverage Information with gcov

Coverage Information with gcov

line never executed

line never executed

GNU coverage is possible for resource constraint embedded systems too: it still needs some extra RAM and code space, but very well spent for gathering metrics and improves the firmware quality. As I wrote in “MCUXpresso IDE V11.3.0 for 2021” things are now easier to use, so here is a short tutorial how to use it.

Outline

For how gcov works, have a read at gcov for Embedded. This tutorial shows how to use gcov with the MCUXpresso IDE V11.3.0. As board I’m using the FRDM-K64F, and you can find the example project on GitHub.

NXP FRDM-K64F Board

NXP FRDM-K64F Board

To generate coverage we need:

  • Eclipse with the gcov plugins (they are already installed in MCUXpresso IDE 11.3.0)
  • GNU binaries without the ‘arm-none-eabi’ prefix: I use a batch file for this
  • A project to collect coverage information

We will instrument source files go produce .gcno files on the host. The instrumented application will generate .gcda files we store on the host using semihosting. The gcov tool then will show the reports in the IDE or on the command line:

General gcov Flow

General gcov Flow

The files and project used in this article can be found on GitHub.

Creating Project

I recommend to get familiar with gcov using a small and bare metal project. I have created a project with the IDE and the MCUXpresso SDK.

Make sure the project is using newlib (nano) with semihosting:

newlib nano with semihost

newlib nano with semihost

This creates the project:

bare metal project

bare metal project

Linker Constructor Symbols

Each instrumented file will generate a special constructor which needs to be called later. To be able to call them, we need to include them into our binary and mark them with special symbols. This is done in MCUXpresso IDE with a special FreeMarker Linker script:

/************************************************
* start of main_text.ldt: *
************************************************/
. = ALIGN(4);
*(.text*)
/* added in template for gcov: */
. = ALIGN(4);
PROVIDE_HIDDEN (__init_array_start = .);
KEEP (*(SORT(.init_array.*)))
KEEP (*(.init_array*))
PROVIDE_HIDDEN (__init_array_end = .);
/************************************************ 
* end of main_text.ldt *
************************************************/

Create a file named main_text.ldt and place it into the folder linkscripts in the project root (or copy it from GitHub):

linker script entry for gcov

linker script entry for gcov

Support Files

To write the data, a few hooks and helpers are needed. You can find them in the ‘gcov’ folder of my project on GitHub:

gcov support files

gcov support files

Add it to the project; we will use it in the next step.

Init and Data Writing

The gcov library needs to be initialized. Include the header file:l

#include "../gcov/gcov_support.h"

Call

gcov_init(); /* initialize library */

at the start of main(). Then run the instrumented code. At the end, call

gcov_write();

to write the data;

gcov init and writing data

gcov init and writing data

Heap and Stack

The application on the board writes the collected data to the host via semi-hosting file I/O. Unfortunately file I/O is not very lightweight, and most issues with using gcov for embedded systems are because lack of stack and heap space.

Typically the default heap and stack size will not be enough, so I recommend to assign a good chunk of memory in case of issues: the more the better.

Heap and Stack Space

Heap and Stack Space

Instrumenting Source Files

Each file which shall be instrumented for coverage needs to have the following options added:

-fprofile-arcs -ftest-coverage

I recommend as a starter just to instrument one file, or very few.

added instrumentation option for coverage

custom options for board.c

custom options for board.c

Finally, add the option

-fprofile-arcs

to the linker settings to ensure it links with the proper library support.

Linker Flags

Linker Flags

💡 instead of -fprofile-arcs the option --coverage can be used.

Now it is a good time to perform a Project > Clean followed by a Project > Build. It shall now generate .gcno files in the output folder:

gcno file

gcno file

Collecting Coverage

Now you can run the application: it will collect information from the instrumented code and write the .gcda files on the host using semi-hosting file I/O:

gcda coverage data files in Debug folder on host

gcda coverage data files in Debug folder on host

Viewing Coverage Information

Double-click on any of the .gda files and it will open a graphical view:

Coverage Information with gcov

Coverage Information with gcov

Tadaaaaa! 🙂

Double-clicking on a line shows the information with colors in the source view:

execution coverage

execution coverage

Gcovr

If you want to use the gcov data outside of Eclipse, have a look at https://github.com/gcovr/gcovr. It has a feature to add and combine different coverage data files too.

Summary

Coverage tells which lines of code have been executed. This is very important to get good test coverage and can increase software quality. Traditionally coverage as GNU gcov has been used on host applications, but it is possible to use it for embedded targets as well. The latest NXP MCUXpresso IDE 11.3.0 makes it even easier as all the needed tools are already installed: that way I can start collecting coverage information from the target and view and analyze it on the host.

Happy covering 🙂

Links

8 thoughts on “Tutorial: GNU Coverage with MCUXpresso IDE

  1. Thanks Erich, very interesting! A couple questions:
    1) C++ projects will already have everything need in linker script, right?
    2) Can you explain why we need GNU binaries without the ‘arm-none-eabi’ prefix?
    Thanks!
    Best Regards, Dave

    Liked by 1 person

    • Hi Dave,
      1) Yes, in that case the constructors get called by __libc_init_array()
      2) Because they are used by the Eclipse plugins (e.g. to find the address of a symbol, etc). Eclipse does not care (or know) if the device is an ARM, so it just calls gcov instead of arm-none-eabi-gcov
      I hope that makes sense.

      Like

  2. Hi Erich,

    Thank you for the tutorial.

    Two questions:
    1. What is the overhead of adding the GCOV functionality? You mention increasing stack size “a good chunk” but is there a rule of thumb of how much to add to the stack and how much is added to the executable (Flash) code?
    2. Is there a way of turning this on/off using build flags? I use a few PRINTFs in my code for development but control whether or not they’re in the build using the “SDK_DEBUGCONSOLE” flag (“0” & “1” for debug, “2” for release). From what I see here, it seems like you have to have two copies of the code, one with the GCOV functionality for testing and one for release.

    Thanx,

    myke

    Liked by 1 person

    • Hi Mike,
      1. There is some data and rational about how much will be used in https://mcuoneclipse.com/2014/12/26/code-coverage-for-embedded-target-with-eclipse-gcc-and-gcov/. Basically there is the overhead in the code to store the data plus the data needed for the arcs. The amount of arcs depend on the control flow (how many changes in flow there are), plus there is the code for the coverage data library. For the example in this article I compared it with a uninstrumented version: FLASH 11kByte vs. 57kByte, Heap 1.4kByte vs. 4.7kByte and stack 120 Bytes vs. 400 Bytes.
      This was with newlib-nano: newlib tends to use more heap and stack (but I rarely use newlib).

      2. Yes, you can turn it off with having a build configuration without the options (instrumentation and linker) and one without. No need to have separate copies of the code.

      Liked by 1 person

      • Thank you for the answers.

        That’s a very significant amount of overhead. I was expecting 10% to 25%, not multiples of the various memory blocks. Years ago, I worked with similar tools on (IBM) mainframe applications and the overhead there was in the range of 5% in terms of additional code and RAM requirements.

        I have to wonder about the practicality of using this tool for MCUs other than in an educational setting. If I’m developing for a product, I want the minimum MCU that will do the job which means as little Flash/SRAM as possible to minimize costs. If I’m doing a reasonably small application, then this tool could be considered for family parts with multiple Flash/SRAM sizes BUT they don’t tend to be in the same packages as the family parts with the smaller Flash/SRAM sizes.

        myke

        Liked by 1 person

        • Hi myke,
          there is base amount which gets added, and this does not grow that much per file: again it depends on the complexity of the code. I just instrumented two more files, and the code size increased from 56752 bytes to 57976 bytes. This was compiling without any optimizations.
          It clearly does not work if you don’t have some headroom in RAM and FLASH: but you don’t have to instrument everything: you can do incremental coverage too.
          Some Kinetis devices are pin compatible with larger RAM and FLASH sizes: for some projects I did the coverage on the larger one.
          The other approach I use is to have the firmware ported to larger device (say the K64F) and with this one I’m able to do coverage for the application part and some parts of the application (like specific hardware on the final device) replaced with mock objects.

          Liked by 1 person

        • Hey Erich,

          That sounds a lot more reasonable in terms of overhead – thank you for doing the checks. The 2% you’re reporting is much more in line with what I would think is reasonable.

          It’s not just Kinetis with the package pin counts being a function of the Flash/SRAM, many other manufacturer’s devices follow a similar approach when selecting packages for the chips inside.

          Keep well and safe!

          Liked by 1 person

  3. Pingback: GNU Code Coverage on Embedded Target with Eclipse Neon and ARM gcc 5 | MCU on Eclipse

What do you think?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.