GNU Code Coverage on Embedded Target with Eclipse Neon and ARM gcc 5

For a research project, we are going to send a satellite with an embedded ARM Cortex microcontroller into space early next year. Naturally, it has to work the first time. As part of all the ESA paperwork, we have to prove that we tested the hardware and software thoroughly. One pice of the that is to collect and give test coverage evidence. And there is no need for expensive tools: Free-of-charge Eclipse and GNU tools can do the job for a space mission 🙂

Eclipse with Coverage Views

Eclipse with Coverage Views

A while back (see “Code Coverage for Embedded Target with Eclipse, gcc and gcov“) I wrote tutorials about how to use GNU Coverage tools (gcov) with Eclipse for development on embedded ARM Cortex-M devices. Since then, new Eclipse and ARM toolchain versions have been released. As I received recently many questions and requests how to make it work with the MCUXpresso IDE, here we go!

Outline

This tutorial is about how to collect coverage information using GNU gcov with the MCUXpresso IDE (Eclipse Neon with GNU Tools for ARM Embedded 5 toolchain). It describes what has to be added to projects to enable coverage, what compiler and linker settings have to be used and how to retrieve coverage information with semihosting.

💡 The article does *not* go into details how coverage works. For this please check the links at the end of this article, and especially https://mcuoneclipse.com/2014/12/26/code-coverage-for-embedded-target-with-eclipse-gcc-and-gcov/.

The sources of the example project used in this tutorial can be found on GitHub: https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/MCUXpresso/FRDM-K64F/FRDM-K64F_gcov

Preconditions

This article assumes you have all the needed tools for using GNU coverage installed. You need

  • Eclipse IDE: I’m using MCUXpresso IDE v10.0.0 (build 344) which is Eclipse Neon based.
  • GNU Toolchain for ARM: I’m using the ‘GNU Tools for ARM Embedded 5 – Q3‘ which is installed with the MCUXpresso IDE, or use the one from https://developer.arm.com/open-source/gnu-toolchain/gnu-rm.
  • GCov Eclipse plugins: see “Adding GNU Coverage Tools to Eclipse”.
  • Debug probe and debug connection capable to do file I/O semihosting: This means that the application can open, read and write files on the host through the debug connection. In this article I’m using the Segger J-Link debug connection.
  • A functional and working project. I recommend you start playing with an ’empty’ or very simple project. In this tutorial I’m using a project created by the ‘new project wizard’ in the IDE for the NXP FRDM-K64F board.

I’m using the MCUXpresso IDE and its included toolchain in this tutorial. But I describe things in a generic way so it should be applicable to any other configuration you have.

Semihosting with File I/O

Because the approach presented in this article depends on semihosting file I/O, I have found that not every library/debug connection configuration is working for me.

With the ARM gcc 5 Q3-2015 version I was able to use semihosting file I/O with the newlib-nano (slow, but works) and with the newlib library (much faster than newlib-nano). I have not tried the gcc 6 version yet. The RedLib library in MCUXpresso IDE cannot be used as it does not include the necessary gcov libraries.

Update: In case semihosting is not working, try to increase the heap size. To have it working with newlib, I had to increase the heap size from the default 4K to 8K to get it working, or set the stack to 0x2000 and the heap to even 0x5000.

Newlib Heap Size

Newlib Heap Size

💡 I highly recommend to use the newlib (and not newlib-nano) for semihosting and file I/O. Yes, newlib needs more heap and stack (I have set as large as possible), but in In my applications, newlib file I/O is about 30-50 times faster than newlib-nano!

At the time of writing this article, I have tried the following version of the P&E Eclipse GDB plugin, but I was not able to get semihosting file I/O working, but a future version might be able to support it:

GNU ARM PEMicro Interface Debugging Support 3.0.3.201706082119 com.pemicro.debug.gdbjtag.pne.feature.feature.group P&E Microcomputer Systems Inc.

What I have used successfully in this article with semihosting I/O is the following Segger J-Link version, both with the embedded OpenSDA (as on the FRDM-K64F board) and the external J-Link Pro/EDU:

SEGGER J-Link GDB Server V6.16b

What supports semihosting with file I/O is as well the LPC-Link2/CMSIS-DAP debug probe:

LPC-Link2 debugging FRDM-K64F

LPC-Link2 debugging FRDM-K64F

Project

I’m using a SDK v2.2 project with MCUXpresso IDE:

Coverage Example Project in MCUXpresso IDE

Coverage Example Project in MCUXpresso IDE

The project is a simple project created with the wizard. It have pushed the project to GitHub: https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/MCUXpresso/FRDM-K64F/FRDM-K64F_gcov

💡 In addition to the changes below, I usually create a dedicated build configuration (e.g. named ‘Coverage’) so I can switch between coverage and non-coverage mode. For how to use build configurations, see “Build Configurations in Eclipse“.

Coverage Support Files

To record and store coverage information, I have created support files for it. Add the following two files to your project:

  • gcov_support.h
  • gcov_support.c
coverage support files added

coverage support files added

It provides the following:

  • GCOV_DO_COVERAGE: macro/setting to turn on/off coverage.
  • gcov_check(): function to check if semihosting file I/O is working.
  • gcov_write(): function to write the coverage data.
  • gcov_init(): function to initialize the coverage data.

The latest version is inside the GitHub project, and the files are pasted below.

/**
 * \file gcov_support.h
 * \brief Support helpers to use gcov for embedded targets.
 * \author Erich Styger
 * \copyright
 * Web:         https://mcuoneclipse.com
 * SourceForge: https://sourceforge.net/projects/mcuoneclipse
 * Git:         https://github.com/ErichStyger/McuOnEclipse_PEx
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without modification,
 * are permitted provided that the following conditions are met:
 *
 * - Redistributions of source code must retain the above copyright notice, this list
 *   of conditions and the following disclaimer.
 *
 * - Redistributions in binary form must reproduce the above copyright notice, this
 *   list of conditions and the following disclaimer in the documentation and/or
 *   other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
 * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
 * ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 * ###################################################################*/

#ifndef GCOV_SUPPORT_H_
#define GCOV_SUPPORT_H_

#define GCOV_DO_COVERAGE               (1)
  /*<! 1: to enable coverage; 0: to disable it */

/*!
 * \brief Test function to verify file I/O needed for gcov information generation.
 * \return 1 if file I/O does work, 0 otherwise
 */
int gcov_check(void);

/*!
 * \brief Flush and write the coverage information collected so far
 */
void gcov_write(void);

/*!
 * \brief Initialize the coverage information/constructors. Need to call this at the start of main().
 */
void gcov_init(void);

#endif /* GCOV_SUPPORT_H_ */

Below is the implementation file:

/**
 * \file gcov_support.h
 * \brief Support helpers to use gcov for embedded targets.
 * \author Erich Styger
 * \copyright
 * Web:         https://mcuoneclipse.com
 * SourceForge: https://sourceforge.net/projects/mcuoneclipse
 * Git:         https://github.com/ErichStyger/McuOnEclipse_PEx
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without modification,
 * are permitted provided that the following conditions are met:
 *
 * - Redistributions of source code must retain the above copyright notice, this list
 *   of conditions and the following disclaimer.
 *
 * - Redistributions in binary form must reproduce the above copyright notice, this
 *   list of conditions and the following disclaimer in the documentation and/or
 *   other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
 * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
 * ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 * ###################################################################*/

#include <gcov_support.h>
#include <stdint.h>
#include <stdio.h>

void __gcov_flush(void); /* internal gcov function to write data */

int gcov_check(void) {
#if GCOV_DO_COVERAGE
  FILE *file = NULL;

  file = fopen ("c:\\tmp\\test.txt", "w");
  if (file!=NULL) {
    fputs("hello world\r\n", file);
    (void)fwrite("hello\r\n", sizeof("hello\r\n")-1, 1, file);
    fclose(file);
    return 1; /* ok */
  }
  return 0; /* failed */
#else
  return 1; /* ok */
#endif
}

void gcov_write(void) {
#if GCOV_DO_COVERAGE
  __gcov_flush();
#endif
}

/* call the coverage initializers if not done by startup code */
void gcov_init(void) {
#if GCOV_DO_COVERAGE
  void (**p)(void);
  extern uint32_t __init_array_start, __init_array_end; /* linker defined symbols, array of function pointers */
  uint32_t beg = (uint32_t)&__init_array_start;
  uint32_t end = (uint32_t)&__init_array_end;

  while(beg<end) {
    p = (void(**)(void))beg; /* get function pointer */
    (*p)(); /* call constructor */
    beg += sizeof(p); /* next pointer */
  }
#endif /* GCOV_DO_COVERAGE */
}

Enabling Coverage in Application

The following shows the needed modifications in the application to enable coverage:

Enabling Coverage in Application

Enabling Coverage in Application

  1. Include the header file:
    #include “gcov_support.h”
  2. Initialize coverage information, best right after main():
    gcov_init();
  3. Optional: check if file I/O is working properly:
    if (!gcov_check()) {
       /* file I/O is not working */
    }
  4. Anytime in the application, dump the data with:
    gcov_write();

Linker: Symbols for gcov_init()

In gcov_init() I have to initialize the data and constructors for the gcov library. Each source file instrumented for coverage needs to be initialized properly. Because the NXP SDK startup code does not initialize these constructors, I have to call gcov_init() from my application. gcov_init() expects two special symbols for the start and end of the constructors, see ‘Coverage Constructors’ in “Code Coverage for Embedded Target with Eclipse, gcc and gcov” for details.

The default linker script files in MCUXpresso IDE does not generate the needed symbols. The easiest way add this is to use the concept of Linker Script Templates (see menu Help > Help Contents > MCUXpresso IDE User Guide > Memory Configuration and Linker Scripts).

Inside the project root folder, create a folder named ‘linkscripts‘:

Linkscripts Folder

Linkscripts Folder

Inside that folder, create a file named ‘main_text.ldt‘ with the following content:

 /************************************************
 * start of main_text.ldt: *
 ************************************************/
 *(.text*)
 /* added in template for gcov: */
 PROVIDE_HIDDEN (__init_array_start = .);
 KEEP (*(SORT(.init_array.*)))
 KEEP (*(.init_array*))
 PROVIDE_HIDDEN (__init_array_end = .);
 /************************************************ 
 * end of main_text.ldt *
 ************************************************/

This will create the two extra symbols __init_array_start and __init_array_end which are needed by gcov_init().

💡 If not using managed linker scripts, then follow this article how to add it to the linker script file.

Linker: Semihosting

The coverage library will use normal file I/O (fopen(), fwrite(), etc) to write the information to the host using semihosting.

💡 Usually, I don’t recommend to use semihosting, unless you really know what you are doing. In the case of coverage and profiling, it is actually very useful. Semihosting means that using if calling things like printf() or fopen() will trap/halt the target to start communication with the debugger on the host. So it is rather slow, plus if there is no debugger attached, the application on the board might stall! So don’t use semihosting if no active debug session is going on!

In the linker settings, choose the semihosting library:

Semihosting in Linker Settings

Semihosting in Linker Settings

💡 The newlib (semihost) needs more stack and heap, but is by factors faster than newlib-nano.

Linker: –coverage flag

The tell the linker to link with the necessary coverage libraries, add the following flag to the linker settings:

--coverage
Linker Coverage Flag

Linker Coverage Flag

Source Files –coverage flag

Add the following compiler flag to each source file you want to have coverage for:

--coverage
Coverage enabled for main.c

Coverage enabled for main.c

💡 Right-click on a file or folder to set special options for it. See as well “Icon and Label Decorators in Eclipse“.

I recommend only to enable coverage only for the files needed:

  • RAM: coverage needs counters in RAM
  • Time: writing coverage might need several seconds for each file

Coverage information is combined at the end. I can run coverage for one part of the application and then for the other.

Debugger Settings

Make sure your debug (launch) configuration has semihosting enabled:

Segger with Enabled Semihosting

Segger with Enabled Semihosting

For the LinkServer/CMSIS-DAP connection the setting is here:

LinkServer Semihosting Support

LinkServer Semihosting Support

Build

Now build the application. For every coverage instrumented source file, it will generate *.gcno files:

Generated gcno files

Generated gcno files

The files are generated in the same folder as the object files.

Debug

Start a normal debug session. With executing the call to gcov_write() it will write the *.gcda files:

Coverage information file generated with semihosting

Coverage information file generated with semihosting

Inspecting Coverage

Double click on any of the coverage files, and it opens a dialog where I have to specify the binary file used:

Open Coverage File

Open Coverage File

This opens a gcov view in Eclipse:

Coverage Overview

Coverage Overview

With double-clicking on a file/function I get the detail information, including how many times a line has been executed:

Coverage Source View

Coverage Source View

Trouble Shooting

As always, things might not work out the first time. Here are a few tips:

  1. Make sure that you use the same version of gcov and gcc. Check if you have any other GNU tools in your system path.
  2. If the *.gcda files do not show up in the Eclipse Project Explorer view, do a view refresh
  3. File I/O and other semihosting functions need a lot of stack. Increase the application and/or task stack size.
  4. Writing *.gcda files can be very slow with semihosting, depending on file size. Start instrumenting with just one file. Consider using a different debug interface/probe. For me the Segger J-Link is 3 times faster with semihosting file I/O compared to CMSIS-DAP.
  5. Check if gcov_check() can open a file, check the file created. If file I/O fails, verify that you are actually using a semihosting library. Make sure you are not overwriting the file I/O funtions (e.g. do not use a custom _write() function which is part e.g. in the Segger RTT library).
  6. Perform a ‘clean’ or delete the output (usually ‘Debug’) folder and do a clean build.
  7. Increase the heap size for the application. Semihosting with newlib needs at least 8K of heap memory.

Summary

Using the Eclipse IDE for code coverage with GNU gcov is a great addition. It helps me showing progress with my automated testing, and I can easily see which code pieces are not touched by my testing yet.

Using semihosting with file I/O is not the fastest way to save the coverage information to the host, but is a workable solution. I started on an experimental project to port the gcov library so I can use it to save the data to a memory device on the target system or send it to the host using any other connection method (serial, RTT, USB CDC, …). Let me know if you are interested in such a thing and I’ll see if I could write a tutorial about this.

The example project of this tutorial can be downloaded from GitHub: https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/MCUXpresso/FRDM-K64F/FRDM-K64F_gcov

Links

Advertisements

7 thoughts on “GNU Code Coverage on Embedded Target with Eclipse Neon and ARM gcc 5

    • Hi Chad,
      In MCUXpresso IDE there is a SWO profiler which does ‘statistical’ profiling and gives coverage information (statistical by instructions only). It is similar to the concept of gprof (see https://mcuoneclipse.com/2015/08/23/tutorial-using-gnu-profiling-gprof-with-arm-cortex-m/) but using the ARM hardware and SWO pin. MCUXpresso IDE has ETM/ETB and MTB tracing functionality so this information could be used for coverage. However, the concepts are somewhat different: with gcov it uses counters to identify which code blocks have been executed, while the information from ETM/ETB/MTB could be used to do the same thing, it provides a different set of data. The advantage would be that it would not require code instrumentation, but in my view the one of gcov is a rather light one. The biggest challenge with gcov is to get the data off the target: I’m looking into different ways how to get this done without the overhead of semihosting.
      About having SEGGER J-Trace support with or inside Eclipse: that’s a big wish I have for a long time already too.

      Like

  1. Hello Erich

    Thanks for the tutorial! But I have a question, how can I do this in regular eclipse? because I don’t have the “managed linker script” options and I don’t know how to choose the semihosting library.

    Like

  2. Pingback: Using FreeRTOS with newlib and newlib-nano | MCU on Eclipse

What do you think?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s