For a research project, we are going to send a satellite with an embedded ARM Cortex microcontroller into space early next year. Naturally, it has to work the first time. As part of all the ESA paperwork, we have to prove that we tested the hardware and software thoroughly. One piece of the that is to collect and give test coverage evidence. And there is no need for expensive tools: Free-of-charge Eclipse and GNU tools can do the job for a space mission 🙂

A while back (see “Code Coverage for Embedded Target with Eclipse, gcc and gcov“) I wrote tutorials about how to use GNU Coverage tools (gcov) with Eclipse for development on embedded ARM Cortex-M devices. Since then, new Eclipse and ARM toolchain versions have been released. As I received recently many questions and requests how to make it work with the MCUXpresso IDE, here we go!

💡 For a newer tutorial how to use gprof in MCUXpresso IDE, see “Tutorial: GNU Coverage with MCUXpresso IDE“.

Outline

This tutorial is about how to collect coverage information using GNU gcov with the MCUXpresso IDE (Eclipse Neon with GNU Tools for ARM Embedded 5 toolchain). It describes what has to be added to projects to enable coverage, what compiler and linker settings have to be used and how to retrieve coverage information with semihosting.

💡 The article does *not* go into details how coverage works. For this please check the links at the end of this article, and especially https://mcuoneclipse.com/2014/12/26/code-coverage-for-embedded-target-with-eclipse-gcc-and-gcov/.

The sources of the example project used in this tutorial can be found on GitHub: https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/MCUXpresso/FRDM-K64F/FRDM-K64F_gcov

Preconditions

This article assumes you have all the needed tools for using GNU coverage installed. You need

• Eclipse IDE: I’m using MCUXpresso IDE v10.0.0 (build 344) which is Eclipse Neon based.
• GNU Toolchain for ARM: I’m using the ‘GNU Tools for ARM Embedded 5 – Q3‘ which is installed with the MCUXpresso IDE, or use the one from https://developer.arm.com/open-source/gnu-toolchain/gnu-rm.
• GCov Eclipse plugins: see “Adding GNU Coverage Tools to Eclipse”.
• Debug probe and debug connection capable to do file I/O semihosting: This means that the application can open, read and write files on the host through the debug connection. In this article I’m using the Segger J-Link debug connection.
• A functional and working project. I recommend you start playing with an ’empty’ or very simple project. In this tutorial I’m using a project created by the ‘new project wizard’ in the IDE for the NXP FRDM-K64F board.

I’m using the MCUXpresso IDE and its included toolchain in this tutorial. But I describe things in a generic way so it should be applicable to any other configuration you have.

Semihosting with File I/O

Because the approach presented in this article depends on semihosting file I/O, I have found that not every library/debug connection configuration is working for me.

With the ARM gcc 5 Q3-2015 version I was able to use semihosting file I/O with the newlib-nano (slow, but works) and with the newlib library (much faster than newlib-nano). I have not tried the gcc 6 version yet. The RedLib library in MCUXpresso IDE cannot be used as it does not include the necessary gcov libraries.

Update: In case semihosting is not working, try to increase the heap size. To have it working with newlib, I had to increase the heap size from the default 4K to 8K to get it working, or set the stack to 0x2000 and the heap to even 0x5000.

💡 I highly recommend to use the newlib (and not newlib-nano) for semihosting and file I/O. Yes, newlib needs more heap and stack (I have set as large as possible), but in In my applications, newlib file I/O is about 30-50 times faster than newlib-nano!

At the time of writing this article, I have tried the following version of the P&E Eclipse GDB plugin, but I was not able to get semihosting file I/O working, but a future version might be able to support it:

GNU ARM PEMicro Interface Debugging Support 3.0.3.201706082119 com.pemicro.debug.gdbjtag.pne.feature.feature.group P&E Microcomputer Systems Inc.

What I have used successfully in this article with semihosting I/O is the following Segger J-Link version, both with the embedded OpenSDA (as on the FRDM-K64F board) and the external J-Link Pro/EDU:

SEGGER J-Link GDB Server V6.16b

What supports semihosting with file I/O is as well the LPC-Link2/CMSIS-DAP debug probe:

Project

I’m using a SDK v2.2 project with MCUXpresso IDE:

The project is a simple project created with the wizard. It have pushed the project to GitHub: https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/MCUXpresso/FRDM-K64F/FRDM-K64F_gcov

💡 In addition to the changes below, I usually create a dedicated build configuration (e.g. named ‘Coverage’) so I can switch between coverage and non-coverage mode. For how to use build configurations, see “Build Configurations in Eclipse“.

Coverage Support Files

To record and store coverage information, I have created support files for it. Add the following two files to your project:

• gcov_support.h
• gcov_support.c

It provides the following:

• GCOV_DO_COVERAGE: macro/setting to turn on/off coverage.
• gcov_check(): function to check if semihosting file I/O is working.
• gcov_write(): function to write the coverage data.
• gcov_init(): function to initialize the coverage data.

The latest version is inside the GitHub project, and the files are pasted below.

/**
 * \file gcov_support.h
 * \brief Support helpers to use gcov for embedded targets.
 * \author Erich Styger
 * \copyright
 * Web:         https://mcuoneclipse.com
 * SourceForge: https://sourceforge.net/projects/mcuoneclipse
 * Git:         https://github.com/ErichStyger/McuOnEclipse_PEx
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without modification,
 * are permitted provided that the following conditions are met:
 *
 * - Redistributions of source code must retain the above copyright notice, this list
 *   of conditions and the following disclaimer.
 *
 * - Redistributions in binary form must reproduce the above copyright notice, this
 *   list of conditions and the following disclaimer in the documentation and/or
 *   other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
 * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
 * ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 * ###################################################################*/

#ifndef GCOV_SUPPORT_H_
#define GCOV_SUPPORT_H_

#define GCOV_DO_COVERAGE               (1)
  /*<! 1: to enable coverage; 0: to disable it */

/*!
 * \brief Test function to verify file I/O needed for gcov information generation.
 * \return 1 if file I/O does work, 0 otherwise
 */
int gcov_check(void);

/*!
 * \brief Flush and write the coverage information collected so far
 */
void gcov_write(void);

/*!
 * \brief Initialize the coverage information/constructors. Need to call this at the start of main().
 */
void gcov_init(void);

#endif /* GCOV_SUPPORT_H_ */

Below is the implementation file:

/**
 * \file gcov_support.h
 * \brief Support helpers to use gcov for embedded targets.
 * \author Erich Styger
 * \copyright
 * Web:         https://mcuoneclipse.com
 * SourceForge: https://sourceforge.net/projects/mcuoneclipse
 * Git:         https://github.com/ErichStyger/McuOnEclipse_PEx
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without modification,
 * are permitted provided that the following conditions are met:
 *
 * - Redistributions of source code must retain the above copyright notice, this list
 *   of conditions and the following disclaimer.
 *
 * - Redistributions in binary form must reproduce the above copyright notice, this
 *   list of conditions and the following disclaimer in the documentation and/or
 *   other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
 * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
 * ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 * ###################################################################*/

#include <gcov_support.h>
#include <stdint.h>
#include <stdio.h>

void __gcov_flush(void); /* internal gcov function to write data */

int gcov_check(void) {
#if GCOV_DO_COVERAGE
  FILE *file = NULL;

  file = fopen ("c:\\tmp\\test.txt", "w");
  if (file!=NULL) {
    fputs("hello world\r\n", file);
    (void)fwrite("hello\r\n", sizeof("hello\r\n")-1, 1, file);
    fclose(file);
    return 1; /* ok */
  }
  return 0; /* failed */
#else
  return 1; /* ok */
#endif
}

void gcov_write(void) {
#if GCOV_DO_COVERAGE
  __gcov_flush();
#endif
}

/* call the coverage initializers if not done by startup code */
void gcov_init(void) {
#if GCOV_DO_COVERAGE
  void (**p)(void);
  extern uint32_t __init_array_start, __init_array_end; /* linker defined symbols, array of function pointers */
  uint32_t beg = (uint32_t)&__init_array_start;
  uint32_t end = (uint32_t)&__init_array_end;

  while(beg<end) {
    p = (void(**)(void))beg; /* get function pointer */
    (*p)(); /* call constructor */
    beg += sizeof(p); /* next pointer */
  }
#endif /* GCOV_DO_COVERAGE */
}

Enabling Coverage in Application

The following shows the needed modifications in the application to enable coverage:

1. Include the header file:
   #include “gcov_support.h”
2. Initialize coverage information, best right after main():
   gcov_init();
3. Optional: check if file I/O is working properly:
   if (!gcov_check()) {
      /* file I/O is not working */
   }
4. Anytime in the application, dump the data with:
   gcov_write();

Linker: Symbols for gcov_init()

In gcov_init() I have to initialize the data and constructors for the gcov library. Each source file instrumented for coverage needs to be initialized properly. Because the NXP SDK startup code does not initialize these constructors, I have to call gcov_init() from my application. gcov_init() expects two special symbols for the start and end of the constructors, see ‘Coverage Constructors’ in “Code Coverage for Embedded Target with Eclipse, gcc and gcov” for details.

The default linker script files in MCUXpresso IDE does not generate the needed symbols. The easiest way add this is to use the concept of Linker Script Templates (see menu Help > Help Contents > MCUXpresso IDE User Guide > Memory Configuration and Linker Scripts).

Inside the project root folder, create a folder named ‘linkscripts‘:

Inside that folder, create a file named ‘main_text.ldt‘ with the following content:

 /************************************************
 * start of main_text.ldt: *
 ************************************************/
 . = ALIGN(4);
 *(.text*)
 /* added in template for gcov: */
 PROVIDE_HIDDEN (__init_array_start = .);
 KEEP (*(SORT(.init_array.*)))
 KEEP (*(.init_array*))
 PROVIDE_HIDDEN (__init_array_end = .);
 /************************************************ 
 * end of main_text.ldt *
 ************************************************/

This will create the two extra symbols __init_array_start and __init_array_end which are needed by gcov_init().

💡 If not using managed linker scripts, then follow this article how to add it to the linker script file.

Linker: Semihosting

The coverage library will use normal file I/O (fopen(), fwrite(), etc) to write the information to the host using semihosting.

💡 Usually, I don’t recommend to use semihosting, unless you really know what you are doing. In the case of coverage and profiling, it is actually very useful. Semihosting means that using if calling things like printf() or fopen() will trap/halt the target to start communication with the debugger on the host. So it is rather slow, plus if there is no debugger attached, the application on the board might stall! So don’t use semihosting if no active debug session is going on!

In the linker settings, choose the semihosting library:

💡 The newlib (semihost) needs more stack and heap, but is by factors faster than newlib-nano.

Linker: –coverage flag

The tell the linker to link with the necessary coverage libraries, add the following flag to the linker settings:

--coverage

Source Files –coverage flag

Add the following compiler flag to each source file you want to have coverage for:

--coverage

💡 Right-click on a file or folder to set special options for it. See as well “Icon and Label Decorators in Eclipse“.

I recommend only to enable coverage only for the files needed:

• RAM: coverage needs counters in RAM
• Time: writing coverage might need several seconds for each file

Coverage information is combined at the end. I can run coverage for one part of the application and then for the other.

Debugger Settings

Make sure your debug (launch) configuration has semihosting enabled:

For the LinkServer/CMSIS-DAP connection the setting is here:

Build

Now build the application. For every coverage instrumented source file, it will generate *.gcno files:

The files are generated in the same folder as the object files.

Debug

Start a normal debug session. With executing the call to gcov_write() it will write the *.gcda files:

Inspecting Coverage

Double click on any of the coverage files, and it opens a dialog where I have to specify the binary file used:

This opens a gcov view in Eclipse:

With double-clicking on a file/function I get the detail information, including how many times a line has been executed:

Trouble Shooting

As always, things might not work out the first time. Here are a few tips:

1. Make sure that you use the same version of gcov and gcc. Check if you have any other GNU tools in your system path.
2. If the *.gcda files do not show up in the Eclipse Project Explorer view, do a view refresh
3. File I/O and other semihosting functions need a lot of stack. Increase the application and/or task stack size.
4. Writing *.gcda files can be very slow with semihosting, depending on file size. Start instrumenting with just one file. Consider using a different debug interface/probe. For me the Segger J-Link is 3 times faster with semihosting file I/O compared to CMSIS-DAP.
5. Check if gcov_check() can open a file, check the file created. If file I/O fails, verify that you are actually using a semihosting library. Make sure you are not overwriting the file I/O funtions (e.g. do not use a custom _write() function which is part e.g. in the Segger RTT library).
6. Perform a ‘clean’ or delete the output (usually ‘Debug’) folder and do a clean build.
7. Increase the heap size for the application. Semihosting with newlib needs at least 8K of heap memory.

Summary

Using the Eclipse IDE for code coverage with GNU gcov is a great addition. It helps me showing progress with my automated testing, and I can easily see which code pieces are not touched by my testing yet.

Using semihosting with file I/O is not the fastest way to save the coverage information to the host, but is a workable solution. I started on an experimental project to port the gcov library so I can use it to save the data to a memory device on the target system or send it to the host using any other connection method (serial, RTT, USB CDC, …). Let me know if you are interested in such a thing and I’ll see if I could write a tutorial about this.

The example project of this tutorial can be downloaded from GitHub: https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/MCUXpresso/FRDM-K64F/FRDM-K64F_gcov

Links

• Adding GNU coverage tools to Eclipse: https://mcuoneclipse.com/2017/06/18/adding-gnu-coverage-tools-to-eclipse/
• GNU Coverage Tool: https://gcc.gnu.org/onlinedocs/gcc/Gcov.html
• Code Coverage for Embedded Target with Eclipse, gcc and gcov: https://mcuoneclipse.com/2014/12/26/code-coverage-for-embedded-target-with-eclipse-gcc-and-gcov/
• Code Coverage with gcov, launchpad tools and Eclipse Kinetis Design Studio V3.0.0: https://mcuoneclipse.com/2015/05/31/code-coverage-with-gcov-launchpad-tools-and-eclipse-kinetis-design-studio-v3-0-0/
• Tutorial: Using GNU Profiling (gprof) with ARM Cortex-M: https://mcuoneclipse.com/2015/08/23/tutorial-using-gnu-profiling-gprof-with-arm-cortex-m/