Unit testing is a common practice for host development. But for embedded development this still seems mostly a ‘blank’ area. Mostly because embedded engineers are not used to unit testing, or because the usual framework for unit testing requires too many resources on an embedded target?
What I have used is the μCUnit framework which is a small and easy to use framework, targeting small microcontroller applications.
The framework is very simple: two header files and a .c file:
Use the original ones from the uCUnit GitHub site or use the ones I have slightly tuned and modified from GitHub to be used with the MCUXpresso SDK and IDE.
The concept is that a unit test includes the uCunit.h header file which provides test macros.
A #define in the header file configures the output as verbose or normal:
System.c and System.h is the connection to the system, basically used for startup, shutdown and printing the test results to a console. Below an implementation using a printf() method to write the output, but this could be replaced by any writing routine or extended to log text on an SD card.
/* Stub: Transmit a string to the host/debugger/simulator */ void System_WriteString(char * msg) { PRINTF(msg); } void System_WriteInt(int n) { PRINTF("%d", n); }Framework Overview
First I have to include the unit test framework header file:
#include "uCUnit.h"First I have to initialize the framework with
UCUNIT_Init(); /* initialize framework */One more test cases are wrapped with a UCUNIT_TestcaseBegin() and UCUNIT_TestcaseEnd():
UCUNIT_TestcaseBegin("Crazy Scientist"); /* test cases ... */ UCUNIT_TestcaseEnd();To write a summary at the end use
UCUNIT_WriteSummary();and if the system shall be shut down use a
UCUNIT_Shutdown();Tests
The framework provides multiple testing methods, such as:
UCUNIT_CheckIsEqual(x, 0); /* check if x == 0 */ UCUNIT_CheckIsInRange(x, 0, 10); /* check 0 <= x <= 10 */ UCUNIT_CheckIsBitSet(x, 7); /* check if bit 7 set */ UCUNIT_CheckIsBitClear(x, 7); /* check if bit 7 cleared */ UCUNIT_CheckIs8Bit(x); /* check if not larger then 8 bit */ UCUNIT_CheckIs16Bit(x); /* check if not larger then 16 bit */ UCUNIT_CheckIs32Bit(x); /* check if not larger then 32 bit */ UCUNIT_CheckIsNull(p); /* check if p == NULL */ UCUNIT_CheckIsNotNull(s); /* check if p != NULL */ UCUNIT_Check((*s)==’\0’, "Missing termination", "s"); /* generic check: condition, msg, args */This is explained best with a few examples.
Example: Crazy Scientist
Below is a ‘crazyScientist’ function which combines different materials:
typedef enum { Unknown, /* first, generic item */ Hydrogen, /* H */ Helium, /* He */ Oxygen, /* O */ Oxygen2, /* O2 */ Water, /* H2O */ ChemLast /* last, sentinel */ } Chem_t; Chem_t crazyScientist(Chem_t a, Chem_t b) { if (a==Oxygen && b==Oxygen) { return Oxygen2; } if (a==Hydrogen && b==Oxygen2) { return Water; } return Unknown; }A test for this could look like this:
void Test(void) { Chem_t res; UCUNIT_Init(); /* initialize framework */ UCUNIT_TestcaseBegin("Crazy Scientist"); res = crazyScientist(Oxygen, Oxygen); UCUNIT_CheckIsEqual(res, Oxygen2); UCUNIT_CheckIsEqual(Unknown, crazyScientist(Water, Helium)); UCUNIT_CheckIsEqual(Water, crazyScientist(Hydrogen, Oxygen2)); UCUNIT_CheckIsEqual(Water, crazyScientist(Oxygen2, Hydrogen)); UCUNIT_CheckIsInRange(crazyScientist(Unknown, Unknown), Unknown, ChemLast); UCUNIT_TestcaseEnd(); /* finish all the tests */ UCUNIT_WriteSummary(); UCUNIT_Shutdown(); }With the different checks we can verify if the function is doing what we expect. It produces the following as output:
====================================== Crazy Scientist ====================================== ../source/Application.c:60: passed:IsEqual(res,Oxygen2) ../source/Application.c:61: passed:IsEqual(Unknown,crazyScientist(Water, Helium)) ../source/Application.c:62: passed:IsEqual(Water,crazyScientist(Hydrogen, Oxygen2)) ../source/Application.c:63: failed:IsEqual(Water,crazyScientist(Oxygen2, Hydrogen)) ../source/Application.c:64: passed:IsInRange(crazyScientist(Unknown, Unknown),Unknown,ChemLast) ====================================== ../source/Application.c:65: failed:EndTestcase() ====================================== ************************************** Testcases: failed: 1 passed: 0 Checks: failed: 1 passed: 4 ************************************** System shutdown.💡 I recommend to write the unit tests *before* doing the implementation, because this way it let me consider all the different corner cases and refines the requirements.
The above output is with UCUNIT_MODE_VERBOSE set. Using UCUNIT_MODE_NORMAL it uses a more compact format and prints the failed tests only:
====================================== Crazy Scientist ====================================== ../source/Application.c:63: failed:IsEqual(Water,crazyScientist(Oxygen2, Hydrogen)) ====================================== ../source/Application.c:65: failed:EndTestcase() ====================================== ************************************** Testcases: failed: 1 passed: 0 Checks: failed: 1 passed: 4 ************************************** System shutdown.Trace Points
In the above example we were only testing from the outside what the function does.
How to check that in the following function to be tested it indeed checks for the division by zero case?int checkedDivide(int a, int b) { if (b==0) { PRINTF("division by zero is not defined!\n"); return 0; } return a/b; }To check if that if() condition has been really entered I can add a trace point. The number of tracepoints are configured in μCUnit.h with:
/** * Max. number of checkpoints. This may depend on your application * or limited by your RAM. */ #define UCUNIT_MAX_TRACEPOINTS 16With
UCUNIT_ResetTracepointCoverage();I can reset the trace points.
I mark the execution of a trace point with an id (which is in the range 0..UCUNIT_MAX_TRACEPOINTS-1)
UCUNIT_Tracepoint(id);With
UCUNIT_CheckTracepointCoverage(0);I can check if a given trace point has been touched.
Below the function to be tested instrumented with a trace point:int checkedDivide(int a, int b) { if (b==0) { UCUNIT_Tracepoint(0); /* mark trace point */ PRINTF("division by zero is not defined!\n"); return 0; } return a/b; }The corresponding unit test code:
UCUNIT_TestcaseBegin("Checked Divide"); UCUNIT_CheckIsEqual(100/5, checkedDivide(100,5)); UCUNIT_ResetTracepointCoverage(); /* start tracking */ UCUNIT_CheckIsEqual(0, checkedDivide(1024,0)); UCUNIT_CheckTracepointCoverage(0); /* check coverage of point 0 */ UCUNIT_TestcaseEnd();Which then produces:
====================================== Checked Divide ====================================== ../source/Application.c:69: passed:IsEqual(100/5,checkedDivide(100,5)) division by zero is not defined! ../source/Application.c:71: passed:IsEqual(0,checkedDivide(1024,0)) ../source/Application.c:72: passed:TracepointCoverage(1)String Test
There are many other ways to use checks, up to have user configured checks and messages. Below is an example of a function to test:
char *endOfString(char *str) { if (str==NULL) { return NULL; } while(*str!='\0') { str++; } return str; }with the following test code:
UCUNIT_TestcaseBegin("Strings"); UCUNIT_CheckIsNull(endOfString(NULL)); str = endOfString("abc"); UCUNIT_Check( (str!=NULL), /* condition to check */ "string shall be not NULL", /* message */ "str" /* argument as string */ ); UCUNIT_CheckIsEqual('\0', *endOfString("")); UCUNIT_CheckIsEqual('\0', *endOfString("hello")); str = endOfString("world"); UCUNIT_CheckIsNotNull(str); UCUNIT_CheckIsEqual('\0', *str); UCUNIT_TestcaseEnd();which produces:
====================================== Strings ====================================== ../source/Application.c:76: passed:IsNull(endOfString(NULL)) ../source/Application.c:82: passed:string shall be not NULL(str) ../source/Application.c:83: passed:IsEqual('\0',*endOfString("")) ../source/Application.c:84: passed:IsEqual('\0',*endOfString("hello")) ../source/Application.c:86: passed:IsNotNull(str) ../source/Application.c:87: passed:IsEqual('\0',*str)Summary
μCUnit is a very simple but yet powerful unit testing framework for embedded devices and microcontroller. It is easy to use and requires only minimal resources and helps increasing the quality of embedded software with automated unit tests. I hope you find it useful too.
Happy Testing 🙂
Links
- μCUnit web page: https://www.mikrocontroller.net/articles/Unittests_mit_uCUnit
- μCUnit Github site: https://github.com/ucunit/ucunit
- μCUnit example usage: https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/MCUXpresso/FRDM-K64F/FRDM-K64F_uCUnit
- Port of μCUnit for MCUXpresso: https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/MCUXpresso/FRDM-K64F/FRDM-K64F_uCUnit/uCUnit
Thank you for this blog entry. I my company, we are currently trying to establish a more formal testing process for our firmware products. I’ll check the framework out next week 🙂
It’s a pity that software testing in embedded systems is still uncommon, even in big companies…
LikeLike
Yes, I think because it is mostly unknown to embedded developers that these kind of things actually do exist. And it is really not hard to use a framework as uCUnit, and it only needs minimal resources on the target.
LikeLike
Check out the book “Test-Driven Development for Embedded C” by James W. Grenning
“Still chasing bugs and watching your code deteriorate? Think TDD is only for desktop or web apps? It’s not: TDD is for you, the embedded C programmer. TDD helps you prevent defects and build software with a long useful life. This is the first book to teach the hows and whys of TDD for C programmers. …”
https://pragprog.com/book/jgade/test-driven-development-for-embedded-c
LikeLiked by 1 person
Hi Bob,
thanks for that book recommendation! Sounds like a really good one too, and it is available for a reasonable price too, and available as e-book too (which I’m going to order).
LikeLike
Thanks for the shout out on my book! If you want to share ideas, ping me on twitter or my website. I’ve got numerous articles to share. http://blog.wingman-sw.com and https://wingman-sw.com/papers/titles
LikeLike
Thank you for that great book!
LikeLike
My commercial projects include integrated testing, often starting on host, then migrating to embedded target. For example matrix math and numerical routines are verified on host and rechecked on embedded target (which may have lower precision math). C++ classes typically have a public member function like “static bool QwikTest();” with a (perhaps debug-build-only) command to run all qwiktests, which does require some space for test code and datasets in flash. Some details here from a class I’ve taught:
Click to access ESC-111paper_Nadler_corrected.pdf
Hope that’s helpful!
Best Regards, Dave
LikeLike
Thanks for that paper, and as always it is useful stuff from you! And I love that picture at the end of the paper 🙂
LikeLike
PS: I also recommend James Grennings’ TDD book.
LikeLike
Reading it right now, and already put it on https://mcuoneclipse.com/books/ 🙂
LikeLike
Thanks and it looks easy to use uCUnit framework for unit testing.
LikeLike
I also recommend you to read James Grenning’s book about TDD for embedded systems. Also note that if you’re really interested into this topic, he gives courses on this theme through the Barr Group.
At work we use another unit test framework : Unity (huge set of assertions), CMock (generate mocks for files) and Ceedling (manages the configuration, build, results, etc). The three are done by the same team (ThrowTheSwitch) and can be found on Github. The idea is to run the tests on a PC, not directly on the target. It’s really useful, although it takes some work to set it up the first time if you’ve never exercised compiling your code for a PC and if you have many dependencies. This framework is also targeting embedded systems by the way, so you will find many advice on their site & blogs regarding mocking the low-level parts of your embedded code.
LikeLiked by 1 person
This is the same config we use at work – all done through ceedling.
qemu is used to run the unit tests – means we can use the same arm target compiler. Never found the need to perform actual on embedded target testing.
LikeLike
qemu did not help in my case because I really needed the hardware portion of the drives tested too. Yes, I can create mocks for that, but ultimately I have to run it on hardare. Qemu as more or less an instruction similator did not help me. But this all depends how close to the hardware the testing needs to be.
LikeLike
Implementing the UCUNIT_WriteString(msg) Function ( and all other Host depending functions ) with Segger RTT would be nice.
LikeLike
Hi Uwe,
yes, I’m using it with SEGGER RTT as well. Instead that PRINTF macro I’m calling SEGGER_printf() instead.
LikeLike
+1 For James Grenning’s excellent book but it was still a trial to apply it for embedded apps.
Another great resource is the kit from Matt Chernosky who has taken the time to make the whole unit test thing a bit more doable on GCC and within Eclipse with some startup config and detailed documentation.
http://www.electronvector.com/unit-testing-in-eclipse/
LikeLiked by 1 person
Hi Don,
thanks for that link! That sounds interesting, but have you used that plugin as well with running tests on an embedded target?
LikeLike
Hey, that’s my kit! Thanks for recommending it 🙂
It’s not for on-target testing though, just for running host-based tests (compiled with GCC and run with Ceedling).
If you don’t necessarily need IDE integration right away, this article (http://www.electronvector.com/blog/add-unit-tests-to-your-current-project-with-ceedling) works through an example of setting up Ceedling to work from the command line with an existing project.
LikeLike
Hey, it is a great kit! I have not actively used it. I know there are pros and cons with running things on the hardware or on the device. And I agree that it can make sense to run the ‘business logic’ on the host. But most of my applications really depend on the hardware and need to test the functionality of the device itself. Any thoughts on this, or recommendation how to run your framework on the device itself?
LikeLiked by 1 person
Hey Erich,
Ceedling (not *mine* but I’m a big fan) is built from the Unity unit test framework and the CMock mocking framework. You can compile and run both of these on your target to execute your tests there, although I haven’t done it personally.
It’ll require some configuration — e.g. Unity needs to know how to “print” test results to a serial port or whatnot. Also CMock uses some memory and could be difficult to fit in on something small.
Ceedling provides the out-of-the-box automation you need to build and run tests on the host, but you can configure it to use any compiler you want, including a cross-compiler. I suspect you could configure it to use command line tools to download to the target, but I’m not exactly sure where those hook would be as I’ve never tried it.
My primary strategy (as recommended by Grenning) has been to build a HAL to decouple the code from the hardware and run tests on the host.
LikeLike
Thanks for the clarification! I agree on the general strategy to decouple from the hardware. It is only that most of projects really heavily depend on the hardware and proper funtionality with the hardware is the primary goal. So I rather save the work to have a HAL if I have the hardware anyway.
LikeLike
Thanks for your contents. What do you think about Cmocka? (https://cmocka.org/)
I’m also looking for a test framework for embedded projects.
LikeLike
I have not used cmocka, but from the description it looks very useful. I was more about to look into CMock.
LikeLike
Also check out throwtheswitch.org for resources on unit testing embedded systems.
LikeLike