Debugger Shell: Test Automation

The development cycle does not end with debugging. Debugging is something manual, but for testing and automation I want to develop scripts I can run in an automated fashion. For this I use a tool in CodeWarrior: the Debugger Shell as command line debugger and using TCL as scripting language. This gives me a powerful way into automation and scripting with the debugger: from basic access to memory, to stepping and controlling the execution up to programming the flash memory.

Testing Application

I’m running my application on the Freedom-KL25Z board which is using the RGB LED on that board. Now I want to write a small automated script which tests if the LED is working properly. So this is my simple test program ‘Test_LED.c’ for one of the LEDs:

#include <stdint.h>
#include "Test_LED.h"
#include "LED1.h"

static volatile uint32_t val;
static volatile uint32_t reg;

void TestLED(void) {
  reg=(GPIO1_GetPortValue(NULL)&GPIO1_LED_R_MASK); /* get raw register value */
  val = LED1_Get(); /* LED shall be off, so val should be zero */
  LED1_On();
  val = LED1_Get(); /* LED shall be on, so val should be one */
  LED1_Off();
  val = LED1_Get(); /* LED shall be off, so val should be zero */
}

Test Automation with the Debugger Shell

The Debugger shell is using TCL as scripting language (see Scripting: The Debugger Shell, Getting started…). And instead typing in commands, I can save them into a file and execute it with the source command:

source myScript.tcl

That gives a lot of flexibility. I’m going to

  • start debugging session
  • set breakpoints at specific locations
  • run my program
  • comparing variables with expected values
  • print the number of errors
  • exit debugging session

Before putting everything into a TCL script file, it makes sense to execute one command after each other from the Debugger Shell view.

The first thing is to start the debugger:

debug

Next I set a breakpoint in my test program. I’m using an ‘automatic’ break point (see Software and Hardware Breakpoints) on line 9, column 1:

bp -auto Test_LED.c 9 1

Then let’s run the application and hit that breakpoint:

# run program
go

The debugger stops on line 9:

#include <stdint.h>
#include "Test_LED.h"
#include "LED1.h"

static volatile uint32_t val;
static volatile uint32_t reg;

void TestLED(void) {
  reg = (GPIO1_GetPortValue(NULL)&GPIO1_LED_R_MASK); /* get raw register value */
  val = LED1_Get(); /* LED shall be off, so val should be zero */
  LED1_On();
  val = LED1_Get(); /* LED shall be on, so val should be one */
  LED1_Off();
  val = LED1_Get(); /* LED shall be off, so val should be zero */
}

Next, I’m going to step over that line 9:

step over

Now we I stay on line 10:

#include <stdint.h>
#include "Test_LED.h"
#include "LED1.h"

static volatile uint32_t val;
static volatile uint32_t reg;

void TestLED(void) {
  reg = (GPIO1_GetPortValue(NULL)&GPIO1_LED_R_MASK); /* get raw register value */
  val = LED1_Get(); /* LED shall be off, so val should be zero */
  LED1_On();
  val = LED1_Get(); /* LED shall be on, so val should be one */
  LED1_Off();
  val = LED1_Get(); /* LED shall be off, so val should be zero */
}

To count the number of errors in my test script, I use an error counter:

set test_nofErrors 0

Time to check the value of ‘reg’. For this I set up a scripting variable ‘test_expected’ and initialize it with a value of 0x40000 (the expected LED port value):

set test_expected 0x40000

In a similar way I set up a variable to read the ‘reg’ variable of my application and store it into ‘test_val’:

set test_val [evaluate reg]

A one-liner compares the actual value with the expected value. If it does not match, it increases the error counter. To reference the TCL variables I use the $ prefix:

if {$test_expected != $test_val} {set test_nofErrors [expr {$test_nofErrors + 1}]}

To clear all breakpoints set I use this:

bp all off

To print the number of errors:

puts "Number of errors: $test_nofErrors"

And finally exit the debugging session:

kill

Additionally I can use the wait command to suspend things as I want to follow the flow while the program runs. To wait for 1 second I use this:

wait 1000

Now time to put everything together into a script file (myScript.tcl):

# start debug session usin current active debug configuration
debug;

# wait for 1 second
wait 1000;

# set auto breakpoint in Test_Led.c at line 9, column 1
bp -auto Test_LED.c 9 1;

# resume execution, run to breakpoint
go;

# wait for the breakpoint to get hit for 1 second
wait 1000;

# remove all breakpoints
bp all off;

# step over line of code
step over;

# define error counter variable
set test_nofErrors 0;

#########################################################
# TEST: register value shall be 0x40000
#########################################################
# define test expected value variable to the expected value
set test_expected 0x40000;

# evaluate 'reg' variable and define a variable with it
set test_val [evaluate reg];

#compare the expected value with the actual value
if {$test_expected != $test_val} {set test_nofErrors [expr {$test_nofErrors + 1}]};

#########################################################
# TEST: LED shall be off
#########################################################
step over;
set test_expected 0
set test_val [evaluate val];
if {$test_expected != $test_val} {set test_nofErrors [expr {$test_nofErrors + 1}]};

#########################################################
# TEST: LED shall be on
#########################################################
step over;
step over;
set test_expected 1
set test_val [evaluate val];
if {$test_expected != $test_val} {set test_nofErrors [expr {$test_nofErrors + 1}]};

#########################################################
# print number of errors
puts "*** Number of errors: $test_nofErrors ***";

# terminate the debug session
kill;

And this is how it runs inside the Debugger Shell:

Debugger Shell with Test Script Running

Debugger Shell with Test Script Running

Breakpoints on Functions

There is as always room for improvements. A thing is to set breakpoints on functions instead using file names and line numbers. This is done with this:

# set breakpoint on function foo
bp -auto foo

Summary

It takes some time to get used to script automate things. But if done well, it saves a lot of time at the end. I’m still working through many details and exploring more functionality, but I think it is a good start.

Happy Automating 🙂

Advertisements

9 thoughts on “Debugger Shell: Test Automation

  1. Erich,
    I think that’s amazing! It will be very handy for me as I often have very elaborate debugging setups that take lots of temporary breakpoints, etc. to get to the place I want to actually exercise.

    Was this stuff in the old Classic CodeWarrior 6.x, too?
    Bill

    Like

    • Bill, yes, a subset of this was already available in classic. But it was different depending if you were using S08 or ColdFire. Now in Eclipse based CodeWarrior things are unified and extended.
      Erich

      Like

  2. Pingback: Standalone Flash Programmer | MCU on Eclipse

  3. Pingback: CodeWarrior Flash Programming from a DOS Shell | MCU on Eclipse

  4. Pingback: Adding Symbols to the CodeWarrior Debugger | MCU on Eclipse

  5. Pingback: Programming Kinetis with CodeWarrior from the DOS Shell | MCU on Eclipse

What do you think?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s