Unit Testing & CxxTest

C/C++ Users Journal, December 2005

By Eric Gufford

Presentation by Phil Ulrich

What is unit testing?

A unit test is a procedure used to verify that a particular module of code is working properly.

The idea about unit tests is to write test cases for all functions and methods so that whenever a change causes a problem, it can be identified and fixed quickly. Ideally each test is separate from the others.

Unit testing in C++ is still not widely understood or used, however, despite its usefulness.

Why isn’t unit testing widely used? There are many explanations, all of which boil

down to one of three reasons.– It is hard. Many classes aren’t easy to adapt to unit

testing. It’s easy to say “my code is too complicated to unit test” and leave it at that.

– It is time-consuming. Unit test frameworks can be difficult to learn and set up, thus using valuable time that could be used for other tasks.

– It is complicated. There are many different frameworks on the market (some open source, some commercial), and all of them are large, unwieldy, and cannot easily be used in an IDE.

What is CxxTest?

While the first excuse to not use unit testing can be countered, the second and third are generally valid for most C++ unit testing frameworks.

CxxTest aims to counter those two excuses, too.

CxxTest is a framework that uses Perl to construct robust testing classes.

How does it work?

You include a header file (TestSuite.h) and declare a class that inherits from CxxTest::TestSuite, the base class of all unit tests.– In the child class, you insert all the various tests the

class is meant to perform.

– From there, a Perl (or Python) script is run that actually writes the class that will perform the tests.

– Thus, the script does all the heavy lifting, keeping the developer from wasting hours of time trying to understand the arcane mechanics of the framework.

How does it work? (cont.)

Once the testing class is declared, each test method can be defined. The methods must start with a lower case ‘test’ in order for the Perl script to see it and have no return type or any parameters.– i.e. void testMain(void) to test main().

There are no real restrictions on the testing functions; they can perform any actions that make sense – instantiating classes, interacting with the file system, etc.

Does CxxTest support logging? CxxTest supports a number of macros to record test

status, called TS_FAIL, TS_WARNING, and TS_TRACE.

Passing a test does not have a status – if the test function ends with no error, the test was passed.

These macros print to stdout as well as providing an unambiguous statement of test status.– TS_FAIL fails the test and stops execution of the testing

function.– TS_WARNING does not end execution of the testing

function, but it does mark the test as not succeeding.– TS_TRACE simply writes to stdout without affecting the test

status.

Does CxxTest support logging? (cont.) There are also a number of assertions that can be used for

quality assurance purposes. These include equality operators (TS_ASSERT_EQUALS,

TS_ASSERT_LESS_THAN, TS_ASSERT_GREATER_THAN), a variation for data equality (TS_ASSERT_SAME_DATA), and even a variation that tests for exceptions (TS_ASSERT_THROWS_ANYTHING, TS_ASSERT_THROWS_NOTHING, or TS_ASSERT_THROWS_EQUALS).

All macros have a variant beginning with TSM_ that will print a user-defined message, and ETS_/ETSM_ variants that throw exceptions.

What are test fixtures?

Test fixtures are objects that are constructed at the start of a test and are destroyed at the end of the test.

CxxTest supports test fixtures. By defining fixtures in one place, one

insures that each test will be using objects constructed in the same way – thus, the code to set up the test only has to be tested once itself.

What are text fixtures? (cont.) CxxTest supports the concept of suite-level

fixtures, which are objects created at the beginning of a suite of tests and destroyed at the end of them.

This is useful when an object is complicated or time-consuming to create, or support classes are required that can be shared across a test suite.

Static functions called createSuite() and destroySuite() are called when the suite is created and disposed of to create/destroy these suite-level fixtures.

How does CxxTest output data? The method of output is specified on the command line of the Perl

script that generates the class. The default mechanism is designed for use in an IDE and is controlled

via the --runner option.– This can use regular formatting like stdout, print line numbers like

Visual Studio expects, or simply provide true/false answers.– This can be combined with a switch to describe the type of user

interface you would like in the testing application – options are Win32, X11, or Qt. The behavior is similar in all three windowing environments.

Once the testing app is built, the executable can be run on demand. On Windows, the executable shows its progress as a progress-bar-like mechanism that shows failures as red and successes as green. At the end, the overall result is shown on the bar – red for failures, yellow for warnings, green for all successes.

How does CxxTest tie into a normal build? Ideally, unit tests should be incorporated into a

makefile or a build process so they are ran when the program is compiled.

CxxTest comes with makefiles for a variety of platforms and compilers, such as Linux/gcc, UNIX/gcc, and Windows/Visual Studio.

Under the Windows configuration, the testing application is incorporated into the build process such that if any tests fail, the build fails; if there are any warnings, VS reports a warning. Setting the system to fail on a warning will perform as expected.

Conclusion

In the past, unit testing was difficult, but CxxTest helps easily create a test harness to enable unit testing without spending hours setting up and learning to use a complex unit testing framework.

More info about (and downloads of) CxxTest can be found online at:– http://cxxtest.sourceforge.net/