Unit Testing choiyj - ICER HPCC

Unit testing with examples in C++ iCER  Workshop  02/19/2015

Yongjun  Choi  choiyj@msu.edu  

Research  Specialist,  Institute  for  Cyber-­‐Enabled  Research  

Download  the  slides:  https://wiki.hpcc.msu.edu/x/_AJiAQ

Agenda

• Test-driven development • Unit testing and frameworks • Unit testing in C++ & Boost.Test

How this workshop works

• We are going to cover some basics. – A few of hands on examples

• Exercises are denoted by the following icon in this presents:

Test-driven development (TDD)

• TDD is a software development process which relies on the repetition of a very short development cycle

1. Add a failing test case that defines a desired improvement or new function

2. Run all tests and see if the new one fails 3. Write some code to pass the test 4. Run tests 5. Refactor code

Test-driven development (TDD)

• Let’s assume you want to develop a software module which calculates – defInt(f(x), a, b)

• You need to know if your new module works correctly.

• So, before you start coding, a test case is prepared. • eg: defInt(f(x), a, b) == 1/2 • Now you develop a module to pass this test

TDD workflow

http://en.wikipedia.org/wiki/Test-driven_development

Software testing

• Many different forms of tests: – unit tests: – integration tests: to test if Individual units are combined and

functioned as a group. – regression tests: to uncover new software bugs in existing

functional – performance tests: to determine how a system performs in

terms of responsiveness and stability under a particular workload.

Unit testing

• Unit testing is a method by which individual units of source code are tested to determine if they are correctly working.

• A unit is the smallest testable part of an application. – an individual function or procedure

Benefits of unit testing

• Facilitate changes: unit tests allow programmers to refactor code at later date, and be sure that code still works correctly;

• Simplify integration: unit tests may reduce uncertainty in the units, and can be used in a bottom-up testing style approach.

• Living documentation for the system: easy to gain a basic understanding of implemented API

How to organize tests

• Each test case should test only one thing • A test case should be short • Test should run fast, so it will possible to run it very

often • Each test should work independent of other test • Tests should NOT be dependent on the order of their

execution

• How should a test program report errors? Displaying an error message would be one possible way.

– requires inspection of the program’s output after each run to determine if an error exists.

– Human inspection of output message is time consuming and unreliable.

• Unit testing frameworks are designed to automate those tasks.

Test methods: man in the loop

if( something_bad_detected ) std::cout << “Something bad has been detected” << std::endl;

Test methods: EXIT_SUCCESS/EXIT_FAILURE

• There are several issues with the test above – You need to convert is_valid result in proper result code – If exception happens in test_object when is_valid

invocation, the program will crash. – You won’t see any output, if you run it manually.

#include <my_class.hpp> int main( int, char* [] ) { my_class test_object( "qwerty" ); return test_object.is_valid() ? EXIT_SUCCESS : EXIT_FAILURE; }

• The Unit Test Framework solves all these issues. To integrate it the above program needs to be changed to:

Test methods: Unit testing frameworks

#include <my_class.hpp> #define BOOST_TEST_MODULE MyTest #include <boost/test/unit_test.hpp>

BOOST_AUTO_TEST_CASE( my_test ) { my_class test_object( "qwerty" ); BOOST_CHECK( test_object.is_valid() ); }

• To simplify development of unit tests, unit test frameworks are usually used.

• Almost any programming language has several unit testing frameworks.

Test methods: Unit testing frameworks

Unit testing frameworks in C++

• There are many unit testing frameworks for C++ – Boost.Test, Google C++ Testing Framework, etc, etc, etc!

• Boost.Test – Suitable for novice and advanced users – Allows organization of test cases into test suites – Test cases can be registered automatically and/or

manually – Fixtures (initialization and cleanup of resources) – Large number of assertion/checkers

• exceptions, equal, not equal, greater, less, floating point numbers comparison etc.

Preparation

• connect to the MSU hpc – open a terminal and do 1. ssh your_user_id@gateway.hpcc.msu.edu 2. ssh dev-intel10 3. module load powertools 4. getexample boostUnitTests 5. cd boostUnitTests – Now you are ready to run boost unit test samples!

simpleUnitTest1.cpp

#define BOOST_TEST_MODULE Simple testcase // name of this test #include <boost/test/unit_test.hpp> //header file for boost.test

// a module that we want to develop int addTwoNumbers(int i, int j ) { return 0; }

//Here is a test BOOST_AUTO_TEST_CASE(addition) { BOOST_CHECK(addTwoNumbers(2, 3) == 5); }

simpleUnitTest1

• Now let’s build and run it • You should be ~/boostUnitTests. Please do

1. mkdir build 2. cd build 3. cmake ../ 4. make 5. ./simpleUnitTest1

• It will give you an error message because we did not implement the module yet.

simpleUnitTest1

• A failed BOOST_CHECK will not exit the test case immediately - the problem is recorded and the case continues. To immediately fail a test, BOOST_REQUIRE is used.

#define BOOST_TEST_MODULE Simple testcases #include <boost/test/unit_test.hpp> int add(int i, int j ) {return 0;}

BOOST_AUTO_TEST_CASE(addition) { BOOST_CHECK(add(2, 3) == 5); }

simpleUnitTest2.cpp

#define BOOST_TEST_MODULE Simple testcase //declares name of this test #include <boost/test/unit_test.hpp> // header file for boost.test

// a module that we want to develop int addTwoNumbers(int i, int j ) { return i - j; // need to be fixed to pass the test }

//Here is a test BOOST_AUTO_TEST_CASE(addition) { BOOST_CHECK(addTwoNumbers(2, 3) == 5); BOOST_CHECK_MESSAGE(addTwoNumbers(2, 3)== 5, "<addTwoNumbers> has some problem!"); BOOST_CHECK_EQUAL(addTwoNumbers(2, 3), 5); BOOST_REQUIRE(addTwoNumbers(2, 3) == 5); BOOST_WARN(addTwoNumbers(2, 3) == 5); //will never reach this check }

simpleUnitTest2

• Run and verify the difference between assertions. • Fix the bug, and test it again.

Assertions

• BOOST_WARN is not checked here, because the test has failed with BOOST_CHECK_REQUIRE

BOOST_AUTO_TEST_CASE(addition) { BOOST_CHECK(addTwoNumbers(2, 3) == 5); BOOST_CHECK_MESSAGE(addTwoNumbers(2, 3)== 5, "<addTwoNumbers> has some problem!"); BOOST_CHECK_EQUAL(addTwoNumbers(2, 3), 5); BOOST_REQUIRE(addTwoNumbers(2, 3) == 5); BOOST_WARN(addTwoNumbers(2, 3) == 5); //will never reach this check }

Testing tools/Checkers

• WARN – produces warning message check failed, but error counter

does not increase and test case continues

• CHECK – reports error and increases error counter when check fails,

but test case continues

• REQUIRE – similar to CHECK, but it is not used to report “fatal” errors.

Testing tools/Checkers

• expression examples – BOOST_WARN( sizeof(int) ==sizeof(short)); – BOOST_CHECK(i == 1); – BOOST_REQUIRE (i > 5); – BOOST_REQUIRE(‘h’, s[0]);

• If the check fails, Boost.Test will report the line in the source code where this occurred and what condition was specified.

• See the example

More Assertions

• CHECK, REQUIRE, and WARN (with “—log_level=all”) • BOOST_CHECK_MESSAGE • BOOST_CHECK_EQUAL

• The full list of available assertions can be found – http://goo.gl/xQczVR

simpleUnitTest3.cpp

• If you have many test cases in one program, then their maintenance could be hard. – test suite is available – In a normal run, you won’t

see that the tests have been categorized.

– With “—log_level=test_suite” or “—log_level=all”, you will.

#define BOOST_TEST_MODULE Suites #include <boost/test/unit_test.hpp> int add(int i, int j) { return i + j;} BOOST_AUTO_TEST_SUITE(Maths) BOOST_AUTO_TEST_CASE(addition) { BOOST_CHECK(add(2, 2) == 4); } BOOST_AUTO_TEST_SUITE_END() BOOST_AUTO_TEST_SUITE(Physics) BOOST_AUTO_TEST_CASE(specialRelativity) { int e = 32; int m = 2; int c = 4; BOOST_CHECK(e == m * c * c); } BOOST_AUTO_TEST_SUITE_END()

Fixtures

• Fixtures are special objects that are used to implement setup and cleanup of data/resources required to execution of unit tests.

• Separation of code between fixtures and actual test code, simplifies the unit test code, and allows the same initialization code to be used for different test cases and test suites.

Fixtures example

and you can use it in the following way:

struct MyFixture { int* i; MyFixture() i = new int; *i = 0;} ~MyFixture() {delete[] i;} };

BOOST_AUTO_TEST_CASE (test_case1) { MyFixture f; // do something with f.i }

Boost.Test’s fixture macro

and you can use it following way:

#define BOOST_TEST_MODULE fixture example #include <boost/test/unit_test.hpp>

struct F { int* i; F(): i(1) {} ~F() {} };

BOOST_FIXTURE_TEST_CASE(simple_test, F) { BOOST_CHECK_EQUAL*i, 1); }

simpleUnitTest4.cpp

• The “Massive” fixture was created, it holds one variable m, and it is directly accessible in our test case.

#define BOOST_TEST_MODULE Fixtures #include <boost/test/unit_test.hpp> struct Massive { int m; Massive() : m(3) { BOOST_TEST_MESSAGE("setup mass"); } ~Massive() { BOOST_TEST_MESSAGE("teardown mass"); } }; BOOST_FIXTURE_TEST_SUITE(Physics, Massive) BOOST_AUTO_TEST_CASE(specialTheory) { int e = 48; int c = 4; BOOST_CHECK(e == m * c * c); } BOOST_AUTO_TEST_CASE(newton2) { int f = 10; int a = 5; BOOST_CHECK(f == m * a); } BOOST_AUTO_TEST_SUITE_END()