S Ramakrishnan 1

Systems V & V, Quality and Standards

Dr Sita RamakrishnanDr Sita Ramakrishnan

School CSSESchool CSSE

Monash University Monash University

S Ramakrishnan 2

JUnit

Eric Gamma and Kent Beck created a simple & Eric Gamma and Kent Beck created a simple & effective unit testing framework for Java called effective unit testing framework for Java called

Junit in 1997.Junit in 1997. part of XP (Extreme Programming method) part of XP (Extreme Programming method)

but can be used independently.but can be used independently.Junit (Framework) provides a reusable structure Junit (Framework) provides a reusable structure

which can be shared between apps. which can be shared between apps. Developers can incorporate this framework into Developers can incorporate this framework into

their apps. and extend it to meet their specifictheir apps. and extend it to meet their specific

requirements.requirements.

S Ramakrishnan 3

JUnit

Junit (junit.org) is an open source software Junit (junit.org) is an open source software and is hosted on SourceForge.and is hosted on SourceForge.

defacto standard framework for developing defacto standard framework for developing unit tests in Java.unit tests in Java.

many unit testing frameworks available: many unit testing frameworks available: Eiffel, PhP, Perl, ASP, C++, C# and …Eiffel, PhP, Perl, ASP, C++, C# and …

S Ramakrishnan 4

JUnit

A typical unit test: CheckA typical unit test: Check

that a method accepts input in an expected range & returns the expected result value for each test input.

test the behaviour of a method through its interface

. Test for expected values, also for exception

check if the method obeys the design by contract principles in terms of API contract.

S Ramakrishnan 5

Unit testing frameworks

A simple Calculator class: The core method, add: takes 2 doubles & returns the sum as a doublepublic class Calculator

{

public double add(double num1, double num2)

{

return num1 + num2;

}

}

Put in some work to test this simple “unit of work” will do the adding 2 doubles & returning the sum as a double correctly. And, also preserve this effort so that you can replicate this test and run the tests again later! Objective test cases turned into reusable test cases -> Test Program

S Ramakrishnan 6

Unit testing frameworks

A simple TestCalculator program – manual testpublic class TestCalculator

{

public static void main(String[ ] args)

{

Calculator calculator = new Calculator ( );

double result = calculator.add(10, 20);

if (result != 30)

{

System.out.println(“Incorrect result: “ + result);

}

}

}

The test will pass when you compile & run this test program. If you change this code deliberately to make it fail, you are not testing the code! You should handle error conditions in Java by throwing exceptions.

S Ramakrishnan 7

Unit testing frameworks A simple TestCalculator program - Modifiedpublic class TestCalculator

{

private int errCount = 0;

public void testAdd ( ); // moving the test into its own method

{

Calculator calculator = new Calculator ( );

double result = calculator.add(10, 20);

if (result != 30)

{

throw new RuntimeException(“Incorrect result: “ + result);

}

}

public static void main (String [ ] args)

{

TestCalculator test = new TestCalculator ( );

try

{

test.testAdd ( ); // can add more methods like testAdd with more unit tests in the test calculator & invoke them here

}

catch (Throwable e)

{

test.errCount++;

e.printStackTrace ( );

}

if (test.errCount > 0)

{

throw new RuntimeException(“Number of Errors : “ + test.errCount “ );

}

} }

S Ramakrishnan 8

Unit testing frameworks

• Lessons from the last simple example re: unit testing frameworks:

• Each unit test must run independently of all other unit tests• must be easy to define which unit tests will run• errors must be detected and reported for each test separately.• The modified vers.(last slide) can be improved. Each unit test must be independent & run in a different classloader instance.

S Ramakrishnan 9

Unit testing frameworks

• A better option is to consider a unit test suite.• problem with this approach:

• large try/catch block can be a maintenance headache

• Another approach is to use Java’s reflection & introspection features. A program looks at itself & decide to run methods which are named in a certain way, eg. those begin with letters, test.

• The Junit framework supports introspecting methods, supports the use of a different classloader instance for each test and reports errors for each test separately.

S Ramakrishnan 10

Testing with JUnit• Create a test script with a number of small java methods

• the idea is to create java objects, do something with these and check if the objects exhibit correct properties

• Assertions• methods to check properties such as identity of objects, equality of variables• can check if objects are null or non-null; equality of objects (via == or equals () depending on the type)• use assertions to determine the verdict of the test case.

• In XP methodology, a JUnit test should be written first before any code, and executed.

• the implementation code should be written to get the test to pass• re-execute the test with this code and it should pass.

S Ramakrishnan 11

Junit TestCase(Usually, a test case is considered to be a single test that can pass or fail, and a testsuite is a bunch of related testcases).

In Junit, run multiple test cases withTestSuite & Testcase. They are implemented as classes. No difference between running a testsuite & a testcase. A TestCase is a class and a single test is a method.

TestCalculator program written with Junitimport junit.framework.TestCase; // Junit needs it to automatically run the tests

public class TestCalculator extends TestCase;

{

public void testAdd ( ); // method name follows the pattern testXXX

{

Calculator calculator = new Calculator ( ); // start the test by creating an instance (object under test)

double result = calculator.add(10, 20); // execute the test by calling the method to test, passing it known values

assetEquals ( 30, result , 0); // check the result by calling assertEquals method, inherited from base class, TestCase.

}

}• Javadoc for assertEquals method/***Asserts that two doubles are equal concerning a delta. …*/Static public void assertEquals (double expected, double actual, double delta)

In the above code, assertEquals is passed: expected = 30, actual = result, delta = 0

Often, delta parameter can be zero and can be ignored. It is used when floating point calc. are done. Delta provides a + / - factor. If the actual data is within the range of expected +/- delta, the test passes.

S Ramakrishnan 12

Junit TestCase

public void run(TestResult result) // run method in TestCase Class{ result.startTest (this); setUp ( ); // set up for all tests in class try { runTest ( ); // run the test } catch (AssertionFailedError e) { result.addFailure (this, e); } catch (Throwable e) { result.addError (this, e); } finally { tearDown ( ); // cleanup for all tests in class }}

S Ramakrishnan 13

The Junit Framework

TestResult

Test

TestCase TestSuite

yourTestClass-1 yourTestClass-n

run(TestResult)

run(TestResult)runTest ( )setup ( )tearDown ( )

run(TestResult)addTest(Test)

fName

fTests

*suite ( ): TestSuite *suite ( ): TestSuite

S Ramakrishnan 14

Junit workings

• Junit uses reflection to build the testsuite dynamically• use getClass ( ) on an object to know which class it belongs to• use getMethods ( ) to find out the methods in a class• use invoke ( ) on each method to run it• This ensures that test suite need not be updated if a method is added or deleted from the test suite.

• Test class is re loaded each time the tests are run• this means that JUnit test execution window need not be restarted if test cases are recompiled

• To run a Test object - either a TestCase or a TestSuite, simply invoke the run ( ) method.

S Ramakrishnan 15

More info on JUnit available at

• Junit web site: http://junit.org• JUnit Cookbook: Kent Beck and Erich Gamma http://junit.sourceforge.net/doc/cookbook/cookbook.htm• JUnit Cook tour: http://junit.sourceforge.net/doc/cookstour/cookstour.htm• Mocks are’nt stubs - Martin Fowler (jul 2004), http://www.martinfowler.com/articles/mocksArentStubs.html• Inversion of control containers and the dependency injection pattern, M Fowler(jan 2004), http://www.martinfowler.com/articles/injection.html

Vincent Massol with Ted Husted (2004), JUnit in Action, Mannings Publ.J B Rainsberger (2005), JUnit Recipes, Mannings Publ.E Hatcher and S Loughran (2002), Java Development with Ant, Mannings Publ.(Refer to Ch.4 Testing with Ant and Appendix E, Ant Task Reference available Online)

S Ramakrishnan 16

Core JUnit Classes

• TestCase + TestSuite + BaseTestRunnerTestCase + TestSuite + BaseTestRunner = TestResult - JUnit trioJUnit trio - backbone of JUnit Framework – to create test results

• To create a single test case, we extended the TestCase class (slide11)

• To run several TestCase objects at once, create another object called a TestSuite.

• TestRunner is an user interface for launching test suites.

• BaseTestRunner is the superclass for all testrunners.

S Ramakrishnan 17

Core JUnit Classes

• JUnit distribution provides 3 TestRunner classes that can be used to execute your tests:

one for text console, one for AWT and one for Swing test runner. With the Swing TestRunner, the progress indicator

across the screen is the JUnit green to indicate pass and the red bar to indicate a failing test.

• compile the simple Testcalculator program & use the swing TestRunner for running a testcase. When JUnit is started by typing java junit.swinggui.TestRunner TestCalculatorjava junit.swinggui.TestRunner TestCalculator the JUnit framework does 3 things: creates a TestSuite, creates a TestResult and executes the test methods (testAdd in slide 11).• Ok, how to run multiple test cases? -> TestSuite! •TestRunner launches (runs) the TestSuite. Which test cases to run is

up to the TestSuite.

S Ramakrishnan 18

JUnit TestSuites

• For the simple Testcalculator class with a testcase (slide11), a default TestSuite is:

public static Test suite ( ) { return new TestSuite(TestClaculator.class); }

which is equivalent to: public static Test suite ( ) { TestSuite suite = new TestSuite ( ); suite.addTest (new TestCalculator (“testAdd”)); return suite;

}

S Ramakrishnan 19

JUnit TestSuites

• If one adds another test case, testSubtract, the default TestSuite would automatically include it like so:public static Test suite ( ) { TestSuite suite = new TestSuite ( ); suite.addTest (new TestCalculator (“testAdd”)); suite.addTest (new TestCalculator (“testSubtract”)); return suite; }• Framework can generate a default run-time testsuite for you.• You must provide the test case class• Provision of such automatic testsuite ensures that you don’t forget to add additional tests to the test suite.• Default TestSuite is enough for simple code & simple testing.• What if you need something more than what the default can do?

• may want to combine suites from different packages into one• may want to run only a subset of the tests when adding new features & testing for those

S Ramakrishnan 20

JUnit TestSuites

• TestCase and TestSuite implement the Test Interface.package junit.framework;public interface Test { public abstract int countTestCases ( ); public abstract void run (TestResult result);}•the ability to add test suites & test cases to a suite makes it easier to create specific suites as well as a master TestAll class for your apps.

• TestAll class is a static suite method that registers which Test Objects (TestCase or TestSuite objects) your apps. should be running on a regular basis.

S Ramakrishnan 21

A TestAll class

import junit.framework.Test;import junit.framework.TestSuite;Import junitbook.sampling.TestDefaultController;public class TestAll { public static Test suite ( ) // create a suite method to call your other tests or suites { TestSuite suite = new TestSuite(“All tests from Part 1”); // give a name to testsuite to identify // it later suite.addTestSuite(TestCalculator.class); // addTestSuite to TestCase or TestSuite objects // that you want to run together suite.addTestSuite(TestDefaultController.class); // addTestSuite works with this type as well // as this method accepts a Test object as a parameter, & both TestCase // and TestSuite implement Test interface

Refer to Chapter 5 of JUnit in Action text book at techniques for automating tasks so that you Don’t have to create & maintain a TestAll class.

But, you still need to create specific suites to cater for discrete subsets of your tests.

S Ramakrishnan 22

TestResult

• A TestResult collects the results of a TestCase execution• TestResult stores the details of tests: pass or fail• assertEquals(30, result, 0) ; -- If result is not equal to 30, JUnit will create a TestFailure object & store it in TestResult• TestRunner uses the TestResult to report the outcome of your tests

• if no TestFailures in the TestResults collection, green bar!• if failures, TestRunner reports the failure count and the stack trace for failing tests.

• JUnit differentiates between failures & errors. Failure – for eg. - assertion failing when code is modified; Error is an unexpected condition such as an exception.

S Ramakrishnan 23

Design Patterns and JUnit

• Composite PatternCompose objects into tree structures to represent part-wholehierachies. Composite lets clients treat indiv. objects & Composition of objects uniformly (E Gamma et al. GOF1995)

• JUnit use of the Test interface to run a single test or a suite of tests is an e.g. of the composite pattern.

• When you add an object to a TestSuite, you are adding a Test, and not just a TestCase. Since both TestCase & TestSuite implement Test, you can add either to a suite. If the Test is a TestCase, the single test is run, when the Test is a TestSuite, group of tests is run, which in turn can have other testsuites.

S Ramakrishnan 24

Design Patterns and JUnit

• Command pattern “Encapsulate a request as an object, thereby letting you parameterize clients with different requests, queue or log requests, and support undoable operations” (E Gamma et al. 1995)

• E.g. of the command pattern : the use of the Test interface to provide a common run method

• Collecting parameter “ When you need to collect results over several methods, you should add a parameter to the method and pass an object that will collect the results for you” (K Beck, Smalltalk Best Practices Pattern, Prentice-Hall, 1996)

• E.g. TestResult class is an eg. of the collecting parameter pattern

S Ramakrishnan 25

Design Patterns and JUnit

• Observer Pattern“Design a one-to-many dependency between objects so that when one object changes state, all its dependencies are notified andupdated automatically”.

• TestRunner registering as a TestListener with the TestResult is and eg. of an Observer pattern.

S Ramakrishnan 26

The Core JUnit Classes

Source: JUnit in Action

<<Interface>> Test

Assert

TestCase TestSuite

O .. *

TestResult BasicTestRunner

<<Interface>> TestListener

O .. 1

0 .. *

S Ramakrishnan 27

JUnit’s TestListener interface

• TestResult collects info about the test and TestRunner reports it.• Can more than one object report on a test at the same time?• Does an object have to be a TestRunner to report on a test?

• JUnit framework provides the TestListener interface to help access the TestResult and produce meaningful reports.

• The TestRunners implement TestListener• A number of TestListeners can register with the framework and do anything required with the info provided by the TestResult.• TestListener is not an interface you need to implement to write your tests with the JUnit framework, unless you are extending the framework.

S Ramakrishnan 28

TestCase components – Fixtures and Unit Tests

• Fixture is a set of resources or data that you may need to run a series of tests, eg. DB connection & access test tables

• avoids putting the common set up code in your tests

• Fixture is automatically created & destroyed by a TestCase through its setUpsetUp & tearDowntearDown methods.

• TestCase calls setUpsetUp before running each of its tests and calls the tearDowntearDown when each test is completed. • TestCase life cycle recreates the scaffolding (fixture) for each test method. Putting a number of test methods in the same TestCase enables sharing the fixture code.

setUp () testXXX ( ) tearDown ( )

S Ramakrishnan 29

Fixtures – to manage resources

• Mock objects or stubs can be used to simulate db connections – refer to ch. 6 & 7 of JUnit in Action text book & urls in the reference

• Managing resources such as db connections• Testcase may include several db tests, each needing a fresh connection to the db• fixture makes it easy for you to open a connection for each test without replicating code

• can use a fixture to generate input files, so you don’t carry test files with your tests & also able to have a known state before a test is executed.

S Ramakrishnan 30

Unit Test methods

Assert Interface and assert methodsAssert Interface and assert methods

•The JUnit framework encapsulates common testing tasks with assertssert methods. The assert methods are defined in a utility class called AssertAssert. The class provides 8

core public methods (assertXXX) for building tests:assertTrue, assertFalse, assertEquals, assertNotNull, assertNull, assertSame, assertNotSame, fail - with a number of forms for these methods to make it easy to pass various types that you may need in your test (Refer to Javadoc for details on Assert interface)

• commonly used form of assertEquals • assertEquals (String message, Objects expected, Object actual)

S Ramakrishnan 31

Unit Test methods

• TestCase implements 10 TestCase methods, which are not part of the Assert interface:

countTestCase, createResult, getName, run, runBare, runTest, setName, setUp, tearDown, toString

• TestCases mainly use setUp & tearDown methods. Developers involved with JUnit extensions may be interested in the other 8 methods given above.

• 8 assert methods from Assert and 10 testcase methods from TestCase are available for building unitTests with JUnit.

S Ramakrishnan 32

Unit Tests

• As the name suggests, unit tests should run independently of other unit tests.• unit tests must run in any order and must not depend on effects of previous tests.• Dependency between unit tests are a problem in JUnit

• Portability issues - JUnit finds test methods by reflection. Reflection API does not guarantee the order in which test method names are returned. If your tests depend on ordering, your testsuite may work on one JVM but fail in another JVM.• Maintenance issues – if tests are dependent & you change one test, you may find that other tests are affected. Need to understand how each works. Tests become hard to read & maintain.

S Ramakrishnan 33

JUnit Results- Reporting

• If you run JUnit in a text mode, it gives a very limited written report on the console.

• JUnit reports with Swing testrunners are visual with the green & red bar, and with exception traces for reporting failures & errors but

non permanent.

• If reports are needed for later review and for documentation purposes,you can do it using Ant, Eclipse and some other tools.

.

S Ramakrishnan 34

JUnit Results- Reporting

• Can improve the default JUnit reporting by using the Ant’s “JUnit’s report” feature to produce XML output & html pages

• the Java build tool “Ant” formatting features can turn an ordinary JUnit report into a nice set of web pages.

• use “Ant” to run JUnit to produce pretty printed JUnitReports.