Software Testing and Quality Assurance: Class Testing Basics Reading Assignment: –John McGregor and David A. Sykes, A Practical Guide to Testing Object-Oriented

Software Testing and Quality Assurance: Class Testing Basics

• Reading Assignment: – John McGregor and David A. Sykes, A Practical

Guide to Testing Object-Oriented Software, Addison-Wesley, 2001, ISBN: 0-201-325640.

• Chapter 5: Class Testing Basics

– JUnit Cookbook• http://junit.sourceforge.net/doc/cookbook/cookbook.htm

Objectives

• To know what to consider when testing a class.

• To identify test cases for testing a class.

• To know a good way to implement a test driver for a class.

• Get familiar with a software testing tool (JUnit).

Topics Covered

• Class testing

• Constructing test cases

• Constructing a test driver

Class Testing

• A class is the fundamental unit of an object-oriented program.

• Class testing comprises the activities associated with verifying that the implementation of a class corresponds exactly with the specification for that class.

• Class testing is roughly analogous to unit testing in traditional testing processes.

• The main focus is on execution-based testing.

Class Testing (cont...)

• Assumptions:– Class Under Test (CUT) has a complete

specification expressed in a specification language like Object Constraint Language (OCL), a natural language, and/or state transition diagram

– If more than one form of specification is used for a class:

• All forms are consistent.

• Information can be taken from any model.

Class Testing: Ways to Test a Class

• Reviews:– Reviews are subject to human error.– Reviews require considerably more effort with

respect to regression testing.

• Executing test cases:– Considerable effort can be required for the

identification of test cases and the development of test drivers.

Class Testing and Unit Testing

• The purpose of unit testing is to ensure that each unit meets its specification.

• If the unit meets its specification, then any bugs that appear when units are integrated are more likely caused by incorrect interfacing of units.

• Unit testing is done as units are developed.• Units are tested by code inspection and execution testing

(more emphasis on execution testing).• Units that have parameters can be unit tested if the driver

can initialize the actual parameters.• Should the test driver itself be tested?

Class Testing and Unit Testing (cont...)

• Identify test cases

• Test driver construction: create one or more instances of a class to run a test case

• Execution

• Result analysis

Class Testing: Dimensions of Class Testing

• Should we test a class independently as a unit or as a component of a larger part of the system, this decision is based on:– The role of the class in the system; the risk associated with

it.– The complexity of the class measured in terms of:

• The number of states• Operations• Associations with other classes

– The amount of effort associated with developing a test driver for the class.

Class Testing: Dimensions of Class Testing (cont...)

• Who — developer• What — code implements the specification

exactly• When —

– A test plan should be developed soon after the specification of the class is ready for coding

– Class testing should be done prior to its use in other portion of the software

– Regression class testing should be performed whenever the class implementation is changed

Class Testing: Dimensions of Class Testing (cont...)

• How — develop a test driver that creates instances of the class and sets up a suitable environment around these instances to run a test case

• How much — – Test operations and state transitions in all sorts

of combinations– Exhaustive or selective with combination with

risk analysis.

Constructing Test Cases

• Possible ways for identifying test cases– Class specification

• Pre- and post-conditions (OCL)

• State transition diagram

– Class implementation


• Test case construction from Pre- and Post-conditions– Programming style: defensive vs. contract

– Steps in generating test cases from Pre- and Post-conditions• Identify a list of pre-condition contributions

• Identify a list of post-condition contributions

• Form test case requirements by making all possible combinations of entries from the contributions lists

• Eliminate any conditions generated that are not meaningful

– Example• Figures 5.2, 5.3, 5.4, 5.5, and 5.6

Constructing Test Cases: Example

Velocity class as specified in UML model (Figure 5.2)Velocity

speed: Speeddirection: Direction

Velocity()Velocity (speed: Speed, direction: Direction)

getSpeed(): SpeedgetSpeedX(): SpeedgetSpeedY(): SpeedgetDirection(): Direction

setSpeed(speed: Speed)setDirection(direction: Direction)Reverse()ReverseX()ReverseY()

Constructing Test Cases: Example (cont...)

• OCL specification for the method setDirection of Velocity class

Velocity::setDirection(dir:Direction)

Pre: 0 <= dir and dir < 360

Post: direction=dir and speed=speed@pre

• Figures 5.4, 5.5, and 5.6




Constructing Test Cases• Test case construction from state transition diagram:

– Each transition on the diagram represents a requirement for one or more test cases

– When testing based on state transition diagrams, make sure you investigate the boundaries and results of each transitions

• Example: Figure 5.8: Total 9 transitions: 6 between states; 1 for construction, 2 for destruction

Constructing Test Cases: Adequacy of Test Suites for a Class

• Commonly used measures of adequacy:– State-based coverage: how many of the

transitions in a state transition diagram are covered by the test suite.

– Constraint-based coverage: how many pairs of pre- and postconditions have been covered.

– Code-based coverage: how much of the code that implements the class is executed across all test cases in the suite.


• Boundary conditions:– Boundary is the input value on which a large change

occurs and must be identified when test cases are identified.– Example: sorting an array the boundary values can be

• An array with zero elements• An array with one element• An array with two elements• An array with many elements

– We need implementation-based testing, test cases derived from the specifications may not be enough:

• Example: the size of the array is used to choose the sorting algorithm for sorting an array

Constructing a Test Driver

• A test driver is a program that runs test cases and collects the results.

• Two approaches to build a test driver– Embed the testing into the class under test itself– Implement a separate class

• Keep relationship between CUT and its tester class through name, like a tester for Velocity class as VelocityTester.

Constructing a Test Driver: Test Driver Requirements

• Test driver should have relatively simple design.

• Test driver must be easy to maintain and adapt in response to changes.

• Test driver should be reusable to create new drivers.

Constructing a Test Driver: Test Driver Requirements (cont...)

• A class model for requirements for a tester class:– Public interface– Tester is an

abstract class.

Constructing a Test Driver: Test Driver Requirements (cont...)

• Test case suites:– Functional (specification-based)— if derived

from specification.– Structural (implementation-based)— if

identified from code.– Interaction — if derived from sequence of

events on an object such as pairs of input/output transitions.

Constructing a Test Driver: Tester Class Design

• Tester class example

Constructing a Test Driver: Test Class Design (cont...)

• Pseudo code for a typical test case method:

Constructing a Test Driver

• The main responsibility of the tester class is to run test cases and report results.

• Test case methods: one method per test case or group of closely related test cases:– Execute a test case by creating the input state,

generating a sequence of events, and checking the output state.

– Provide traceability to the test plan.

Constructing a Test Driver (cont...)

• Baseline testing: a baseline test suite is a set of test cases that tests the operations of the CUT that are needed for the other test cases to verify their outcome.

• This suite includes testing constructors and accessors.

• Two basic approaches to baseline testing: specification-based and implementation-based:– Check that all the constructors and accessors are self-consistent.

Create a test case for each constructor and verify that all attributes are correct by invoking accessors.

– Check that all the constructors and accessors use the variables in an object correctly.


• Running test suites: – The abstract Tester class includes in its

protocol some operations to run all test cases or selected test suites.

– Each calls a sequence of test case methods.– Make sure that the baseline test suite is

executed before any of these other suites are executed.


• Reporting test results: – A test case method determines the success of a

test case.– The test case should report results to the tester

instance.

Key Points• Ways to test a class: reviews and executing test cases• The purpose of unit testing is to ensure that each unit

meets its specification.• Possible ways for identifying test cases

– Pre- and post-conditions (OCL)

– State transition diagram

• Common ways used measures of adequacy: state-based coverage, constraint-based coverage and code-based coverage.

• Boundary conditions is the input value on which a large change occurs.

Key Points (cont...)• Constructing test drivers:

– Embed the testing into the class under test itself

– Implement a separate class

• Test driver should have relatively simple design, easy to maintain and reusable to create new drivers.

• Test case suites: functional, structural, and interaction• Test case methods: one method per test case or group of

closely related test cases:• Baseline — checking test cases results and accessor and

modifier methods.

JUnit TutorialJUnit

http://www.junit.org

Unit Test Automation

• Many advocate that unit test cases be automated and run. These test cases are then added to the code base (along with the code) and are re-run whenever new code is added to the code base.

• If new code breaks an old test case, it is easy to know right away that the new code broke some already implemented functionality. Most easily isolating the defect is of prime importance!

• Once you have written code that implements new functionality and this code passes its unit test cases, these test cases are then referred to as ‘regression’ test cases.

• We’ll be using JUnit for this…

What is JUnit?• JUnit is a regression testing framework.

• Written by Erich Gamma and Kent Beck.

• Used by developers to implement unit tests in Java.

• JUnit is integrated into a number of IDEs: BlueJ, JBuilder, and Eclipse.

• The test framework provides tools for:– assertions.– running tests.– aggregating tests (suites).– reporting results.

Why use JUnit?

• JUnit tests allow you to write code faster while increasing quality: less debugging time, more confidence, and enable refactoring.

• JUnit is simple: compiler tests syntax, JUnit tests integrity.

• JUnit tests check their own results and provide immediate feedback.

• JUnit test can be composed into a hierarchy of test suites.

• Create test assets... Retain their value and can be run and interpreted by other than original author.

• Effective, open source, integrated.

Terminology• A unit test is a test of a single class.• A test case tests the response of a single method to a

particular set of inputs.• A test suite is a collection of test cases.• A test runner is software that runs tests and reports

results.• An integration test is a test of how well classes work

together (JUnit provides some limited support for integration tests).

• A test fixture sets up the data (both objects and primitives) that are needed to run tests:– Example: If you are testing code that updates employee record,

you need an employee record to test it on.

Fixtures• Handle common objects under test.• setup() and tearDown() used to initialize

and release common objects.

• The JUnit framework automatically invokes the setUp() method before each test is run and the tearDown() method after each test is run.

• Used to insure there are no side effects between tests.

• Enforce the test independence rule, test execution order is not guaranteed.

What should be tested and what should not be tested?

• What should be tested:– Test things which could break (tests should

succeed quietly: don’t print “Doing foo…done with foo!”.

– Test boundary conditions– Test special conditions– Test exceptions

• What should not be tested:– Do not test set/get methods.– Do not test the compiler.

JUnit

JUnit• TestCase – a class that extends the JUnit TestCase class.

It contains one or more tests represented by testXXX methods. A test case is used to group together tests that exercise common behaviors.

• TestSuit - a group of tests. Test suite groups together related tests.

• TestRunner – A launcher of test suites. JUnit provides a number of test runners that you can use to execute your tests.– There is no test runner interface, only a base test runner that all

test runners extend.• You only need to write test cases. The other classes work

behind the scenes to bring your tests to life.

JUnit core classes

JUnit core classes

JUnit Mechanics• Define a subclass of TestCase.• Override the setUp() & tearDown() methods.• Define one or more public testXXX() methods:

– Exercise the object(s) under test.

– Asserts the expected results.

• Define a static suite() factory method:– Create a TestSuite containing all the tests.

• Optionally define main() to run the TestCase in batch mode.

Assert

• Assert within a test:– Call the method being tested and get the actual

result.

– assert what the correct result should be with one of the provided assert methods.

– These steps can be repeated as many times as necessary.

JUnit assert methods• static void assertTrue(boolean test)

static void assertTrue(String message, boolean test)– Throws an AssertionFailedError if the test fails (the optional message is

included in the Error)

• static void assertFalse(boolean test)static void assertFalse(String message, boolean test)– Throws an AssertionFailedError if the test fails.

• assertEquals(expected, actual)assertEquals(String message, expected, actual)– If the objects are not equal it throws an

AssertionFailedError – For objects, uses your equals method, if you have defined it properly, as

public boolean equals(Object o)--otherwise it uses ==

JUnit assert methods (cont...)• assertSame(Object expected, Object actual)assertSame(String message, Object expected, Object actual) – Asserts that two objects refer to the same object (using ==). If

they are not, it throws an AssertionFailedError.

• assertNotSame(Object expected, Object actual)

assertNotSame(String message, Object expected, Object actual) – Asserts that two objects do not refer to the same object. If they

do, it throws an AssertionFailedError.

JUnit assert methods (cont...)• assertNull(Object object)

assertNull(String message, Object object)– Asserts that the object is null.

If it isn’t, it throws an AssertionFailedError.

• assertNotNull(Object object)assertNotNull(String message, Object object)– Asserts that the object is not null.

If it isn’t, it throws an AssertionFailedError.

• fail()fail(String message)– Causes the test to fail and throw an AssertionFailedError– Useful as a result of a complex test, when the other assert methods aren’t

quite what you want.

When to use an assert statement

• Use assertTrue to document a condition that you “know” to be true.

• Use assertFalse; in code that you “know” cannot be reached (such as a default case in a switch statement).

Example: Counter class

• We will create and test a trivial “counter” class:– The constructor will create a counter and set it to zero.– The increment method will add one to the counter

and return the new value.– The decrement method will subtract one from the

counter and return the new value.

• We write the test methods before we write the code.

• Don’t be alarmed if, in this simple example, the JUnit tests are more code than the class itself.

Example: JUnit tests for Counter class (cont...)

Note that each test begins with a brand new counter

This means you don’t have to worry about the order in which the tests are run

public class CounterTest extends junit.framework.TestCase {

Counter counter1;

public CounterTest() { } // default constructor

protected void setUp() { // creates a (simple) test fixturecounter1 = new Counter();

}

protected void tearDown() { } // no resources to release

public void testIncrement() {assertTrue(counter1.increment() == 1);

assertTrue(counter1.increment() == 2);

}

public void testDecrement() {assertTrue(counter1.decrement() == -1);

}

}

Example: The Counter class itself (cont...)

public class Counter { int count = 0;

public int increment() {

return ++count; }

public int decrement() {

return --count; }

public int getCount() {return count;

} }

• Is JUnit testing overkill for this little class?

• The Extreme Programming view is: If it isn’t tested, assume it doesn’t work

• You are not likely to have many classes this trivial in a real program, so writing JUnit tests for those few trivial classes is no big deal

• Often even XP programmers don’t bother writing tests for simple getter methods such as getCount()

• We only used assertTrue in this example, but there are additional assert methods

Test suites

• Obviously you have to test your code to get it working in the first place:– You can do ad hoc testing (running whatever

tests occur to you at the moment).– You can build a test suite (a thorough set of

tests that can be run at any time).

Test suites: advantages and disadvantages

• Disadvantages of a test suite:– It’s a lot of extra programming:

• This is true, but use of a good test framework can help quite a bit.

– You don’t have time to do all that extra work:• False--Experiments repeatedly show that test suites

reduce debugging time more than the amount spent building the test suite.

• Advantages of a test suite:– Reduces total number of bugs in delivered code.

TestRunners: text

• Text:– Lightweight, quick quiet.

– Run from command line.

java StringTest

.......

Time: 0.05Tests run: 7, Failures: 0, Errors: 0

TestRunners: swing

• Run with java junit.swingui.TestRunner.

Example: point classpublic class Point {

private int x, y;

public Point(int x, int y){ this.x = x; this.y = y;}

public void setX(int x) { this.x = x; }

public void setY(int y) { this.y = y; }

public int getX() { return x; }

public int getY() { return y; }}

Example (cont...): point test without using JUnit

public class PointTestNoJunit {public static void main(String [] args) {

testSetX(new Point(10, 20), 0);testSetX(new Point(10, 20), 1);testSetX(new Point(10, 20), 30);

}private static void testSetX(Point p, int x) {

System.out.print("Before: " + toString(p) + ";");p.setX(x);System.out.println(" After setX(" + x + "): " +

toString(p));}private static String toString(Point p) {

return "<" + p.getX() + ", " + p.getY() + ">";}

}

Example (cont...): point test without using JUnit —output

Before: <10, 20>; After setX(0): <0, 20>



Before: <10, 20>; After setY(0): <10, 0>



Example (cont...): point test using JUnit

import junit.framework.*;/** A JUnit test class to test the class Point. */public class PointTest extends TestCase {

/** Tests constructor. */public void testPoint() {

Point p = new Point(10,20);assertEquals(10, p.getX());assertEquals(20, p.getY());

}/** Tests setX */public void testSetX() {

Point p = new Point(10, 20);p.setX(30);assertEquals(30, p.getX());

assertEquals(20, p.getY());}

//other test methods, e.g., for setY(), getX(), and getY().// cont…

Example (cont...): point test suite using JUnit

/** Returns the test suite for this test class. */public static Test suite() {

return new TestSuite(PointTest.class);}

/** Run the tests. */public static void main(String[] args) {

junit.textui.TestRunner.run(suite());// junit.swingui.TestRunner.run(suite());

}}

Example (cont...): point test using JUnit —output

C:/> javac Point.java PointTest.javaC:/> java PointTest

...F.....Time: 0.016There was 1 failure:testSetX1(PointTest)junit.framework.AssertionFailedError:

expected:<30> but was:<1> at PointTest.testSetX1(PointTest.java:58) … at PointTest.main(PointTest.java:17)

FAILURES!!!Tests run: 8, Failures: 1, Errors: 0

Example (cont...): point test using JUnit —Test Fixture

public class PointTest extends TestCase {private Point p; // test fixture variablepublic void setUp() { // initializes text fixture variables

p = new Point(10, 10);}

public void tearDown() { }//clean up text fixture variablespublic void testSetX() { // tests SetX

p.setX(20); assertEquals(20, p.getX());

}

public void testSetY() { // tests SetYp.setY(30); assertEquals(30, p.getY());

}

// template and other test methods here…}

JUnit Testing tips• Code a little, test a little, code a little, test a little . . .• Run your tests as often as possible, at least as often as you run the

compiler.• Begin by writing tests for the areas of the code that you’re the

most worried about . . .write tests that have the highest possible return on your testing investment.

• When you need to add new functionality to the system, write the tests first.

• If you find yourself debugging using System.out.println(), write a test case instead.

• When a bug is reported, write a test case to expose the bug.• Don’t deliver code that doesn’t pass all the tests.• Separate production and test code:

– But typically in the same packages.

Documents

Software Testing and Quality Assurance: Class Testing Basics Reading Assignment: –John McGregor and David A. Sykes, A Practical Guide to Testing Object-Oriented