View
217
Download
1
Tags:
Embed Size (px)
Citation preview
Unit Testing
CSSE 376, Software Quality Assurance
Rose-Hulman Institute of Technology
March 27, 2007
2
Outline
Role of unit testingTesting strategies
Black box methodsWhite box methods
3
Role of Unit Testing
Assure minimum quality of units before integration into system
Focus attention on relatively small units
Marks end of development step
4
Testing versus Debugging
Testing Debugging
planned unplanned
results are recorded
unrecorded
may be done by non-developers
always done by developers
look for errors look for causes of errors
5
Limits of Testing
Testing can never demonstrate the absence of errors
Exhaustive testing is infeasibleTesting may be done imperfectly
6
Strategies for Unit Testing
Black boxuse only specification of programtest implementation against its
specification
White boxuse structure or other properties of a
program to generate tests
7
Black Box Methods
Equivalence partitioningBoundary value analysis
8
White Box Methods
9
White Box Methods
Coveragestatementbranchpath
CyclomaticDataflowMutationError Based
10
Coverage Methods
Statementexecute each statement
Branchexecute each branch
Pathexecute each path
11
Statement Coverage
Execute each statement in the program
Considered minimum criterion for most unit testing
May be difficult to achieve for error cases
12
Example Program
1: if (a < 0) {
2: return 0 }
3: r = 0;
4: c = a;
5: while (c > 0) {
6: r = r + b;
7: c = c - 1; }
8: return r;
13
Statement Tests
a = 3, b = 4executes 1, 3, 4, 5, 6, 7, 5, 6, 7, 5, 6, 7, 5,
8
a = -3, b = 2executes 1, 2
14
Branch Coverage
Execute each branch of the program at least once
Differs from statement coverage only for "if" statements without "else"s and case statements without default cases.
15
Dataflow TestingMore than branch coverage, but less
than path coverageUses information about references to
variables to select paths
16
Definitions and UsesDefining node
input statementlhs of assignment
Usage nodeoutput statementrhs of assignmentcontrol statement
17
Suspicious PathsVariable is defined (set to a new
value) but never referencedVariable is referenced but never
definedVariable is defined twice before it is
used
18
DU PathsDefinition-use (du) path wrt V:
a path containing a defining node for V at the beginning a usage node for V at the end, and no def's or use's in between
DU testing: execute each du-path of each variable
19
Example Program1: if (a < 0) {
2: return 0 }
3: r = 0;
4: c = a;
5: while (c > 0) {
6: r = r + b;
7: c = c - 1; }
8: return r;
20
Example DU Paths(corrected 3/28/07)
Def (c) = {4, 7}Use (c) = {5, 7}Def (r) = {3, 6}Use (r) = {6, 8}du-paths for c:
4 - 5, 7 – 5
du-paths for r:3 - 4 - 5 - 6, 6 - 7 - 5 - 6, 3 - 4 - 5 - 8,
6 - 7 - 5 - 8
21
Test Cases for DU Paths (corrected 3/28/07)
a = 21 - 3 - 4 - 5 - 6 - 7 - 5 - 6 - 7 - 5 - 8
Covers du-paths:4 - 5, 7 - 5, 3 - 4 - 5 - 6, 6 - 7 - 5 - 6
22
Cartoon of the Day
23
Mutation Testing(As Applied to White-Box)
Path testing exercises the control of a program, not the computations along the paths
Most programs under test are "almost" right
24
Mutation Method1. Pre-process program to generate
mutants2. Execute all the mutants and the
original program3. If a mutant differs from the original,
then it is "killed"4. If all mutants are killed, then test set
is adequate.
25
Mutation Metaphor
26
Example Programfunction square( x: integer): integer;
begin
square := x * x
end
27
Example Mutantfunction square( x: integer): integer;
begin
square := x + x
end
28
Executing MutantsTest set {2} does
not kill the mutantTest set {2, 4} does
kill the mutant, and it reveals the flaw in the program
29
Which Mutants?Examples:
Off by one errors (i+1, i, i-1)
Different variable (i, j, k)
30
Assumptions ofMutation Testing
Competent Programmer: The perfect program is very close to the program under test
Oracle Hypothesis: You know the output for any input
Continuity: Every complicated mistake can be caught by looking for simple mistakes.
31
Problems with Mutation Very expensive
each test runs the whole program many mutants for every program
Some mutants fail to halt May be difficult to detect when
a mutant is really just an equivalent program
Not reliable: may not try the right mutants