1

Software Engineering

Muhammad Fahad Khan

fahad@uettaxila.edu.pkUniversity Of Engineering & University Of Engineering &

Technology Taxila,PakistanTechnology Taxila,Pakistan

Today’s Class Chapter 14

Software Testing TechniquesSoftware Testing Techniques R.S.Pressman

3

OverviewOverview What is it?What is it?

To design a series of test cases that have high likelihood of finding errors.To design a series of test cases that have high likelihood of finding errors. Testing techniques provide systematic guidance of designing tests that Testing techniques provide systematic guidance of designing tests that (1) exercise the internal logic and interface of every software component(1) exercise the internal logic and interface of every software component (2) exercise the input and output domains of the program to uncover (2) exercise the input and output domains of the program to uncover

errors in program function, behavior, and performance.errors in program function, behavior, and performance. Who does it?Who does it?

Early stage- software engineerEarly stage- software engineer Later – testing specialistsLater – testing specialists

Why important?Why important? In order to find the highest possible number of error, test must be In order to find the highest possible number of error, test must be

conduct systematically and test cases must be designed using disciplined conduct systematically and test cases must be designed using disciplined techniquestechniques

Steps?Steps? Work products?Work products?

Test casesTest cases

4

TestabiliTestabilitytyHow easily a program can be tested?How easily a program can be tested?

OperabilityOperability—the better it works, the more efficiently it can be —the better it works, the more efficiently it can be testedtested

ObservabilityObservability—the results of each test case are readily —the results of each test case are readily observed, source code, variables are visible? observed, source code, variables are visible?

ControllabilityControllability—the degree to which testing can be automated —the degree to which testing can be automated and optimizedand optimized

DecomposabilityDecomposability—testing can be targeted. Software is built as —testing can be targeted. Software is built as independent modules that can be tested independentlyindependent modules that can be tested independently

SimplicitySimplicity—reduce complex architecture and logic to simplify —reduce complex architecture and logic to simplify teststests

StabilityStability—few changes are requested during testing—few changes are requested during testing UnderstandabilityUnderstandability—of the design—of the design

5

OutlineOutline Software testing fundamentalSoftware testing fundamental White-box testingWhite-box testing Basic path testingBasic path testing Control structure testingControl structure testing Black-box testingBlack-box testing Object-oriented testing methodsObject-oriented testing methods

6

What is a “Good” Test?What is a “Good” Test?

A good test has a high probability of A good test has a high probability of finding an errorfinding an error

A good test is not redundant.A good test is not redundant. A good test should be “best of breed” A good test should be “best of breed” A good test should be neither too A good test should be neither too

simple nor too complexsimple nor too complex

7

Test Case Test Case DesignDesign

"Bugs lurk in corners"Bugs lurk in corners and congregate atand congregate at boundaries ..."boundaries ..."

OBJECTIVEOBJECTIVE

CRITERIACRITERIA

CONSTRAINTCONSTRAINT

to uncover errorsto uncover errors

in a complete mannerin a complete manner

with a minimum of effort and timewith a minimum of effort and time

8

Exhaustive TestingExhaustive Testing

loop < 20 Xloop < 20 X

There are 10 possible paths! If we execute oneThere are 10 possible paths! If we execute onetest per millisecond, it would take 3,170 years totest per millisecond, it would take 3,170 years to

test this program!!test this program!!

1414

9

Selective TestingSelective Testing

loop < 20 Xloop < 20 X

Selected pathSelected path

10

Software TestingSoftware Testing

Methods

Strategies

white-boxmethods

black-box

methods

11

White-Box White-Box TestingTesting

... our goal is to ensure that all... our goal is to ensure that all statements and conditions havestatements and conditions have been executed at least once ...been executed at least once ...

12

White-Box TestingWhite-Box Testing Glass-box testingGlass-box testing Design philosophyDesign philosophy

Use the control structure described as part of Use the control structure described as part of component-level design to derive test casecomponent-level design to derive test case

What kind of test cases that software engineers What kind of test cases that software engineers can derivecan derive all independent pathsall independent paths Logical decisionLogical decision Loops at the boundary Loops at the boundary Internal structuresInternal structures

13

ExampleExample Path 1:1-11Path 1:1-11 Path 2: 1-2-3-4-5-Path 2: 1-2-3-4-5-

10-1-1110-1-11 Path 3: 1-2-3-6-8-9-Path 3: 1-2-3-6-8-9-

10-1-1110-1-11 Path 4: 1-2-3-6-7-9-Path 4: 1-2-3-6-7-9-

10-1-1110-1-11

1

2

3

64

587

910

11

14

Basis Path Basis Path TestingTestingFirst, we compute the cyclomatic

complexity:number of simple decisions + 1

ornumber of enclosed areas + 1In this case, V(G) = 4

15

Cyclomatic Cyclomatic ComplexityComplexityA number of industry studies have indicatedA number of industry studies have indicated

that the higher V(G), the higher the probability that the higher V(G), the higher the probability or errors.or errors.

V(G)V(G)

modulesmodules

modules in this range are modules in this range are more error pronemore error prone

TECHNICAL DETAIL The cyclomatic complexity of a software module is

calculated from a connected graph of the module (that shows the topology of control flow within the program)

How To Find ? Cyclomatic complexity (CC) = E - N + p where E = the number of edges of the graph N = the number of nodes of the graph p = the number of connected components

17

Basis Path Basis Path TestingTestingNext, we derive the Next, we derive the

independent paths:independent paths:

Since V(G) = 4,Since V(G) = 4,there are four pathsthere are four paths

Path 1: 1,2,3,6,7,8Path 1: 1,2,3,6,7,8Path 2: 1,2,3,5,7,8Path 2: 1,2,3,5,7,8Path 3: 1,2,4,7,8Path 3: 1,2,4,7,8

Path 4: 1,2,4,7,2,4,...7,8Path 4: 1,2,4,7,2,4,...7,8

Finally, we derive testFinally, we derive testcases to exercise these cases to exercise these

paths.paths.

11

22

3344

55 66

77

88

18

Basis Path Testing Basis Path Testing NotesNotes

you don't need a flow chart, you don't need a flow chart, but the picture will help when but the picture will help when you trace program pathsyou trace program paths

count each simple logical test, count each simple logical test, compound tests count as 2 or compound tests count as 2 or

moremore

basis path testing should be basis path testing should be applied to critical modulesapplied to critical modules

19

Graph MatricesGraph Matrices A graph matrix is a square matrix whose size (i.e., A graph matrix is a square matrix whose size (i.e.,

number of rows and columns) is equal to the number of rows and columns) is equal to the number of nodes on a flow graphnumber of nodes on a flow graph

Each row and column corresponds to an identified Each row and column corresponds to an identified node, and matrix entries correspond to connections node, and matrix entries correspond to connections (an edge) between nodes. (an edge) between nodes.

By adding a By adding a link weightlink weight to each matrix entry, the to each matrix entry, the graph matrix can become a powerful tool for graph matrix can become a powerful tool for evaluating program control structure during testingevaluating program control structure during testing Probability that a link will be executedProbability that a link will be executed Process timeProcess time Memory, resourceMemory, resource

20

Control Structure TestingControl Structure Testing Condition testingCondition testing — a test case design method that — a test case design method that

exercises the logical conditions contained in a exercises the logical conditions contained in a program moduleprogram module <, <=, >, >=, not equal, >, or, and, not<, <=, >, >=, not equal, >, or, and, not

Data flow testingData flow testing — selects test paths of a program — selects test paths of a program according to the locations of definitions and uses of according to the locations of definitions and uses of variables in the programvariables in the program DEF(s)={x|statement S contains a definition of X}DEF(s)={x|statement S contains a definition of X} USE(s)={x|statement S contains a use of X}USE(s)={x|statement S contains a use of X} A definition-use (DU) chain of variable X is of the form A definition-use (DU) chain of variable X is of the form

[X,S,S’]. [X,S,S’].

21

Loop TestingLoop Testing

Nested Nested LoopsLoops

ConcatenatedConcatenated Loops Loops Unstructured Unstructured

LoopsLoops

Simple Simple looploop

22

Loop Testing: Simple Loop Testing: Simple LoopsLoops

Minimum conditions—Simple LoopsMinimum conditions—Simple Loops1. skip the loop entirely1. skip the loop entirely

2. only one pass through the loop2. only one pass through the loop3. two passes through the loop3. two passes through the loop

4. m passes through the loop m < n4. m passes through the loop m < n5. (n-1), n, and (n+1) passes through 5. (n-1), n, and (n+1) passes through

the loopthe loopwhere n is the maximum number where n is the maximum number

of allowable passesof allowable passes

23

Loop Testing: Nested Loop Testing: Nested LoopsLoops

Start at the innermost loop. Set all outer loops to their Start at the innermost loop. Set all outer loops to their minimum iteration parameter values.minimum iteration parameter values.

Test the min+1, typical, max-1 and max for the Test the min+1, typical, max-1 and max for the innermost loop, while holding the outer loops at their innermost loop, while holding the outer loops at their

minimum values.minimum values.Move out one loop and set it up as in step 2, holding all Move out one loop and set it up as in step 2, holding all other loops at typical values. Continue this step until other loops at typical values. Continue this step until

the outermost loop has been tested.the outermost loop has been tested.

If the loops are independent of one another If the loops are independent of one another then treat each as a simple loopthen treat each as a simple loop else* treat as nested loopselse* treat as nested loops

endif* endif*

Nested LoopsNested Loops

Concatenated LoopsConcatenated Loops

24

Black-Box TestingBlack-Box Testing

requirementsrequirements

eventseventsinputinput

outputoutput

25

Black-Box TestingBlack-Box Testing Behavioral testingBehavioral testing Focus on the functional requirements of the Focus on the functional requirements of the

softwaresoftware Attempts to find errors in the following categoriesAttempts to find errors in the following categories

Incorrect/miss functionIncorrect/miss function Errors in data structures or external database accessErrors in data structures or external database access Behavior/performance errorsBehavior/performance errors Initialization/termination errorsInitialization/termination errors

26

Black-Box TestingBlack-Box TestingTests are designed to answer the following questionsTests are designed to answer the following questions

How is functional validity tested?How is functional validity tested? How is system behavior and performance tested?How is system behavior and performance tested? What classes of input will make good test cases?What classes of input will make good test cases? Is the system particularly sensitive to certain input Is the system particularly sensitive to certain input

values?values? How are the boundaries of a data class isolated?How are the boundaries of a data class isolated? What data rates and data volume can the system What data rates and data volume can the system

tolerate?tolerate? What effect will specific combinations of data have on What effect will specific combinations of data have on

system operation?system operation?

27

Graph-Based MethodsGraph-Based Methods

newfile

menu select generates(generation time 1.0 sec)

documentwindow

documenttex

t

is represented as

contains

Attributes:

background color: whitetext color: default color

or preferences

(b)

object#1

Directed link(link weight)

object#2

object#3

Undirected link

Parallel links

Node weight(value

)

(a)

allows editingof

To understand To understand the objects that the objects that are modeled in are modeled in software and software and the the relationships relationships that connect that connect these objectsthese objects

In this context, we In this context, we consider the term consider the term “objects” in the “objects” in the broadest possible broadest possible context. It context. It encompasses data encompasses data objects, traditional objects, traditional components components (modules), and (modules), and object-oriented object-oriented elements of elements of computer software.computer software.

28

Equivalence PartitioningEquivalence Partitioning Black-box testingBlack-box testing Divides the input domain of a program into Divides the input domain of a program into

classes of data from which test cases can be classes of data from which test cases can be derived.derived.

One or more tests are derived from each of the One or more tests are derived from each of the class. class.

IdeaIdea If one test fails, most likely all the tests in the domain If one test fails, most likely all the tests in the domain

will failwill fail

29

Equivalence PartitioningEquivalence Partitioning

useruserqueriesqueries mousemouse

pickspicks

outputoutputformatsformats

promptsprompts

FKFKinputinput

datadata

30

Sample Equivalence Sample Equivalence ClassesClasses

user supplied commandsuser supplied commandsresponses to system promptsresponses to system prompts

file namesfile namescomputational datacomputational data

physical parameters physical parameters bounding valuesbounding values initiation valuesinitiation valuesoutput data formattingoutput data formatting

responses to error messagesresponses to error messagesgraphical data (e.g., mouse picks)graphical data (e.g., mouse picks)

data outside bounds of the program data outside bounds of the program physically impossible dataphysically impossible data

proper value supplied in wrong placeproper value supplied in wrong place

Valid dataValid data

Invalid dataInvalid data

31

Boundary Value AnalysisBoundary Value Analysis A complement of equivalence partitioningA complement of equivalence partitioning Select the edges of the classSelect the edges of the class Include Include inputinput and and outputoutput

32

Boundary Value Boundary Value AnalysisAnalysis

useruserqueriesqueries mousemouse

pickspicks

outputoutputformatsformats

promptsprompts

FKFKinputinput

datadata

outputoutputdomaindomaininput domaininput domain

33

Boundary Value AnalysisBoundary Value AnalysisHow do I create BVA test cases?How do I create BVA test cases?

Input:Input: range a and b range a and b a, b, just above and just below a and ba, b, just above and just below a and b

Input:Input: a number of value a number of value Min, max, just above and just below min and maxMin, max, just above and just below min and max

Output:Output: Data structure has boundariesData structure has boundaries

arrayarray

34

Comparison TestingComparison Testing Used only in situations in which the reliability of Used only in situations in which the reliability of

software is absolutely critical (e.g., human-rated software is absolutely critical (e.g., human-rated systems)systems) Separate software engineering teams develop Separate software engineering teams develop

independent versions of an application using the same independent versions of an application using the same specificationspecification

Each version can be tested with the same test data to Each version can be tested with the same test data to ensure that all provide identical output ensure that all provide identical output

Then all versions are executed in parallel with real-time Then all versions are executed in parallel with real-time comparison of results to ensure consistencycomparison of results to ensure consistency

35

Orthogonal Array TestingOrthogonal Array Testing Used when the number of input parameters is

small and the values that each of the parameters may take are clearly bounded

One input item at a time L9 orthogonal array

XY

Z

XY

Z

36

OOT—Test Case OOT—Test Case DesignDesignImplications of OO conceptsImplications of OO concepts

1.1.EncapsulationEncapsulationa)a) Testing operation outside of the class is unproductiveTesting operation outside of the class is unproductiveb)b) Obstacle. Why? Information hideObstacle. Why? Information hide

2.2.InheritanceInheritancea)a) Each mew context of usage requires retesting, even Each mew context of usage requires retesting, even

though reuse has been achieved.though reuse has been achieved.3.3.Polymorphism - take on multiple formsPolymorphism - take on multiple forms

a)a) attribute have more than one set of values and an attribute have more than one set of values and an operation may be implemented by more than one operation may be implemented by more than one method. method.

b)b) Ex –abstract card classEx –abstract card class

37

Testing MethodsTesting Methods Fault-based testingFault-based testing

The tester looks for plausible faults (i.e., aspects of the implementation The tester looks for plausible faults (i.e., aspects of the implementation of the system that may result in defects). To determine whether these of the system that may result in defects). To determine whether these faults exist, test cases are designed to exercise the design or code. faults exist, test cases are designed to exercise the design or code.

Integration testing-three types in the client (Integration testing-three types in the client (not servernot server)) Unexpected resultUnexpected result Wrong operation/message usedWrong operation/message used Incorrect invocationIncorrect invocation

Class Testing and the Class HierarchyClass Testing and the Class Hierarchy Inheritance does not obviate the need for thorough testing of all Inheritance does not obviate the need for thorough testing of all

derived classes. In fact, it can actually complicate the testing process.derived classes. In fact, it can actually complicate the testing process. Scenario-Based Test DesignScenario-Based Test Design

Scenario-based testing concentrates on what the user does, not what Scenario-based testing concentrates on what the user does, not what the product does. This means capturing the tasks (via use-cases) that the product does. This means capturing the tasks (via use-cases) that the user has to perform, then applying them and their variants as tests.the user has to perform, then applying them and their variants as tests.

38

OOT Methods: Random OOT Methods: Random TestingTesting

Random testingRandom testing identify operations applicable to a classidentify operations applicable to a class define constraints on their usedefine constraints on their use identify a minimum test sequenceidentify a minimum test sequence

an operation sequence that defines the minimum life history an operation sequence that defines the minimum life history of the class (object)of the class (object)

generate a variety of random (but valid) test sequencesgenerate a variety of random (but valid) test sequences exercise other (more complex) class instance life historiesexercise other (more complex) class instance life histories

ExampleExample AccountAccount Open->deposit->withdraw->closeOpen->deposit->withdraw->close

39

OOT Methods: Partition OOT Methods: Partition TestingTesting Partition TestingPartition Testing

reduces the number of test cases required to test a class in much reduces the number of test cases required to test a class in much the same way as equivalence partitioning for conventional softwarethe same way as equivalence partitioning for conventional software

state-based partitioningstate-based partitioning categorize and test operations based on their ability to change the state categorize and test operations based on their ability to change the state

of a classof a class AccountAccount

Operations changes the state (withdraw, deposit)Operations changes the state (withdraw, deposit) Operations do not change the state (balance, report)Operations do not change the state (balance, report)

attribute-based partitioningattribute-based partitioning categorize and test operations based on the attributes that they usecategorize and test operations based on the attributes that they use

Balance, credit limitBalance, credit limit category-based partitioningcategory-based partitioning

categorize and test operations based on the generic function each categorize and test operations based on the generic function each performsperforms

AccountAccount Init: open, setupInit: open, setup Computation: deposit, withdrawComputation: deposit, withdraw Query: balance, summarizeQuery: balance, summarize

40

OOT Methods: Inter-Class OOT Methods: Inter-Class TestingTesting Inter-class testingInter-class testing

For each client class, use the list of class operators to For each client class, use the list of class operators to generate a series of random test sequences. The generate a series of random test sequences. The operators will send messages to other server classes.operators will send messages to other server classes.

For each message that is generated, determine the For each message that is generated, determine the collaborator class and the corresponding operator in collaborator class and the corresponding operator in the server object.the server object.

For each operator in the server object (that has been For each operator in the server object (that has been invoked by messages sent from the client object), invoked by messages sent from the client object), determine the messages that it transmits.determine the messages that it transmits.

For each of the messages, determine the next level of For each of the messages, determine the next level of operators that are invoked and incorporate these into operators that are invoked and incorporate these into the test sequencethe test sequence

41

OOT Methods: Behavior TestingOOT Methods: Behavior Testingemptyacctopen setup Accnt

set upacct

deposit(initial)

workingacct

withdrawal(final)

deadacct close

nonworkingacct

deposit

withdrawbalance

creditaccntInfo

Figure 14.3 State diagram for Account class (adapted from [ KIR94])

The tests to be The tests to be designed designed should achieve should achieve all state all state coverage. That coverage. That is, the is, the operation operation sequences sequences should cause should cause the Account the Account class to make class to make transition transition through all through all allowable allowable statesstates

42

Testing Testing PatternsPatterns

Pattern name:Pattern name: pair testing pair testingAbstract: Abstract: A process-oriented pattern, pair testing describes a A process-oriented pattern, pair testing describes a technique that is analogous to pair programming in which two technique that is analogous to pair programming in which two testers work together to design and execute a series of tests that testers work together to design and execute a series of tests that can be applied to unit, integration or validation testing activities.can be applied to unit, integration or validation testing activities.

Pattern name: Pattern name: separate test interface separate test interfaceAbstract: Abstract: There is a need to test every class in an object-oriented There is a need to test every class in an object-oriented system, including “internal classes” (i.e., classes that do not system, including “internal classes” (i.e., classes that do not expose any interface outside of the component that used them). expose any interface outside of the component that used them). The separate test interface pattern describes how to create “a The separate test interface pattern describes how to create “a test interface that can be used to describe specific tests on test interface that can be used to describe specific tests on classes that are visible only internally to a component.” classes that are visible only internally to a component.”

Pattern name: Pattern name: scenario testing scenario testingAbstract: Abstract: Once unit and integration tests have been conducted, Once unit and integration tests have been conducted, there is a need to determine whether the software will perform in there is a need to determine whether the software will perform in a manner that satisfies users. The scenario testing pattern a manner that satisfies users. The scenario testing pattern describes a technique for exercising the software from the user’s describes a technique for exercising the software from the user’s point of view. A failure at this level indicates that the software point of view. A failure at this level indicates that the software has failed to meet a user visible requirement.has failed to meet a user visible requirement.

Question & Review session

For any query feel free to contact For any query feel free to contact fahad@uettaxila.edu.pk

http://web.uettaxila.edu.pk/CMS/seSE2bs/index.asp