Testing OO Programs(7.1, 2.2.2, 2.4.2)

Course Software Testing & Verification2014/15

Wishnu Prasetya

Plan

• The goal is to discuss how to test OO programs; to that end we also need some concepts from:

• Data flow perspective to testing• Integration testing

2

Basic idea

• The two paths of P above will carry the same value of x “defined” at the beginning to the return statement.

• A plain control flow graph does not describe how “data” flows through the program; so we can’t infer which execution paths need to be tested (or not) to test the influence of a certain variable on the program

3

P(x,y) { if (y<x) y = x+1 ; return x*y }

Data-flow based approach(sec. 2.2.2)

• Data flow graph : CFG, where the nodes and edges are decorated with information about the variables used and defined by each node/edge.

• In my examples I avoid decorating on the edges.• Some subtlety on the granularity of the nodes/edges later.

4

P1(x,y) { if (y<x) y = x+1 ; return x*y } n1

n2

n3

(then)

use(n1) = {x,y}

use{n2) = {x}def(n2) = {y}

use(n3) = {x,y}

n0 def (n0) = { x,y }

Terminologies

• Def/use graph: see above. In general, def/use labels are allowed on the edges as well.

• A path p = [i,...,k] is def-clear wrt v iff v def(j) and v def(e) for all nodes j and edges e in p between i and k.

• It (def-clear) is a du-path wrt v iff vdef(i) and vuse(k), and furthermore it is a simple path.

5

n1

n2

n3

use = {x,y}def = { x,y }

use = {y}def = {y}

n4

n5

use = {x,y}def =

use = {x,y}def =

use = {x,y}def ={x} du-paths wrt x:

13, 134, 135 12, 1234, 1235, 45Not du-path wrt x: 1345 (not def-clear)

Block’s granularity

• When a node both defines and uses the same variable x, we will assume that it uses x first, then assigns to it, as in x = ... x ...

• If this assumption would be broken (middle example) split the node. Else this will conflict the intention of your definition of du-path.

• Check 2.3.2 on how to map a source code to a def/use graph. 6

x = x+1 x = 0y = x+1 ;

x = 0y = x+1

nuse(n) = {x}def (n) = { x }

nuse(n) = {x}def (n) = { x,y }

n1use(n1) = {}def (n1) = { x}

n2use(n2) = {x}def (n2) = { y}

Data-flow based coverage(Rapps-Weyuker 1985, Frankl-Weyuker 1988)

• Def-path set du(i,v): the set of all du-paths wrt v, that starts at i.• Def-pair set du(i,j,v): the set of all du-paths wrt v, that starts at i,

and ends at j.• Same concept of touring.• (C 2.9, All Defs Covrg, ADC) For each def-path set S = du(i,v), TR

includes at least one member of S.7

n1 n2 n3

use = {x}def = { x}

use = {x}def ={x}

n4

use = {x}def =

use ={x}def =

Data-flow based coverage

• (C 2.10, All Uses Covrg, AUC) For each def-pair set S = du(i,j,v), TR includes at least one member of S.

• (C2.11, All du-paths Covrg, ADUPC) For each def-pair set S = du(i,j,v), TR includes all members of S.

• Note: the above example only has one variable, namely x; consequently all TRs above only concern x.

8

ADC TR: 12, 234

AUC TR: 12 23 , 232, 234

ADUC TR: 12 23 , 232, 234, 24

n1 n2 n3

use = {x}def = { x }

use = {x}def ={x}

n4

use = {x}def =

use ={x}def =

Overview of subsumption relations

• PPC ADUPC, because every simple path is a subpath of some prime path.

• AUC EC .... under the following assumptions:– every use (of v) is preceded by a def.– every def reaches at least one use.– multiple branches e.g. as in if(g) then S1 else S2 use the same variables (if

S1 uses x, then S2 at some point also uses x, vice versa). 9

complete path covrg

prime path covrg

edge-pair covrg

edge covrg

node covrg

ADUPC

AUC

ADC

Summary

• We learned another set of white-box test-coverage criteria, namely data-flow based coverage

• an alternative to prime-path-based testing, if the latter becomes too expensive.

10

Integration Test

• Integration test: to test if a composition of program units works correctly.

• “Hidden” in Section 2.4 about using design elements (e.g. your UML models) as your coverage criterion. 2.4.1 says: “testing software based on design elements is usually associated with integration testing”.

• 2.4.2 is actually more about integration testing rather than design-element-based testing.

11

Data flow approach to Integration Test (2.4.2)

• Imagine method f from module A calls method/API g from module B.

• Idea 1: construct the combined CFG of both, then use prime path coverage criteria. This blows up.

• Idea 2: focus on the flow of the “coupling data” around the call point.

12

f (a) { ... g(e) ... }

g(x) { ... } B

A

Some terminologies

• caller, callee, actual parameters, formal parameters• parameter coupling, shared variable coupling,

external device coupling• More general concept: coupling variable a

‘variable’ defined in one ‘unit’, and used in the other. 13

(1) a = ... ;b = ...

(3) a = ...(2) a>b

(4) b = g(b)

(1) g(x)

(2) x<=a

(3) x==a

(4) return ...

(5) return ...

Coupling du-path(Jin-Offutt, 1998)

• (Def 2.40) Last-def of v : set of nodes defining v, from which there is a def-clear path (wrt v) through the call-site, to a use in the other unit.

• Can be at the caller or callee! Example: f1, f3, g4, g5.• (Def 2.41) First-use (of v) E.g.: g2, f4• Coupling du-path of v is a path from v’s last-def to its first-use.

14

(1) a = ... ;b = ...

(3) a = ...(2) a>b

(4) b = g(b)

(1) g(x)

(2) x<=a

(3) x==a

(4) return ...

(5) return ...

First use at the callee ..

• When you just pass a variable as in g(b), it makes sense to define g’s first use to be the first time g uses the parameter (location g.3 in the previous example).

• But when you pass a complex expression like g(a+b) what are g’s first uses of a and b?

in this case it make sense to define g’s first uses (of a and b) to be at the call location itself.

15

Integration Test-Requirement

• ADC All Coupling Def Cov. : for every last-def d of each coupling var v, TR includes one coupling du-path starting from d.

• AUC All Coupling Use Cov. : for every last-def d and first-use u of v, TR includes one couple du-path from d to u.

16

Coupling vars: a, b, g.returnCoupling du-paths : a: [f1,f2,g1,g2] [f3,f4,g1,g2] b: [f1,f2,g1,g2] [f1,f2,f3,g1,g2] g.return: [g4,f4] [g5,f4]

(1) a = ... ;b = ...

(3) a = ...(2) a>b

(4) b = g(b)

(1) g(x)

(2) x<=a

(3) x==a

(4) return ...

(5) return ...

OO is powerful, but also comes new sources of errors

• A function behaves “cleanly”; a procedure does not.• An OO program is more complicated:

– side effects on instance variables and static variables

– access control (priv, pub, default, protected, ...)– dynamic binding – interaction through inheritance

17

Anomalies

• Simply mean non-standard situations. Not necessarily errors, but they tend to be error prone. AO list 9 anomalies related to inheritance and dynamic binding:– Inconsistent Type Use (ITU)– State definition anomaly (SDA) , ... 7 more

• AO also implicitly discuss two more: deep yoyo and data-flow anomaly.

18

Inconsistent Type Use (ITU) anomaly

19

List +insertElemAt(i:int)+remElemAt(i:int)

Stack +push(x)+pop()

This is a quite common subclassing architecture that people use. However note that this does not prevent a Stack to be used as a List, which is error prone (it allows you to call insertElemAt or remElemAt on a Stack).

State Definition Anomaly (SDA)

• ScalableItem (subclass) directly manipulates the part of its state inheritted from UnitItem (rather than through UnitItem’s method). This error prone: may break UnitItem’s class invariant.

20

ScalableItemsetScale(s) { scale = s }

UnitItemscale = 1price() { return 1/scale }

State definition inconsistency due to state variable hiding (SDIH)

• The shadowing of price done by OnSaleItem is error prone; notice the method netPrice (that it also inherits!) would use the old price.

21

ItempricetaxnetPrice() { return price * (1 + tax) }setPrice(p) { price = p }

OnSaleItempricesale(p) { price = 0.8 p }

State Visibility Anomaly (SVA)

22

A - x+ incr()

B C + incr2()

C’s needs a method incr2 that would increase x by 2. However it can’t get to x, because it is private to A. It can however calls super.incr() twice. So far it is ok. Now imagine someone else changes B by overriding incr. This suddenly changes the behavior of C; as it now calls B’s incr instead.

Anomalous Construction Behavior 1 (ACB1)

• Inheritence may radically change the orginal behavior.

• Item’s constructor calls reset().• ExpensiveItem’s constructor calls Item’s constructor,

but implicitly forces it to call the new reset() !• Called a Yoyo-effect phenomenon.

23

Itempricetax Item() { reset() ; tax=0 }reset() { price = 1 }

ExpensiveItembasepriceExpensiveItem() { super() ; baseprice=10 }reset() { price = baseprice }

Call graph

• Here, it is a graph describing which program “units” call which. .

24

f

g h

Here, we’ll define f g means f contains a call to g in its body (note that it does not mean that in the runtime f will actually call g; it may depend on its parameters) The two branches from f in the example says nothing about the order in which the children will be called by f.

The call graph of d() in ABCyoyo graph, Alexander-Offut 2000

25

C +j () { k() }

B +h () { i() ...} + i() { super.i() ...}+k() { }

A +d () { g() }+g () { h() }+h () { i() }+i () { j() }+j () {}

C j()

B h() i()

A d() j()g() h() i()

A d() j()g() h() i()

B h() i() k()

A d() j()g() h() i()

Data flow anomaly

26

B +h ()+i ()

A +d ()+g ()+h ()+i ()+j ()

B h() i()

A d() j()g() h() i()

A d() j()g() h() i()

def x use x

Not necessarily an error. E.g. if at the runtime the use of x in j() when it is called from B’s i() is actually unfeasible. But this is still error prone (anomaly)

def x use x

Check the remaining anomalies yourself

• State Defined Incorrectly (SDI)• Indirect Inconsistent State Definition (IISD)• Anomalous Construction Behavior 2 (ACB2)• Incomplete Construction (IC)

27

Testing your OO programs

• Intra-method testing (method-level unit-testing)• Intra-class testing, as done by class-level unit testing.• Inter-class testing.• The first two levels are usually not enough to expose

errors related to access control, interaction through inheritance, and dynamic binding.

• Let’s now take a look at inter-class testing.

28

Inter-class testing

• Testing the interaction of class A and E : treat this as an integration testing problem we already have a solution (data-flow based integration test), but additionally we need to quantify over the subclasses of A, due to dynamic binding (the exact m called in f depends on the type of actual type of o ).

29

class E { f(x) { A o = FactoryA(x ) ; ... o.m() }}

Testing more complicated interaction

• The way f(x) interacts with o is more complicated. There may be various combinations of which m and n are actually called, leading to complex overall behavior.

• Let’s extend our idea of integration test, to also cover such interaction.

30

class E { f(x) { A o = FactoryA(x ) ; // (1) o.m() ; // (2) ... o.n() } // (3)

Terminologies

• f itself is here called the coupling method.• The field o.x is “indirectly” defined (i-def) if the field is

updated. Analogously, use and i-use. • Focus on coupling within f due to variables o.v (indirectly) def.

by o.m(), and used by o.n(). antecedent and consequent.

31

f(x) { (1) A o = FactoryA(x ) ; (2) o.m() ; ... (3) o.n() }

Terminologies

• Coupling sequence C: a pair of antecedent m, and consequence n, such that there is at least 1x def-clear path from m to n with respect to some field o.x defined in m and used in n.

• o.x is called a coupling var (of C)• The set of all coupling vars of C coupling set.

32

f(x) { (1) A o = FactoryA(x ) ; (2) o.m() ; ... (3) o.n() }

33

With polymorphism the situation will now look like this...

Notice how different types of the context o may give different coupling variables as well as coupling paths.

Binding triple

• Each coupling sequence induces one or more binding “triples”; each specifying the coupling variables for various types of the context-var o. For example, this could be the binding triples of sequence o.m() o.n()

34

A

B

C

type of o coupling vars

A { o.v }

B { o.u }

C { o.u , o.v }

binding triples of c = o.m() o.n()

Inter-class coverage test criteria

• (Def 7.53) All-Coupling-Sequences (ACS): for every coupling sequence c in the coupling method f , TR includes at least one coupling path of c. ignore polymorphism.

• (Def 7.54) All-Poly-Classes (APC): for every binding triple t of c, TR includes at least one coupling path of t. ignore that c may have multiple coupling vars. 35

A

B

C

type(o) coupling vars coupling paths

A { o.v } o.v : { p1 }

B { o.u } o.u : { q1, q2 }

C { o.u , o.v } o.u : { r1 } , o.v :{r2}

binding triples of c = o.m() o.n() , extended with infos over coupling paths.

Inter-class coverage test criteria

• (Def 7.55) All-Coupling-Defs-Uses (ACDU): for every coupling var v of c, TR includes at least one of v’s coupling path.

• (Def 7.56) All-Poly-Coupling-Defs-Uses (APCDU): for every coupling var v of every binding triple of c, TR includes at least one of v’s coupling path.

36

A

B

C

type(o) coupling vars coupling paths

A { o.v } o.v : { p1 }

B { o.u } o.u : { q1, q2 }

C { o.u } o.u : { r1 }