A Formal Analysis of MCDC and RCDC

Test Criteria

Kalpesh Kapoor Jonathan P. Bowen

Centre for Applied Formal Methods

Faculty of BCIM, 103 Borough Road

London South Bank University,

London SE1 0AA. UK

{kapoork, bowenjp }@lsbu.ac.uk

Abstract

Modified Condition Decision Coverage (MCDC) test criterion is a

mandatory requirement for the testing of avionics software as per the

DO-178B standard. This paper presents a formal analysis for the three

different forms of MCDC. In addition, a recently proposed test criterion,

Reinforced Condition Decision Coverage (RCDC), has also been inves-

tigated in comparison with MCDC. In contrast with the earlier analysis

approaches that have been based on empirical and probabilistic models,

the principles of Boolean logic are used here to study the fault detection

effectiveness of MCDC and RCDC criteria. Based on the properties of

Boolean specifications, the analysis identifies the detection conditions for

six kinds of faults. The results allow measurement of effort required in

testing and the effectiveness of generated test sets satisfying the MCDC

and RCDC criteria.

Keywords: Structural coverage analysis, Control-flow test criteria, Formal

analysis, MCDC, RCDC, Boolean specifications

1

1 Introduction

In order to be certified by DO-178B standard [1] of the Federal Aviation Ad-

ministration, avionic software must be tested using Modified Condition Deci-

sion Coverage (MCDC), a control-flow testing criterion. A test criterion is a set

of rules that prescribe some property for the test sets; it is used to determine

whether the implementation has been thoroughly tested by the test set with re-

spect to the specification [2].

Test criteria can be broadly classified into data-flow and control-flow cate-

gories. Data flow criteria delineate rules for test sets that are based on data def-

initions and their usage in the implementation. Control-flow test criteria [3, 4]

are meant to challenge the decisions that are present in an implementation with

the test sets based on the structure and logic of the design.

The MCDC control-flow criterion has remained one of the most contro-

versial testing method in recent years with regard to its cost and fault detection

effectiveness [5–7]. The objective of this paper is to shed light on the fault detec-

tion effectiveness of MCDC using mathematical reasoning. The formal analysis

presented here can also be applied in evaluating the fault detection effectiveness

of MCDC test sets based on the properties of the corresponding specification.

In particular, the objectives of this paper are to:

• give mathematical definitions for three different forms of MCDC viz.

Unique-Cause, Unique-Cause Masking (strong and weak interpretations)

and Masking;

• find the fault sensitivity of MCDC test sets with respect to different types

of faults that can occur in Boolean decisions;

• evaluate the cost and fault detection effectiveness of MCDC test sets with

respect to different classes of Boolean decisions;

2

• investigate and compare a new test criterion, Reinforced Condition Deci-

sion Coverage (RCDC) [8], proposed as an improvement of MCDC.

The remainder of the paper is organised as follows. Sections 2 and 3 de-

scribe the theoretical background, and MCDC, respectively. A synopsis of re-

lated work is given in section 4. Section 5 gives the formal framework and

definitions of the test criteria. The analysis and results are presented in section

6. Finally, section 7 gives the conclusions and possible directions for future

work.

2 Theoretical Background

The main focus of this article is to investigate the control-flow test criteria that

are intrinsically designed to check the logical decisions made by the implemen-

tation based on a corresponding specification.

The logical decisions are Boolean formulas defined on a two-valuedBoolean

algebra, B, with ‘∨’ ( logical or) and ‘∧’ ( logical and) as binary operators, ‘¬’

(logical not) as unary operator, and0 (false) and1 (true) as its two elements [9].

The set ofBoolean formulason then symbolsx1, x2, . . . , xn can be recursively

defined as follows: the symbolsx1, x2, . . . , xn are Boolean formulas; ifG and

H are Boolean formulas, then so are(G) ∨ (H), (G) ∧ (H) and¬(G); a string

is a Boolean formula only if it can be obtained from a finite number of applica-

tions of the earlier rules. In the rest of this paper, formula and decision are used

interchangeably to mean the same thing.

The set of atomic formulas (i.e.,{x1, x2, . . . , xn}) may be represented by

the logical (atomic) variables or relational expressions; however, for the purpose

of this paper they are assumed to be abstract entities. The symbolsx1, x2, . . . , xn

areinputs. Every occurrence of an input in a given formula is acondition. For

example, there are four conditions in the Boolean decisionD ≡ (x1 ∧ x2) ∨(¬x1 ∧ x3), viz., x2, x3, first x1 and the secondx1. The number of inputs inD

3

is three. This conforms to the DO-178B standard [1] which considers inputs and

conditions as distinct entities. A formula can also be viewed as anexpression

tree in which internal nodes and leaves correspond to the logical operators and

conditions, respectively.

The semantics of a Boolean formula is given by a Boolean function. A

Boolean formula is a representation of a function,f : Bn → B, whereB =

{0, 1}. In other words, ann-input functionf : Bn → B is called aBoolean

functionif and only if it can be expressed by a Boolean formula. Formally, then-

input Boolean functions onB, can be defined recursively using base definitions

of constant, projectionandcompositionfunctions [9]. For a given number of

inputs,n, the number of possible Boolean functions is22nwhereas the number

of Boolean formulas is infinite. Throughout this paper, capital letters will be

used to denote formulas (such asF ) and small letter (f ) to denote the Boolean

function represented by the corresponding formula (F ).

Given a Boolean formula,F , on variables{x1, . . . , xn}, an input,xi, is said

to beredundantif there exists a formula equivalent toF that does not include

xi; otherwise the variablexi is said to beirredundant[9, 10]. For example,x2

is redundant in formulaF ≡ x1 ∨ x1 ∧ x2, asF can be rewritten asx1.

Let P be an implementation andS be the specification thatP is intended to

satisfy. An implementation,P , is said to be correct on vectorv if P (v) = S(v),

whereP (v) denotes the value computed byP on vectorv, andS(v) denotes the

intended output forv. Otherwise,P is said tofail, with v as thefailure-causing

vector. Afailure is the inability of a system or a system component to perform

a function as dictated by the specification. Afault in an implementation is a

defect which can result in a failure. In other words, a fault is local incorrect

(computational or control) operation that, when propagated, results in a failure

[11].

A test caseis a vector. Atest setis a set of test cases. If a test case is a

failure-causing vector for an implementation, then the implementation is said to

4

fail on that test case and such a test case is said to expose a fault.

In this paper, both the specification and its implementation are considered

to be Boolean logic formulas. The results of this paper can be applied to two

possible scenarios (of control-flow testing) described below.

Program Testing: The Boolean decisions appear as control points in a pro-

gram. When the test sets are generated from the implemented program,

the objective is to distiguish the given program from a hypothetically cor-

rect implementation [11–13]. Letl be a location in the program where a

fault is hypothesised. In order to detect the fault at locationl, the three

conditions [11, 14, 15] that need to be fulfilled are: (a) execution of the

statement at the locationl (reachability); (b) the execution of the state-

ments at locationl in the implemented and intended programs must result

in different values at least once (infection); (c) propagation of the dif-

ference in state to result in a failure. Note that the infection condition

is referred asoriginal state failure conditionin [11], which consists of

origination conditionandcomputational transfer conditions.

In control-flow testing, reachability is not an issue, however, the test set

must be able to infect and propagate the erroneous state. In this context,

the analysis presented in this paper is concerned with the infection aspect

and investigates if it is guaranteed that infection will occur or not. Since

the control-flow test criteria do not prescribe any rule for propagation, it

will depend on the specifics of any given program and is beyond the scope

of this paper.

Specification-based Testing:In this case, the test sets are generated from the

specification and are used for testing the control logic of the implemen-

tation [16]. It is assumed that the level of abstraction of the specification

and its corresponding implementation is the same. This is often the case

with embedded applications. Such an approach that includes MCDC as

5

a test generator method has been considered in [17–19]. The application

of full predicate coveragecriterion, which is similar to MCDC, has been

investigated in [20].

In this context, both the specification and its implementation can be con-

sidered as Boolean formulas. For example, in [21], various coverage cri-

teria including MCDC are applied to Boolean formulas obtained after

flattening the specification.

For a given Boolean specification, theDecision Coverage(DC) test criterion

requires that the test set must include test cases that evaluate to0 and1 values.

TheFull Predicate Coverage(FPC) test criterion improves over DC by taking

into account values of both the decision and the inputs. It requires that each

input in a decision has taken all possible outcomes where the value of a decision

is directly correlated with the value of an input [22]. This means that test set

must include test cases such that the value of decision is different when the input

is changed.

Although FPC considers inputs that appear in a Boolean decision, it ignores

the direct dependency and influence of the conditions on the decision. The

MCDC and RCDC test criteria improve on this drawback of FPC test criterion

and are described in detail in the following section.

3 MCDC and RCDC

The DO-178B standard [1] for the testing of software recommends structural

coverage analysis. The purpose of such analysis is to determine which structure

in an implementation is not exercised by the requirements-based test procedures.

The Modified Condition Decision Coverage(MCDC) [5–7, 23–28], which

is a mandatory requirement for testing avionics software [1], has been defined

as follows:

6

Every point of entry and exit in the program has been invoked at

least once, every condition in a decision in the program has taken

on all possible outcomes at least once, and each condition has been

shown to independently affect the decision’s outcome.

The rule for entry and exit point is common to all the control-flow testing

criteria. It is the last line of the definition that makes MCDC a unique test cri-

terion. Two important aspects are “each condition” and its “independent effect”

on the output (decision value). As mentioned earlier, a condition is defined as a

leaf node in the Boolean expression tree considering multiple occurrences of an

input as different.

A condition is shown to independently affect a decision’s outcome by vary-

ing just that condition while holding fixed all other possible conditions. The

above definition requires at least one pair for every condition. However, it is not

always possible to apply this rule, for example, in the case of multiple occur-

rences of an input in the Boolean formula. The standard does not specify the

alternative in such a situation [24].

A situation in which a change in one condition also changes some other con-

dition is called conditioncoupling. If a change in one condition always results

in a change of other condition(s), it is said to bestrongcoupling; otherwise it is

said to beweakcoupling. For example, in decisionx1 ∧ x2 ∨¬x1 ∧ x3, the first

and second occurrence ofx1 are strongly coupled. On the other hand, the con-

ditions(i > 50) and(i < 100) are weakly coupled because varyingi from 49

to 55 changes only the first condition but not the second, while changingi from

49 to 101 changes both the conditions. As mentioned in [23], weak coupling

does not cause problems in practice. Therefore, strong coupling will be referred

to as simply coupling in the rest of the paper.

In order to make MCDC applicable to a wider range of specifications and

implementations, Chilenski in [24] has elaborated three different forms of MCDC

in detail. These are summarised as follows:

7

Unique-Cause (UC): This is same as the definition of MCDC given earlier in

this section, where each condition independently affects the final outcome. It is

therefore only applicable to Boolean decisions with uncoupled conditions.

Unique-Cause Masking [23]: This extends the Unique-Cause definition by

handling the coupling of conditions and allows them to change simultaneously

while keeping the other uncoupled conditions fixed. This can be achieved in the

following two ways:

• Weak (UCMW) : This treats the strongly coupled conditions as a single

condition. Therefore for the decisionD ≡ (x1 ∧ x2) ∨ (¬x1 ∧ x3), the

two occurrences ofx1 will be treated as one and either of the two vector

pairsp1 ≡ {(1, 1, 0), (0, 1, 0)} or p2 ≡ {(1, 0, 1), (0, 0, 1)} can satisfy

the rule for the conditionx1 (the ith bit in a vector corresponds to the

value of inputxi). With this view, it is sufficient to know the Boolean

function represented by the formula with no information about its struc-

ture. Therefore, the test set may be identical for all Boolean formulas

representing the same Boolean function.

• Strong (UCMS): This allows coupled conditions to change but it must be

shown that it is only the condition of interest that influences the output.

With this view, it is not possible to consider the decision as black box and

therefore it requires the knowledge about the structure of the formula. For

example, onlyp1 shows the independent effect of the first occurrence of

x1 in D. Similarly, the vector pairp2 shows the independent effect of the

second occurrence ofx1 in the decisionD. Therefore, bothp1 andp2 are

necessary to satisfy the strong criterion while only one of them is required

for the weak criterion.

Masking: This is the flexible form among all the interpretations of MCDC

and allows other (coupled or uncoupled) conditions to change while showing

8

the independent effect of a condition. Consider for example the Boolean de-

cision x1 ∨ (x2 ∧ x3). To show the independent effect ofx1, the Masking

definition only requires the sub-expression(x2 ∧ x3) to have no impact on the

final outcome. This can be achieved if the sub-expression is false; thus the

pair{(1, 0, 1), (0, 1, 0)} satisfies the Masking MCDC criterion, although all the

other corresponding input bits are different. Note that this is not allowed in

UCMS according to which onlyx1 is allowed to change and,x2 andx3 must

remain fixed in the test pair; whereas Masking requires only the sub-expression

x2 ∧ x3 to remain fixed. Masking MCDC can be used when the other stronger

rules are not applicable due to infeasible combinations of the conditions in a

specification.

The MCDC criterion takes into account only those situations where an in-

dependent change in the condition causes a change in the value of a decision.

In [8], a new criterion,Reinforced Condition Decision Coverage(RCDC), has

been proposed as a possible improvement of (UCMW) MCDC. [8] mentions a

shortcoming of MCDC approach because it does not check the situations where

a change in a condition should keep the value of a decision. It suggests, in ad-

dition to MCDC, the test set must show that every condition independently (if

possible)keepsthe value of a decision to both0 and1.

4 Related Work

The effectiveness of test criteria for Boolean specifications has also been studied

in [10,29–32].

The MCDC test criteria has been discussed in several papers; see for exam-

ple [5, 23, 26–28]. Chilenski and Miller, in [23], have presented a detailed de-

scription of the UCMS criterion. In [6], the empirical evaluation of MCDC has

been performed for the testing of an attitude control software for the High En-

ergy Transient Explorer scientific satellite. The empirical evaluation [6] found

9

MCDC to be effective in detecting faults at almost the same cost as that of other

techniques. Algorithms for generating MCDC test sets have been elaborated

in [25,33]. An algorithm to reduce the size of an MCDC test-suite is presented

in [26].

Based on a probability model, a formal analysis of MCDC has been pre-

sented in [28]. The model assumes all outputs of a Boolean function to be

incorrect with a fixed probability. As the specifications and implementations

are represented by the Boolean formulas, the model is not directly applicable to

reasoning about fault sensitivity. The other limitation of using the probability

model for functions is that it cannot be used to identify the class of formulas

that are easier (or more difficult) to test. Further, the analysis suggests the size

of test set as a key factor influencing effectiveness; therefore, it does not point

out the properties that make the MCDC test set better as compared to a random

test set of the same size.

5 Formal Framework

One of the goals that control-flow criteria aim to achieve is to minimize the

number of test cases in a test set while keeping effectiveness as high as possible.

Let S be a specification andF ≡ {S1, . . . , Sn} be a set of possible faulty

implementations with respect toS. Theeffectivenessof a test set,T , satisfying

a given test criterion, is defined to be the percentage of faulty implementations

in F that it can distinguish to be different fromS. Such a measure is used in

fault-based testing [34], in particular mutation testing [12].

In order to completely verify a Boolean formula withn inputs,2n test cases

are required. Therefore, it is computationally infeasible to test all possible com-

binations as the number of inputs and decisions grow in an implementation [21].

The existing criteria attempt to provide a practical solution that only requires a

linear number of test cases in terms of the number of inputs.

10

The following are definitions that will be used to define MCDC and RCDC

criteria and to study their properties. The definition stated below characterises

themeaningful impactproperty defined in [32].

Definition 1 [Input Sensitive Vector]. Let f be ann-input Boolean function

defined on variables{x1, . . . , xn}. A n-bit vector,v, is said to be input sensitive

with respect to a variablexi if f(v) 6= f(vi), otherwisev is said to be insensitive

with respect toxi i.e., whenf(v) = f(vi); wherevi is vectorv with its ith bit

flipped.

Consider for example the Boolean formulaF = x1 ∨ x2 ∧ x3 for which the

vectorv = (0, 1, 0) is sensitive with respect to the variablesx1 andx3 since

the value off(0, 1, 0) is not equal to eitherf(1, 1, 0) or f(0, 1, 1). Further,v

is insensitive forF , with respect to the variablex2. Note that a vector may be

sensitive with respect to zero or more number of variables.

Definition 2 [Neighbours]. Letv1 andv2 be two n-bit vectors.v1 is said to be

a neighbour ofv2 w.r.t. xi if v1 = vi2 for a uniquei, wherevi

2 is the vectorv2

with its ith bit flipped. The set of all (unordered) pairs that are neighbours w.r.t.

an inputxi, will be denoted byηi and its size is2n−1 for any Boolean function

with n inputs.

Definition 3 [Input Sensitive Neighbours]. An unordered pair, (v1, v2), of

vectors are said to be input sensitive neighbours for a Boolean function,f , w.r.t.

an inputxi, if v1 andv2 are neighbours and input sensitive vectors w.r.t.xi. The

set of all input sensitive neighbour pairs w.r.t.xi is denoted byψi.

The size of the input sensitive neighbour set,|ψi|, is a measure of theinflu-

ence[35] of an input on the Boolean function. The influence of variablexi (i.e.,

|ψi|) can be used to identify equivalence classes of Boolean functions based on

the effort required to test them. The effort is defined as the minimum number of

test cases that are required to identify the pre-specified faults. The influence of

an input can vary from0 to the maximum of2n−1.

11

Definition 4 [Weak and Strong Boolean Functions].A Boolean function inn

irredundant inputs is said to be:

• Weak: if the influence of every input is linear in terms ofn;

• Strong: if the influence of at least one input is maximum, i.e.,2n−1.

A Boolean function is said to be balanced if it evaluates to1 (and0) for half

of the elements in its domain [35, 36]. Note that strong Boolean functions are

balanced. In [36], it is shown that a class of balanced functions (referred ask-

balanced) will at least require exponential number of conditions. One example

of such a function is a parity function in three inputs:¬x1 ∧ (x2 ∧ ¬x3 ∨ x3 ∧¬x2) ∨ x1 ∧ (x2 ∧ x3 ∨ ¬x2 ∧ ¬x3). On the other hand, the Boolean functions

x1 ∨ x2 ∨ x3 andx1 ∧ x2 ∧ x3 are weak.

A Boolean function,f(x1, . . . , xn) can be expressed asxi ∧ f |xi=1 ∨¬xi ∧f |xi=0, wheref |xi=b is the formula obtained by replacing variablexi with the

formula b. This is also known asShannon’s expansion. The set,ψi, can be

generated by finding all the vectors that satisfy the Boolean equationf |xi=0 ⊕f |xi=1 = 1, where⊕ is theexclusive–oroperator.

Proposition 1. If an input, xi, is redundant in a given Boolean formula then

|ψi| = 0.

Proof. If an input has no influence on the Boolean function, thenf |xi=0 must be

equal tof |xi=1 in its Shannon’s expansion. Therefore, if an input is redundant,

there is no vector that satisfies the Boolean equationf |xi=0 ⊕ f |xi=1 = 1.

Similarly, for a given Boolean function and an input, an insensitive neigh-

bour pair set can also be defined. The vectors in an insensitive pair either eval-

uate to0 or to1.

Definition 5 [Insensitive Pair Sets]. Let xi be an input for the Boolean func-

tion, f , then the input insensitive sets with respect toxi are defined as follows:

12

a. Input 0-Insensitive Set:

ν0i = {(v1, v2) | v1[i] 6= v2[i] ∧ f(v1) = f(v2) = 0}

b. Input 0-Insensitive Neighbour Set:

µ0i = {(v1, v2) | (v1, v2) ∈ ν0

i ∧ v1 = vi2}

c. Input 1-Insensitive Set:

ν1i = {(v1, v2) | v1[i] 6= v2[i] ∧ f(v1) = f(v2) = 1}

d. Input 1-Insensitive Neighbour Set:

µ1i = {(v1, v2) | (v1, v2) ∈ ν1

i ∧ v1 = vi2}

In the above definitions,v1 and v2 are n-bit vectors and a pair of vectors is

unordered.

For a given number of inputs, the neighbour set is fixed and is partitioned

by the input sensitive and insensitive neighbour sets. The following proposition

illustrates that with the increase in the influence of an input, the number of

insensitive neighbours decreases.

Proposition 2.

a. For any Boolean function,ηi = ψi ∪ µ0i ∪ µ1

i and the setsψi, µ0i , µ1

i are

mutually disjoint.

b. For any strong Boolean function, there exists at least one input for which

the two setsψi andηi are equal and the setsµ0i andµ1

i are empty.

c. There exists at least one variable for which the size ofψi is exponential

for strong Boolean functions.

d. The size of eitherµ0i or µ1

i is exponential for weak functions.

Proof.

13

a. Letp ∈ ηi be a pair of neighbours thenp must be in exactly one of theψi,

µ0i or µ1

i sets. Thus the proposition follows.

b. For a strong Boolean function, at least one variable has maximum influ-

ence (Definition 4). Using part (a),ηi = ψi and the setsµ0i andµ1

i are

empty.

Parts (c) and (d) follow from (a) and noting that, for a given a Boolean function

with n inputs, the size ofηi (i.e., the number of neighbours) is2n−1 with respect

to an input.

A variable can appear more than once in a Boolean formula. The notationxki

denotes thekth occurrence of a variablexi in a formula, and⊆ and⊂ represent

a subformula and a proper subformula, respectively.

Definition 6 [Maximal Subformula]. Let F be a Boolean formula. A sub-

formula,Maxki ⊆ F , is said to be the maximal subformula with respect to a

conditionxki if any other subformula ofF that exists and that includesMaxk

i

also includes an occurrence ofxri (r 6= k).

As mentioned earlier, coupling occurs either due to multiple occurrences of

inputs or restrictions imposed by the specification. A group of coupled condi-

tions can be identified by a representative input in the Boolean formula. Using

the above conventions and definitions, a condition sensitive pair set can be de-

fined as follows.

Definition 7 [Condition Sensitive Vector]. Letxki be thekth occurrence of an

input xi in the Boolean formulaF . Then a vector,v, is said to be condition

sensitive w.r.t.xki if the following holds:

a. for all formulasF ′ ⊇ Maxki , v is input sensitive w.r.t.xi for f ′.

b. if v is input sensitive w.r.t.xi for Maxji for somej 6= k then there exists

F ′ ⊃ Maxji such thatv is input insensitive forf ′.

14

Consider for example the Boolean formulaF = x1 ∧ x2 ∨ ¬x1 ∧ x3. The

vectorv = (1, 1, 0) is condition sensitive with respect to the first occurrence of

x1. Note that in the vectorv, the second occurrence ofx1 is input insensitive in

the corresponding subformula¬x1 ∧ x3.

Definition 8 [Condition Sensitive Neighbour Set].Let xki be thekth occur-

rence of an inputxi in the Boolean formulaF . Then the condition sensitive

neighbour set is defined as:

Ψki = {(v1, v2) | (v1, v2) ∈ ψi ∧ v1 is condition sensitive with respect toxk

i }

In the above definition, the set restriction predicate ensures that it is the con-

dition of interest that independently causes the change in the final output. The

two definitions of condition, and input sensitive neighbour sets are equivalent if

all the conditions are uncoupled (hence coinciding with the inputs).

If for a given condition in a given Boolean formula, no condition sensitive

vector exists, then either the input will be redundant or a shorter formula can be

written with fewer number of occurrences of that input.

The definitions of test sets that include the test vectors which are sensitive

to all inputs and conditions respectively, are given below.

Definition 9 [Independent Test Set].Let F be a Boolean formula. A test set,

T , is said to satisfy:

a. Input-Independence Property: if ∀i ∃t1, t2 ∈ T | (t1, t2) ∈ ψi

b. Condition-Independence Property: if ∀i, j ∃t1, t2 ∈ T | (t1, t2) ∈ Ψji

In other words, input and condition independence test sets include at least

one sensitive neighbour for every input and condition, respectively.

Proposition 3. It is not possible to generate an independent test set for a Boolean

formula with redundant inputs.

15

Proof. If an input is redundant, there is no vector pair that can show the in-

dependent effect of this input on the output (see Proposition 1); therefore an

independence test set cannot be generated.

Using the above definitions, one can formally define RCDC and different

forms of MCDC test criteria.

Definition 10 [Unique-Cause (UC) MCDC]. A test set is said to satisfy UC

MCDC for a Boolean formula, if it satisfies the input-independence property

and all the conditions that appear in the formula are uncoupled.

The UC MCDC definition does not allow multiple occurrences of an input

in the Boolean formula, as that will also cause condition coupling. As men-

tioned earlier, coupling occurs either due to the multiple occurrences of inputs,

or restrictions imposed by the specification.

As noted earlier, a group of coupled conditions is identified by a representa-

tive input in the Boolean formula. Using this convention, the two variations of

unique-cause masking MCDC can be defined as follows:

Definition 11 [Unique-Cause Masking (UCM) MCDC]. A test set is said

to satisfy UCM Weak MCDC for a Boolean formula, if it satisfies the input-

independence property; on the other hand if it satisfies the condition indepen-

dence property it is known as the UCM Strong MCDC test set.

The weak interpretation in the above definition has a drawback since in the

case of multiple occurrences of an input in the formula, only one of them is

shown to independently influence the outcome. Because of this, a test set sat-

isfying the weak criterion may also be used for other semantically different

Boolean formulas. In that case, the test case will not be able to distinguish a

possibly large class of Boolean functions. In the rest of this paper, UCM weak

and UCM strong will be referred as UCMW and UCMS, respectively.

16

The following definition of masking MCDC involves only sensitive vectors,

because while showing the independent effect of a condition it is not required to

keep other conditions fixed; in other words, the test vectors need not be neigh-

bours.

Definition 12 [Masking MCDC]. A test set,T , is said to satisfy masking

MCDC for a Boolean formula,F , if for every conditionxji in F , there ex-

ist vectorsv1, v2 ∈ T that are condition sensitive w.r.t.xji , v1[i] 6= v2[i] and

f(v1) 6= f(v2), wherev[i] gives theith bit of vectorv.

As mentioned before, a RCDC [8] test set also checks the correctness of

insensitive neighbours. The following definition captures this idea behind the

RCDC approach. The definition of RCDC extends the UCMW criterion by

including the insensitive vectors.

Definition 13 [RCDC]. Let choice be a function defined as follows,

choice(α, β) =

α , if α 6= ∅β , otherwise

whereα andβ are two sets. Then a test set,T , satisfies RCDC test criterion for

a Boolean function,f , if the following holds:

a. T satisfies input-independence property

b. ∀i · choice(µ0i , ν

0i ) 6= ∅ ⇒ ∃t1, t2 ∈ T · (t1, t2) ∈ choice(µ0

i , ν0i )

c. ∀i · choice(µ1i , ν

1i ) 6= ∅ ⇒ ∃t1, t2 ∈ T · (t1, t2) ∈ choice(µ1

i , ν1i )

In [8], it is proved that it is always possible to choose a RCDC test set by

showing that the conditionchoice(µ0i , ν

0i ) 6= ∅ ∨ choice(µ1

i , ν1i ) 6= ∅ always

holds. It is also noted in [8] that, for certain Boolean formulas, one of the sets

choice(µ0i , ν

0i ) or choice(µ1

i , ν1i ) can be empty. Apart from these observations

in [8], the analysis done in this paper shows that bothµ0i andµ1

i can be empty

at the same time (cf. Proposition 2(b)).

17

Lemma 1. Letf be ann input Boolean function such that∀i·|ψi| = 1. Further,

let f−1(0) andf−1(1) be the sets of vectors that evaluate to0 and1, respec-

tively, forf . Then,|f−1(0)| = 1 ∨ |f−1(1)| = 1.

Proof. Without loss of generality, letm = |f−1(0)| 6= 1. Since|ψi| = 1 for all

inputs, the two vectors in a neighbour pair with respect to an input evaluate to the

same value (0 or1) except in one case (say pairp). The conditionm 6= 1 implies

that apart fromp at least one more neighbour pair evaluates to0 (sayv, v′).

However, bothv andv′ pair-up with different vectors to form neighbour pairs

with respect to some other input. Therefore, these other vectors must evaluate to

0 to keep the influence of that input to1. However, that would change the value

of m. Continuing this way, for thenth input, the size of|f−1(0)| will become

2n−1, which implies|f−1(1)| = 1.

Lemma 2. Let f be ann input Boolean function such that∀i · |ψi| = 1. Then,

µ1i = ∅ ∨ µ0

i = ∅.

Proof. The proof follows from Lemma 1 by observing that all insenstive neigh-

bours evaluate to the same value.

Based on the above definitions, the following theorem gives the properties

about the size and the number of possible test sets that can be generated for

MCDC and RCDC test criteria.

Theorem 1. Let n be the number of inputs in a Boolean formula. Further,

assume that a test set includes exactly one sensitive neighbour pair for every

input or condition (whichever is appropriate as per the definition); then the

variation in the number of test cases in a test set is as shown in Table I(a). Also,

the number of possible test sets can vary as given in Table I(b).

Proof.

18

CriteriaSize of test set

Minimum Maximum

UC n + 1 2× n

UCMW n + 1 2× n

UCMS n + 1 2n

Masking n + 1 2n

RCDC n + 1 6× n

(a)

CriteriaNumber of possible test sets

Minimum Maximum

MCDC 1 2n(n−1)

RCDC 1 (2n−1 − 1)n

(b)

Table I: Table for Theorem 1

a. In the case of UC and UCMW, the size of the test set depends on the

number of inputs. A test set satisfying these interpretations must have a

sensitive neighbour pair for every input, thus requiring a maximum of2×n test cases. Considering the possibility where a vector from a sensitive

neighbour pair is shared with another pair, in which case a minimum of

n + 1 vectors will be required to obtain the test set.

The number of test cases in a test set satisfying UCMS or Masking MCDC

is decided by the number of conditions in the given Boolean formula.

19

Since the number of conditions in a formula can be exponential [36], the

number of test cases required will be2n. An example of such a function

is the parity function. On the other hand, for the Boolean formulas with

single occurrence of inputs, the number of test cases in a UCMS or Mask-

ing test set will ben + 1 (assuming a vector from a sensitive neighbour

pair is shared with another pair).

As proved in the Proposition 2 (b), for strong Boolean functions there are

no insensitive neighbours, and thus to generate a RCDC test set, theν0i

andν1i sets will be used instead ofµ0

i andµ1i , in which case the vectors

satisfying UCMW may also satisfy RCDC. On the other hand, when the

setsψi, µ0i andµ1

i are non-empty, six test cases for every input may be re-

quired (two each for sensitive, 0-insensitive, and 1-insensitive neighbour

vectors).

b. For a given Boolean function, there may be more than one test set sat-

isfying a test criterion depending on the type of Boolean function. All

possible MCDC test sets can be generated by taking the cross product of

sensitive neighbour sets. Similarly, all possible RCDC test sets can be

generated by including the insensitive neighbour sets in the above cross

product. Consider the following two cases:

Case 1 For every input,xi, |ψi| = 1. In this case, the number of possible

MCDC test sets will be one. Using Lemma 1 and 2, computing the

number of possible RCDC test sets by considering (without loss of

generality)|µ0i | = 2n−1 − 1, µ1

i = ν1i = ∅. Hence total number of

test sets forn inputs is(2n−1 − 1)n.

Case 2 For every input,xi, |ψi| = 2n−1. In this case, the number of

possible MCDC test sets will be2n(n−1). In this case, a RCDC test

set can be generated by using the vectors from theνi sets. Thus, the

20

result follows as shown in Table I(b).

6 Fault Sensitivity Analysis

The approach used for analysing fault sensitivity is based on the idea of fault-

based testing, which assumes that the intended implementation closely matches

the actual implementation [34] (also known as thecompetent programmer hy-

pothesis). The following types of fault can be hypothesised:

Operator Faults: These occur due to incorrect logical operators, and can be

classified as follows:

a. Operator Reference Fault (ORF): ‘∨’ replaced by ‘∧’ or vice versa, e.g.,

x1 ∧ x2 by x1 ∨ x2.

b. Expression Negation Fault (ENF): A subformula replaced by its negation,

for example,x1 ∧ (x2 ∨ x3) implemented asx1 ∧ ¬(x2 ∨ x3).

c. Variable Negation Fault (VNF): One of the conditions is replaced by its

negation in the formula. If the condition appears in negated form, then

missing the negation also causes VNF.

d. Associative Shift Fault (ASF): Incorrect implementation due to misunder-

standing about operator evaluation priorities, and is caused by missing the

brackets. For example,x1∧ (x2∨x3) if implemented asx1∧x2∨x3 will

mean(x1 ∧ x2) ∨ x3, since ‘∧’ has higher priority as compared to ‘∨’.

Operand Faults: These faults involve the conditions that appear in a Boolean

formula, and are of two types:

21

a. Missing Variable Fault (MVF): Absence of a condition in the formula,

e.g.,x1 ∨ x2 implemented asx1.

b. Variable Reference Fault (VRF): A condition is replaced by another input

that appears in the formula, for example,x1∧x2∨¬x1∧x3 implemented

asx1 ∧ x2 ∨ ¬x2 ∧ x3.

Note that in the context of variable related faults, the definition applies to a

condition which is an occurrence of an input in the Boolean formula. Therefore,

it is not sufficient to only consider the Boolean function represented by the for-

mula, but the structure of the Boolean formula must also be taken into account.

However, the analysis presented in this paper does not impose any restriction on

the structure of a formula, and hence is generic.

For the purpose of analysis, it is assumed that all the inputs in a given

Boolean specification are irredundant and the implementation has at most one

fault of the above-mentioned type. The first assumption is justified since the

presence of redundant inputs is an issue of concern with regard to the validity

of the given Boolean specification. Also, as proved in the Proposition 3, it is

not possible to test a specification with respect to the above types of fault, in

the presence of redundant inputs. The validation of the second assumption re-

quires analysis in the presence of multiple faults (known as thefault coupling

effect[37]) and is beyond the scope of this paper.

The above types of fault have also been studied in other research papers

[32, 38–40] in the context of fault-based testing for Boolean specifications. It

may be observed that the faults can be viewed at both syntactic and semantic

levels. Since the test set is generated from the specification, a misunderstanding

about the logic, or under-specification may result in one of the above types of

faults.

A fault-based testing approach alleviates the problem of measuring the relia-

bility of a test set in terms of fault detection. In the absence of information about

22

the fault distribution or fault types, Theorem 2 below states that there cannot be

any testing technique with a good guarantee of detecting faults for an arbitrary

Boolean specification and Boolean implementation; this further explains why a

fault-based testing approach is more appropriate.

Theorem 2. LetS andP be the specification and implementation of a Boolean

function respectively, considered as a black box withn inputs and one output;

i.e., it is only allowed to observe the inputs and the output. If forr(≈ n) vectors

the output ofP differs from that ofS, then the probability of not being able to

detectP to be different fromS using a random test set of sizen approaches 1

asn increases.

Proof. The implementationP can be found different fromS using a random

test set if the test set includes at least one of the vectors for which the output is

different. The probability,pr, that a random test set does not include any such

vector, given by the following formula, proves the above assertion (for instance,

if n = r = 9 thenpr = 0.85).

pr =No. of test sets that does not detect any fault

Total no. of possible test sets=

(2n−r

n

)(2n

n

) =

(2n−n

r

)(2n

r

)

The validity of the above theorem depends on the type of faulty implemen-

tation. Therefore, the result does not hold if every output is equally likely to be

incorrect with certain probability. However, it is straightforward to show that

a fault in an implementation may influence only a linear-size sub-domain (for

example, faults that occur at the lower depths in a Boolean expression tree).

Consider for example, the Boolean formula,F ≡ (x1 ∨ x2) ∧ x3 ∧ x4 ∧ x5.

Assuming that there is a VNF forx1 (i.e., x1 is replaced by¬x1) then only 2

vectors out of 32 vectors can detect the fault. Therefore, a random test set is less

reliable for detecting faults in a Boolean implementation than other test criteria.

23

Note that for a given Boolean specification and implementation the effectiveness

of a random test set can vary from 0 to 100%.

As defined in the previous section, the MCDC and RCDC test criteria re-

quire sensitive and insensitive vectors respectively. In order to investigate effec-

tiveness with respect to fault sensitivity for the different forms of these criteria,

the fault detection capability of insensitive and sensitive vectors is evaluated

first.

Lemma 3 (Necessary and Sufficient Condition for detection of ASF).Let

F ≡ F1∧(F2∨F3) and the formula with ASF fault beF ′ ≡ F1∧F2∨F3. Then

the necessary and sufficient condition for a vectorv to detectF to be different

from F ′ is f1(v) = 0 andf3(v) = 1, which is also equivalent tof(v) = 0 and

f3(v) = 1.

Proof. Consider the Boolean formulaFxor = F ⊕ F ′, where⊕ is exclusive-or

operator. The formulasF andF ′ can be detected to be different by a vectorv

iff f(v) andf ′(v) evaluate to different values, i.e.,fxor(v) = 1. The formula

Fxor is equivalent to¬F1 ∧ F3. Thus,fxor(v) = 1 will require f1(v) = 0 and

f3(v) = 1, which is equivalent tof(v) = 0 andf3(v) = 1.

Theorem 3. Let F be a given Boolean specification withn inputs. Also, letv1

and v2 be two n-bit vectors such that the pair(v1, v2) ∈ µ0i ∪ µ1

i . Then the

vectorsv1 andv2 cannot detect the following:

a. VNF, VRF, MVF faults involving the conditionxki , if v1 andv2 are insen-

sitive w.r.t.xi in Maxki .

b. ASF in a subformulaFa ≡ F1∧ (F2∨F3) (i.e., changed toF1∧F2∨F3)

if v1 andv2 are input sensitive w.r.t.xi in Fa.

c. ENF in a subformulaFe (i.e., changed to¬Fe) if v1 and v2 are input

sensitive w.r.t.xi in Fe.

24

d. ORF in a subformulaFo ≡ Fl ◦Fr (i.e., changed toFl ◦′ Fr) if v1 andv2

are input sensitive w.r.t.xi in Fo.

Proof. As (v1, v2) ∈ µ0i ∪ µ1

i , v2 = vi1.

a. LetG = Maxki . Consider the Shannon expansion ofg with respect to

xi; i.e., g = xki ∧ g|xk

i =1 ∨ ¬xki ∧ g|xk

i =0. Note thatxi has only one

occurrence inG, thereforeg|xki =1 andg|xk

i =0 does not depend onxi. As

v1 is insensitive w.r.t.xki in G, g|xk

i =1(v1) = g|xki =0(v

i1).

Consider a VNF fault for a conditionxki in which case the Shannon expan-

sion for the modified functiong′ can be written asg′ = ¬xi∧g|xki =1∨xi∧

g|xki =0. Without loss of generality, letv1[i] = 1 theng′(v1) = g|xk

i =0(v1)

and g′(vi1) = g|xk

i =1(vi1). Sinceg|xk

i =1(v1) = g|xki =0(v

i1), v1 cannot

distinguishg from g′.

In the case of VRF fault, letxki be replaced withxj , wherexj appears in

the formulaF andxi 6= xj . The modified function,g′′, can be expressed

asxj ∧ g|xki =1 ∨ ¬xj ∧ g|xk

i =0. Here,g′′(v1) andg′′(vi1) will have either

valueg|xki =1(v1) or g|xk

i =0(v1), in either casev1 cannot detect a VRF

fault.

A similar proof holds by observing that a formula with MVF can be ex-

pressed asg′′′ = g|xki =1 ∨ g|xk

i =0.

As the vectorsv1 andv2 are insensitive inMaxki , the value atF remains

unchanged. Hence the faults will not be detected.

b. Using Lemma 3, to detect an ASF fault inFa, it must evaluate to0 for

a given vector. Letfa(v1) = 0, thenfa(v1) 6= fa(vi1) as v1 is input

sensitive inFa. However, asf(v1) = f(vi1) a fault inFa will not change

the value off .

c. & d. The reasoning for ENF and ORF faults is similar to that for case (b).

25

Corollary 1. The statements in Theorem 3(a) (i.e., for VNF, VRF and MVF)

also hold if(v1, v2) ∈ ν0i ∪ ν1

0 .

Proof. The proof is similar since all the arguments hold for insensitive vectors

that are not necessarily neighbours.

Note that in Theorem 3, the pair(v1, v2) can be from eitherµ0i or µ1

0 which

are used in forming RCDC test sets. Moreover, in the theorem ifv1 andv2

are insensitive inFa, Fe and Fo then the ASF, ENF and ORF, respectively,

may or may not get detected. For example, consider the Boolean formulaF ≡x1 ∨ x2 ∨ x3 and following insensitive vector pairs with respect tox2,

p1 ≡ {(1, 1, 1), (1, 0, 1)}p2 ≡ {(0, 1, 1), (0, 0, 1)}

ENF Let F ′ ≡ x1 ∨ ¬(x2 ∨ x3). The pairp1 cannot distinguishF from F ′,

whereasp2 can detectF to be different fromF ′.

ORF The same holds for a formula with an ORF fault:F ′′ ≡ x1 ∨ x2 ∧ x3.

Theorem 4. Let F be a given Boolean specification withn inputs. Also, letv1

andv2 be n-bit vectors such that: (i)v1[i] 6= v2[i], (ii) v1 andv2 are condition

sensitive w.r.t. a conditionxji and, (iii) f(v1) = 0 andf(v2) = 1. Then a test

set that includesv1, v2 will detect the following:

a. ENF in any subformula that hasxji .

b. VNF inxji .

c. VRF in whichxji is replaced by an inputxk (wherek 6= i) iff v1[i] 6= v1[k]

or v2[i] 6= v2[k].

d. MVF inxji .

26

e. ORF in a binary operator,‘◦’, at the root of any subformula,F ′ = F ′l ◦F ′

r,

wherexji appears either inF ′

l or F ′r.

f. ASF in any subformulaF ′ = F ′1 ∧ (F ′

2 ∨ F ′3) iff xj

i appears inF ′1 and

F ′3(v1) = 1.

Proof. Observe that a condition sensitive vector guarantees propagation of value

from the condition to the top of the expression tree and propagates through all

sub-expressions that includes the condition (cf. Definition 7(a)).

a. An ENF fault negates the value of a sub-expression in a formula. Since

the value of the variable is propagated, it will be inverted in the case of an

ENF fault, and therefore the fault will be detected.

b. A VNF fault will also be detected for similar reasons.

c. In the case of a VRF fault, the value of an input (xk) that is replaced with

the condition,xji , will be propagated and hence must be unequal to it for

detecting the fault.

d. An MVF fault can be simulated by replacing a condition with its sib-

ling subformula. Thus, in order to detect an MVF fault, the condition

v1[i] 6= sibji (v1) or v2[i] 6= sibj

i (v2) must hold, whereSibji is the sibling

subformula ofxji . This condition is always true forv1[i] 6= v2[i], sincev1

andv2 are insensitive inSibji , sibj

i (v1) = sibji (v2).

e. To detect an ORF fault inF ′ the conditionF ′l (v1) 6= F ′

r(v1) or F ′l (v2) 6=

F ′r(v2) must hold. This can be shown by considering the two possibilities

for the logical operator‘◦’. Without loss of generality, letxji appear inF ′

l .

As v1 andv2 are condition sensitive with respect toxji , F ′

l (v1) 6= F ′l (v2)

(asf(v1) 6= f(v2)) andF ′r(v1) = F ′

r(v2). Therefore, the above detection

condition holds.

27

f. In the case of an ASF fault, using Lemma 3, in addition tof ′3(v1) = 1, xji

must appear inF ′1 to guarantee thatf ′(v1) = 0.

A pair of vectors can either be sensitive or insensitive with respect to an

input. As shown in Theorem 3 and Corollary 1 earlier, insensitive vectors cannot

guarantee the detection of faults and therefore it is necessary to use sensitive

vectors for fault detection.

Observation 1. The detection condition for an ASF fault has restriction on the

subformula value apart from the sensitivity. According to Theorem 3(b), insen-

sitive vectors also do not guarantee detection of ASF. Therefore, none of the

considered criteria guarantee detection of ASF faults.

Corollary 2. In Theorem 4, a VRF fault will be detected unconditionally, if the

two vectors are also neighbours.

Proof. In addition, if the vectors are neighbours thenv1[k] = v2[k], therefore

v1[i] 6= v1[k] or v2[i] 6= v2[k] will always hold (see Theorem 4(c)).

Observation 2. Since UCMS or the Masking test set includes sensitive vector

pairs for every condition in a given Boolean specification, a test set satisfying

these criteria is guaranteed to detect ENF, VNF, MVF or ORF faults. Using

Corollary 2, a UCMS test set can detect VRF faults whereas Masking test set

does not.

Corollary 3. If all the conditions are uncoupled in a given Boolean specifica-

tion, any input-independence test set is guaranteed to detect ENF, VNF, MVF,

VRF and ORF faults.

Proof. Theorem 4 and Corollary 2 proves the result for ENF, VNF, MVF, VRF

faults. Further, since all the conditions are uncoupled, a vector pair will always

28

be sensitive in one of the subformula with respect to an input. Thus, the fault

detection condition for ORF, as mentioned in the proof for Theorem 4, will

always hold.

Observation 3. As mentioned earlier, the UC test criterion is defined only for

decisions with uncoupled conditions. Therefore, as shown in Corollary 3, a UC

test set is guaranteed to detect all faults except ASF.

Observation 4. As a UCMW test set includes sensitive vector pairs with re-

spect to inputs, it may not detect faults if the inputs occur more than once in the

Boolean formula.

In the case of UCMW, the fault detection effectiveness decreases as the in-

fluence of inputs increases in a Boolean function. As mentioned earlier, the

influence of an input can vary from0 and2n−1 for Boolean functions. The in-

fluence of a Boolean function to its inputs is the average of the influence for all

its inputs. The average influence isO(n) for weak Boolean functions, whereas

it is maximum (2n−1) for a class of balanced functions. Thus, it is expected

that the effectiveness of UCMW will deteriorate with the increase in average

influence of a Boolean function.

Observation 5. The addition of insensitive test cases to an UCMW test set to

satisfy RCDC will not increase its effectiveness (cf. Theorem 3 and Corollary

1).

Further, even in the presence of faults that affect insensitive vector pairs, the

effectiveness of a random test set is expected to be the same as that of a test

set satisfying RCDC. This is due to the exponential number of vector pairs in

setsµ0i or µ1

i in the case of weak Boolean functions (cf. Proposition 2(d) and

Theorem 2).

29

7 Conclusions and Future Work

Three different forms of Modified Condition Decision Coverage (MCDC) have

been formally defined and analysed. In addition, another criterion, Reinforced

Condition Decision Coverage (RCDC), is also mathematically defined and com-

pared with various versions of MCDC test criteria.

Criterion ENF VNF VRF MVF ORF ASF

UC√ √ √ √ √ ⊗

UCMW ⊗ ⊗ ⊗ ⊗ ⊗ ⊗UCMS

√ √ √ √ √ ⊗Masking

√ √ ⊗ √ √ ⊗RCDC ⊗ ⊗ ⊗ ⊗ ⊗ ⊗

Table II: Fault detection capability of various criteria (√

: always detect⊗: may

or may not detect).

It has been shown that a random test set may not be reliable for detecting

operator and operand faults. On the other hand, an MCDC test set may detect all

operator and operand faults since it targets the fault affected sub-domain (which

may be linear) of the exponential sized domain of the specification. The study

also shows that the insensitive vectors are not useful to detect such faults and

thus their addition to UCMW to satisfy RCDC does not further improve the

fault detection effectiveness of UCMW. The summarised results for the fault

detection effectiveness with respect to the six types of faults are presented in

Table II (cf. Observations 1-5).

Unique Cause Masking MCDC Weak (UCMW) is the weakest interpreta-

tion among the different variants of MCDC since it does not guarantee the de-

tection of faults. For a given specification and implementation, a large number

of UCMW test sets may be feasible with varying fault detection effectiveness. It

is possible to design an algorithm that is based on the fault detection conditions

30

(see Theorem 4) to find the most effective UCMW test set. This may be useful

in the cases where the number of test cases in the test sets is a concern since the

UCMW test sets are linear in size in terms of the number of inputs (cf. Table

I(a)).

The analysis shows that the MCDC test criterion is designed to check the

Boolean specifications that are implemented as Boolean decisions. The RCDC

test criterion may be helpful in detecting certain faults only if the implemen-

tation is in tabular form and explicitly maps every input vector to the output.

However, to guarantee the detection of faults that RCDC aims to detect would

require an exponential number of test cases for weak Boolean functions. Thus,

in the case of weak Boolean functions the effectiveness of UCMW and RCDC

test sets is likely to be the same.

It can observed in Table II that none of the test criteria guarantee detection

of Associative Shift Faults (ASF). In other words, for a given specification and

implementation, two test sets satisfying a test criterion may not be equally ef-

fective in detecting ASF faults. However, based on the properties mentioned in

Theorem 4, it is possible to design an algorithm generating MCDC test set that

could guarantee detection of all six types of faults including ASF.

The analysis presented in this paper assumes the presence of only one fault

in a Boolean specification. A possible direction for further study could be to

evaluate fault detection effectiveness in the presence of more than one fault at

same or different control points. In addition, the study of effectiveness in the

presence of short circuit operators (which restricts evaluation) is also a possible

area for further investigation. In this study, it has been assumed that the opera-

tors are not short-circuited. In the case of a short circuit, the number of test sets

will be more restricted; therefore the definitions would require extra clauses to

take into account the evaluation order imposed by the operators.

The formal framework presented in this paper is based on the influence of

inputs in Boolean specifications. As this can be computed a priori, the results

31

stated are directly applicable to compute the effort required for testing. The

classification of Boolean functions based on the influence of inputs also provides

a uniform framework to evaluate other control-flow testing criteria.

References

[1] RTCA/DO-178B. Software Considerations in Airborne Systems and

Equipment Certification. Washington DC, USA, 1992.

[2] H. Zhu, P. Hall, and H. R. May. Software Unit Test Coverage and Ade-

quacy.ACM Computing Surveys, 29(4):336–427, December 1997.

[3] M. Roper.Software Testing. McGraw-Hill Book Company Europe, 1994.

[4] G. Myers.The Art of Software Testing. Wiley-Interscience, 1979.

[5] K. J. Hayhurst and D. S. Veerhusen. A Practical Approach to Modified

Condition/Decision Coverage. In20th Digital Avionics Systems Confer-

ence (DASC), volume 1, pages 1B2/1–1B2/10, October 2001.

[6] A. Dupuy and N. Leveson. An Empirical Evaluation of the MC/DC Cov-

erage Criterion on the HETE-2 Satellite Software. InDASC: Digital Avia-

tion Systems Conference, Phildelphia, pages 1B6/1–1B6/7. IEEE, October

2000.

[7] M. DeWalt. MCDC: A Blistering Love/Hate Relationship. InFAA Na-

tional Software Conference, April 1999. Long Beach, CA, USA.

[8] S. A. Vilkomir and J. P. Bowen. Reinforced Condition/Decision Cover-

age (RC/DC): A New Criterion for Software Testing. In D. Bert, J. P.

Bowen, M. Henson, and K. Robinson, editors,2nd International Confer-

ence, Formal Specification and Development in Z and B, volume 2272 of

32

Lecture Notes in Computer Science, pages 295–313. Springer-Verlag, Jan-

uary 2002.

[9] F. M. Brown. Boolean Reasoning: The Logic of Boolean Equations.

Kluwer Academic Publishers, 1990.

[10] T. Y. Chen and M. F. Lau. Test Case Selection Strategies based on Boolean

Specifications.Software Testing, Verification and Reliability, 11(3):165–

180, 2001.

[11] D. J. Richardson and M. C. Thompson. An Analysis of Test Data Selection

Criteria using the RELAY Model of Fault Detection.IEEE Transactions

on Software Engineering, 19(6):533–553, June 1993.

[12] R. A. DeMillo, R. J. Lipton, and F. G. Sayward. Hints on Test Data Selec-

tion: Help for the Practicing Programmer.IEEE Computer, 11(4):34–41,

April 1978.

[13] R. M. Hierons. Comparing Test Sets and Criteria in the Presence of Test

Hypotheses and Fault Domains.ACM Transactions on Software Engineer-

ing and Methodology, 11(4):427–448, October 2002.

[14] J. M. Voas. PIE: A Dynamic Failure-Based Technique.IEEE Transactions

on Software Engineering, 18(2):717–727, August 1992.

[15] A. J. Offutt and J. Pan. Automatically Detecting Equivalent Mutants and

Infeasible Paths.Software Testing, Verification and Reliability, 7(3):165–

192, September 1997.

[16] B. Beizer. Software Testing Techniques, chapter 10, pages 320–362. Van

Nostrand Reinhold, Inc. New York, 2nd edition, 1990.

33

[17] S. Burton. Towards Automated Unit Testing of Statechart Implementa-

tions. Technical report, Department of Computer Science, University of

York, 1999.

[18] J. Offutt. Generating Test Data From Requirements/Specifications: Phase

III Final Report. Technical Report ISE-TR-00-02, Department of Infor-

mation and Software Engineering, George Mason University, May 2000.

[19] M. Heimdahl, S. Rayadurgam, W. Visser, G. Devaraj, and J. Gao. Auto-

Generating Test Sequences using Model Checkers: A Case Study. InFor-

mal Approaches to Testing of Software (FATES), volume 2931, pages 42–

59. LNCS, October 2003.

[20] J. Offutt and A. Abdurazik. Generating Tests from UML Specifications.

In Second International Conference on the Unified Modeling Language,

pages 416–429, October 1999.

[21] T. Mucke and M. Huhn. Generation of Optimized Testsuites for UML

Statecharts with Time. InTesting of Communicating Systems (TESTCOM),

pages 128–143. LNCS, March 2004.

[22] A. J. Offutt, Y. Xiong, and S. Liu. Criteria for Generating Specification-

based Tests. In5th International Conference on Engineering of Complex

Computer Systems (ISECCS), pages 119–129. IEEE, October 1999.

[23] J. Chilenski and S. Miller. Applicability of Modified Condition/Decision

Coverage to Software Testing.Software Engineering Journal, 9(5):193–

200, September 1994.

[24] J. J. Chilenski. An Investigation of Three Forms of the Modified Condition

Decision Coverage (MCDC) Criterion. Technical Report DOT/FAA/AR-

01/18, U.S. Department of Transportation, Federal Aviation Administra-

tion, April 2001.

34

[25] R. Jasper, M. Brennan, K. Williamson, B. Currier, and D. Zimmerman.

Test Data Generation and Feasible Path Analysis. InInternational Sympo-

sium on Software Testing and Analysis, pages 95–107. ACM, 1994.

[26] J. A. Jones and M. J. Harrold. Test-Suite Reduction and Prioritization for

Modified Condition/Decision Coverage. InInternational Conference on

Software Maintenance (ICSM), pages 92–101. IEEE, November 2001.

[27] Y. Y. Li. Structural Test Cases Analysis and Implementation. In42nd

Midwest Symposium on Circuits and Systems, volume 2, pages 882–885,

August 1999.

[28] A. White. Comments on Modified Condition/Decision Coverage for Soft-

ware Testing. InIEEE Aerospace Conference, Big Sky, Montana, USA,

volume 6, pages 2821–2828. IEEE, March 2001.

[29] A. Paradkar, K. C. Tai, and M. A. Vouk. Automatic Test-Generation for

Predicates.IEEE Transactions on Reliability, 45(4):515–530, December

1996.

[30] M. A. Vouk, K. C. Tai, and A. Paradkar. Empirical Studies of Predicate-

based Software Testing. In5th International Symposium on Software Re-

liability Engineering, pages 55–64. IEEE, 1994.

[31] K.-C. Tai. Theory of Fault-based Predicate Testing for Computer Pro-

grams.IEEE Transactions on Software Engineering, 22(8):552–562, Au-

gust 1996.

[32] E. Weyuker, T. Gorodia, and A. Singh. Automatically Generating Test

Data from a Boolean Specification.IEEE Transactions on Software Engi-

neering, 20(5):353–363, May 1994.

[33] S. Rayadurgam and M. P. E. Heimdahl. Coverage based Test-case Gener-

ation using Model Checkers. In8th Annual International Conference and

35

Workshop on the Engineering of Computer based Systems (ECBS), pages

83–91. IEEE, April 2001.

[34] L. J. Morell. A Theory of Fault-based Testing.IEEE Transactions on

Software Engineering, 16(8):844–857, August 1990.

[35] J. Kahn, G. Kalai, and N. Linial. The Influence of Variables on Boolean

Functions. In29th Annual Symposium on Foundations of Computer Sci-

ence, pages 68–80. IEEE, October 1998.

[36] A. Bernasconi. On the Complexity of Balanced Boolean Functions.Infor-

mation Processing Letters, 70:157–163, 1999.

[37] A. J. Offutt. Investigations of the Software Testing Coupling Effect.ACM

Transactions on Software Engineering and Methodology, 1(1):5–20, Jan-

uary 1992.

[38] D. R. Kuhn. Fault Classes and Error Detection Capability of Specification-

based Testing.ACM Transactions on Software Engineering and Method-

ology, 8(4):411–424, October 1999.

[39] T. Tsuchiya and T. Kikuno. On Fault Classes and Error Detection Ca-

pability of Specification-based Testing.ACM Transactions on Software

Engineering and Methodology, 11(1):58–62, January 2002.

[40] V. Okun, P. E. Black, and Y. Yesha. Comparison of Fault Classes

in Specification-based Testing.Information and Software Technology,

46(8):525–533, June 2004.

36