On the divide-and-conquer approach towards test suite reduction

On the divide-and-conquer approachtowards test suite reduction

T.Y. Chen *, M.F. Lau

School of Information Technology, Swinburne University of Technology, P.O. Box 218,

Hawthorn, Victoria 3122, Australia

Received 30 December 2001; received in revised form 27 November 2002;

accepted 30 December 2002

Abstract

When testing a program, testers need to generate a set of test cases that satisfies a

testing objective. This set of test cases is referred to as a test suite. Usually, the generated

test suite may contain redundancy, that is, some test cases may be removed from the test

suite without rendering the satisfaction of the testing objective. A representative set is a

subset of a test suite that satisfies the same testing objective, and test suite reduction is

aimed at finding the representative sets. This paper studies the properties of represen-

tative sets related to the divide-and-conquer approach towards test suite reduction. The

results provide us a better understanding of the structural properties of representative

sets and how to apply the divide-and-conquer approach towards test suite reduction.

� 2003 Elsevier Science Inc. All rights reserved.

Keywords: Divide-and-conquer approach; Software testing; Test suite reduction

1. Introduction

Before testing a software, software testers need to establish a testing ob-

jective. This testing objective can be considered as a set of testing requirements,

or simply requirements. For example, if the testing objective requires all

Information Sciences 152 (2003) 89–119

www.elsevier.com/locate/ins

*Corresponding author. Tel.: +61-3-9214-8180; fax: +61-3-9819-0823.

E-mail address: [email protected] (T.Y. Chen).

0020-0255/03/$ - see front matter � 2003 Elsevier Science Inc. All rights reserved.

doi:10.1016/S0020-0255(03)00060-4

mail to: [email protected]

branches of a program to be exercised at least once, each branch will be re-

garded as a testing requirement. After the set of testing requirements has been

determined, test cases are constructed to collectively satisfy all testing re-

quirements. A set of test cases that can collectively satisfy all testing require-ments is referred to as a test suite.

A typical method of constructing a test suite is to generate a test case for

each testing requirement. This method has been employed by most automated

test case generators [7,9,12,15,18]. Since the test case for a particular testing

requirement may also satisfy other testing requirements, the generated test

suite may contain redundancy in the sense that some test cases can be removed

without rendering the satisfaction of all testing requirements. A representative

set is defined as a subset of a test suite that can satisfy the same set of testingrequirements as the original test suite [10]. Being a subset of the test suite, a

representative set requires less resources to execute. Therefore, it is important

to find representative sets of the original test suite. Such a process is known as

test suite reduction.

After a program has been modified, regression testing would be conducted

on the modified program. Since new requirements may be introduced, extra

test cases may be required. After these new test cases have been added to the

existing test suite, the newly formed test suite may contain redundancy becausethese new test cases may satisfy some previous requirements. As a result, test

suite reduction should be applied [10,13]. In the context of regression testing,

previous work on test suite reduction concentrates (1) on identifying the

modified relationship between the requirements and test cases, and (2) on re-

ducing the number of test cases to be retested [6,8,10,14,17].

Intuitively speaking, a representative set as a subset of the test suite may

have less fault detecting capability. A natural and yet important concern of

software testers is whether a smaller sized representative set detects less faultsthan the original test suite despite that the same testing objective is satisfied.

However, Wong et al. [19,20] have shown that there is no significant reduction

in the fault detecting capability of representative sets when the test suites are

generated according to the data-flow coverage criteria [16].

Since we like to minimise the testing cost as much as possible, we are in-

terested in the representative sets with the least number of elements, known as

the optimal representative sets. However, the problem of finding the optimal

respresentative sets is NP-complete [10,13]. There are two main pragmaticapproaches towards NP-complete problems. One is the heuristics approach.

Various heuristics (such as, heuristic GRE [4], heuristic H [10], and greedy

heuritics [11]) have been proposed for finding representative sets. The perfor-

mance of these heuristics were studied and compared in [5]. It is found that

none of the heuristics always outperforms the other two heuristics in delivering

the smallest sized representative sets.

90 T.Y. Chen, M.F. Lau / Information Sciences 152 (2003) 89–119

Another approach towards NP-complete problem is the divide-and-conquer

methodology [1]. This approach basically involves three steps, namely (1) to

decompose the original problem into smaller subproblems, (2) to find optimal

solutions for the subproblems, and (3) to construct a solution for the originalproblem from those of the subproblems. This approach has been very suc-

cessful in many areas of computer science [1]. However, not every strategy can

guarantee the delivery of optimal representative sets.

A dividing strategy is said to be complete with respect to optimal or minimal

representative sets if all minimal or optimal representative sets of the original

problem can be constructed from their counterparts of the subproblems. A

representative set is minimal if none of its proper subsets is a representative set.

In other words, minimal representative sets do not contain redundancy. Ob-viously, an optimal representative set is minimal but not vice versa. If the

testing overheads of all test cases are the same, we simply need to know only

one optimal representative set because all optimal representative sets are of the

same size. Otherwise, the most cost effective representative set is not necessarily

optimal, though it must be minimal.

This paper investigates the completeness properties of some dividing strat-

egies with respect to the minimal and optimal representative sets via the

analysis of the structural properties of the representative sets. The rest of thispaper is organised as follows. Section 2 introduces some basic concepts and

notation. Section 3 outlines the definitions and properties of various types of

representative sets. Moreover, we present the concepts and the properties of the

redundant subsets and the essential subsets. Sections 4 and 5 discuss the

completeness properties of some dividing strategies based on essential test cases

and essential subsets, respectively. Finally, Section 6 concludes the paper.

2. Preliminaries

2.1. Notation and terminology

In this section, we introduce the notation and terminologies used in this

paper. For any set A, we use jAj and P ðAÞ to denote the size and the power setof A, respectively. For any two sets of sets B and C, we use B� C to denote theset fb [ c: b 2 B and c 2 Cg.A partial ordering v on a set A is a binary relation on A such that (1) a v a

for every a 2 A; (2) a v b and b v a imply a ¼ b; and (3) a v b and b v c implya v c. ðA;vÞ is a partially ordered set if v is a partial ordering on the set A.Suppose that ðA;vÞ is a partially ordered set. An element a 2 A is a minimal

(maximal) element of A if for each b 2 A, b v a (a v b) implies a ¼ b. Fur-thermore, a 2 A is a least (greatest) element of A if a v b (b v a) for all b 2 A.

T.Y. Chen, M.F. Lau / Information Sciences 152 (2003) 89–119 91

Assume B � A. An element a 2 A is a lower (upper) bound of B if a v b (b v a)for all b 2 B. Moreover, a is a greatest lower (least upper) bound of B if a is thegreatest (least) element of the set of all lower (upper) bounds of B. We use glbBand lubB to denote the greatest lower bound and the least upper bound of Brespectively. If Q is a subset of the partially ordered set ðP ðAÞ;�Þ for any set A,glbQ ¼

TQi2Q Qi and lubQ ¼

SQi2Q Qi.

In this paper, we use R to denote the set of all testing requirements. A testingrequirement is said to be feasible if there is a test case in the input domain that

satisfies or executes it. We also assume that there are finitely many testing

requirements and they are feasible. As mentioned before, a test suite is a set of

test cases that can collectively satisfy R. In this paper, we use T to denote such atest suite. Obviously, a finite test suite T always exists as R is finite. Moreover,we assume that for any t 2 T , there exists r 2 R such that t satisfies r.For non-empty T and R, a test case t 2 T satisfying requirement r 2 R can be

represented as a binary relation SðT ;RÞ from T to R. This binary relation isreferred to as the satisfiability relation from T to R. Whenever it is clear fromthe context, a satisfiability relation from T to R will be denoted by S instead ofSðT ;RÞ. As a reminder, SðT ;RÞ is defined for non-empty T and R.

Definition 2.1. The satisfiability relation SðT ;RÞ from T to R is defined as theset fðt; rÞ 2 T � R : t satisfies rg.

Definition 2.2. Let SðT ;RÞ be the satisfiability relation from T to R.1. The set of all requirements satisfied by t 2 T is defined as ReqðtÞ ¼

fr 2 R : ðt; rÞ 2 SðT ;RÞg.2. The set of all requirements satisfied by T1ð6¼ ;Þ � T is defined as ReqðT1Þ ¼S

t2T1 ReqðtÞ. Moreover, Reqð;Þ ¼ ;.

The following are some simple properties of Req, which are immediate fromthe definition.

Lemma 2.1. Let S be the satisfiability relation from T to R and T1; T2 � T .1. If T1 � T2, then ReqðT1Þ � ReqðT2Þ.2. ReqðT1 [ T2Þ ¼ ReqðT1Þ [ ReqðT2Þ.3. ReqðT1 \ T2Þ � ReqðT1Þ \ ReqðT2Þ.4. ReqðT1Þ n ReqðT2Þ � ReqðT1 n T2Þ.

Definition 2.3. Let SðT ;RÞ be the satisfiability relation from T to R.1. The set of all test cases in T satisfying r 2 R is defined as TestðrÞ ¼

ft 2 T : ðt; rÞ 2 SðT ;RÞg.2. The set of all test cases in T satisfying a non-empty R1ð6¼ ;Þ � R is defined as

TestðR1Þ ¼S

r2R1 TestðrÞ. Moreover, Testð;Þ ¼ ;.


Similar to the characterisations of Req in Lemma 2.1, Test has the followingproperties.

Lemma 2.2. Let S be the satisfiability relation from T to R and R1;R2 � R1. If R1 � R2, then TestðR1Þ � TestðR2Þ.2. TestðR1 [ R2Þ ¼ TestðR1Þ [ TestðR2Þ.3. TestðR1 \ R2Þ � TestðR1Þ \ TestðR2Þ.4. TestðR1Þ n TestðR2Þ � TestðR1 n R2Þ.

3. Structural properties of representative sets

The problem of test suite reduction is to identify the subsets of T that canstill collectively satisfy R. Such a subset is known as a representative set [10].Previous studies [3,10,13] have focussed on the optimal representative sets,

which are representative sets with the smallest size. As explained earlier, this

approach may not be appropriate when the costs of executing test cases are not

uniform. Hence, we search for the minimal representative sets with respect to

set inclusion, instead.

This section discusses some structural properties of the sets of representative

sets. Moreover, the concepts of redundant or essential test case are extended tothe concepts of redundant or essential subset, respectively. The relationships

among the set of redundant subsets, the set of essential subsets, and the set of

representative sets are investigated. These relationships provide the basis for

the development of dividing strategies (to be discussed in Sections 4 and 5) that

preserve the set of all minimal or optimal representative sets.

3.1. Representative sets

This section introduces the concepts and properties of representative sets,

minimal representative sets and optimal representative sets.

Definition 3.1. Let S be the satisfiability relation from T to R and T1 � T . T1 issaid to be a representative set of S if ReqðT1Þ ¼ R.

Let REP ðSÞ denote the set of all representative sets of S. Since T 2 REP ðSÞ,REP ðSÞ 6¼ ;. Obviously, REP ðSÞ has lubREP ðSÞ ¼

ST12REP ðSÞ T1 ¼ T as its

greatest element, but its least element may not exist.

A minimal representative set does not contain any redundancy. In fact,

the minimal representative sets of S are exactly the minimal elements ofðREP ðSÞ;�Þ. It should be noted that the minimal representative sets may havedifferent sizes. The set of all minimal representative sets of S is denoted byMIN REP ðSÞ. Since REP ðSÞ is non-empty, MIN REP ðSÞ is non-empty.


Definition 3.2. Let S be the satisfiability relation from T to R and T1 2 REP ðSÞ.T1 is said to be an optimal representative set of S if jT1j6 jT2j for everyT2 2 REP ðSÞ.

In other words, an optimal representative set is a representative set with

the smallest number of elements. Although the optimal representative sets

of S may not be unique, they have the same size. The set of all optimalrepresentative sets of S is denoted by OPT REP ðSÞ. Since REP ðSÞ is non-empty, OPT REP ðSÞ is non-empty. It follows immediately from the defini-

tions that optimal representative sets are exactly the minimal representative

sets with the smallest size, as stated in Lemma 3.1. Moreover, it is easy to see

that a minimal representative set is not necessarily an optimal representativeset.

Lemma 3.1. Let S be the satisfiability relation from T to R. Then,1. OPT REP ðSÞ � MIN REP ðSÞ.2. OPT REP ðSÞ ¼ fT1 2 MIN REP ðSÞ : jT1j6 jT2j for all T2 2 MIN REP ðSÞg.

3.2. Redundant test cases and essential test cases

There are two types of test cases in a test suite, namely, redundant test

cases and essential test cases. Intuitively speaking, redundant test cases

are those that can be removed without destroying the complete satisfaction ofall requirements; while essential test cases must appear in every representative

set. In this section, the properties of these two types of test cases are investi-

gated.

Definition 3.3. Let S be the satisfiability relation from T to R. A test case t 2 Tis said to be a redundant test case with respect to S if R ¼ ReqðT n ftgÞ.

The following proposition gives a necessary and sufficient condition for atest case to be redundant.

Proposition 3.1. Let S be the satisfiability relation from T to R. A test case t 2 Tis redundant if and only if for every r 2 ReqðtÞ, jTestðrÞj > 1.

Proof. Immediate from definition. �

Suppose that t is a redundant test case. Obviously, any representative set ofSðT n ftg;RÞ is also a representative set of SðT ;RÞ. Moreover, any minimalrepresentative set of SðT n ftg;RÞ is also minimal with respect to SðT ;RÞ.


However, an optimal representative set of SðT n ftg;RÞ may not be optimalwith respect to SðT ;RÞ as illustrated in Example 3.1.

Example 3.1. The satisfiability relation S between ft1; . . . ; t6g and fr1; . . . ; r7gis given in Table 1. The test case t4 is a redundant test case and OPT REP �ðSðT ;RÞÞ ¼ fft1; t4; t5gg. When t4 is removed from T , OPT REP � ðSðT n ft4g;RÞÞ ¼ fft1; t2; t3; t5g; ft1; t2; t3; t6gg.

The concept of essential test cases is complementary to the concept of re-

dundant test cases. A test case is said to be essential if it must appear in everyrepresentative set. Therefore, any subset of a test suite missing any essential test

case is not a representative set.

Definition 3.4. Let S be the satisfiability relation from T to R. A test case t 2 Tis said to be an essential test case with respect to S if R 6¼ ReqðT n ftgÞ.

It is clear from the definitions that a test case is essential if and only if it is

not redundant. The following proposition gives a necessary and sufficient

condition for a test case to be essential.

Proposition 3.2. Let S be the satisfiability relation from T to R. A test case t 2 Tis an essential test case if and only if there exists a requirement r 2 R such thatTestðrÞ ¼ ftg.

Proof. Immediate from Proposition 3.1 and the assumption that TestðrÞ 6¼ ; forany r 2 R. �

The set of all essential test cases with respect to a satisfiability relation S isdenoted by EssðSÞ, or simply Ess whenever it is clear from the context. It

follows immediately from the definition that Ess is included in every repre-sentative set.

Proposition 3.3. Every representative set has Ess as a subset.

Table 1

The satisfiability relation S for Example 3.1

r1 r2 r3 r4 r5 r6 r7

t1 X X

t2 X X X

t3 X X

t4 X X X

t5 X X X

t6 X X X


3.3. Redundant subsets and essential subsets

This section extends the concepts of redundant test case and essential test

case to the concepts of redundant subset and essential subset, respectively.Similar to the concept of redundant test case, a subset T1 of a test suite T is saidto be redundant if the set R of all requirements can still be satisfied by the testcases in T n T1.

Definition 3.5. Let S be the satisfiability relation from T to R. T1 � T is said tobe a redundant subset with respect to S if R ¼ ReqðT n T1Þ.

The set of all redundant subsets with respect to S is denoted by RSðSÞ. Sincesubsets of a redundant subset are redundant and the process of test suite re-

duction can be considered as the removal of as many redundant test cases as

possible, it is natural to focus the maximal elements of RSðSÞ. The set of allmaximal redundant subsets with respect to S is denoted by MAX RSðSÞ. Ob-viously, MAX RSðSÞ is non-empty. The following proposition states an inter-esting relationship between Ess and RSðSÞ.

Proposition 3.4. Let S be the satisfiability relation from T to R. Then,lubMAX RSðSÞ ¼ T n Ess.

Proof. A test case t 2 T belongs to some maximal redundant subsets if and onlyif t is redundant. Moreover, t is redundant if and only if t 62 Ess. SincelubMAX RSðSÞ ¼

ST12MAX RSðSÞ T1, lubMAX RSðSÞ ¼ T n Ess. �

The concepts of essential subsets and redundant subsets are complementary

to each other because when an essential subset is removed from the test suite,the remaining test cases cannot collectively satisfy the set of all requirements.

Definition 3.6. Let S be the satisfiability relation from T to R and T1 � T . T1 issaid to be an essential subset with respect to S if R 6¼ ReqðT n T1Þ.

The set of all essential subsets with respect to S is denoted by ESðSÞ. SinceT 2 ESðSÞ, ESðSÞ 6¼ ;. It should be noted that an essential subset is non-emptyand T1 is an essential subset whenever T1 contains an essential test case. Sincethe least element of ESðSÞ may not exist, we need to investigate the minimalessential subsets. The set of all minimal elements of ESðSÞ is denoted byMIN ESðSÞ. Since ESðSÞ is non-empty, MIN ESðSÞ is non-empty.It is obvious from the definitions that a subset of T is essential if and only if

it is not redundant. Thus, ESðSÞ \ RSðSÞ ¼ ;. The following proposition givesa necessary and sufficient condition for a subset of test cases to be essential.


Proposition 3.5. Let S be the satisfiability relation from T to R and T1 � T . T1 isan essential subset with respect to S if and only if there exists a requirement r 2 Rsuch that TestðrÞ � T1.

Proof (if part). Suppose there exists r 2 R such that TestðrÞ � T1. Now, forevery t1 2 ðT n T1Þ, t1 62 TestðrÞ. Thus, r 62 ReqðT n T1Þ. Therefore, R 6¼ ReqðT nT1Þ. The result follows.(only-if part) Since T1 is an essential subset, R 6¼ ReqðT n T1Þ. Thus, there

exists r 2 R and r 62 ReqðT n T1Þ. Therefore, for every t 2 T such that t 2TestðrÞ, t 62 ðT n T1Þ. Since TestðrÞ � T , t 2 T1. As a result, TestðrÞ � T1. �

Let TSðSðT ;RÞÞ ¼ fT1 � T : T1 ¼ TestðrÞ for some r 2 Rg. Similarly, TSðSÞwill be used instead of TSðSðT ;RÞÞ whenever it is clear from the context. Thefollowing relationship between TSðSÞ and ESðSÞ also follows immediately afterProposition 3.5.

Proposition 3.6. Let S be the satisfiability relation from T to R. Then,TSðSÞ � ESðSÞ.

In other words, TestðrÞ is an essential subset of T . However, not every es-sential subset of T is of the form TestðrÞ. This is illustrated in Example 3.2below.

Proposition 3.7. Let S be the satisfiability relation from T to R. Then,MIN ESðSÞ � TSðSÞ.

Proof. Let T1 2 MIN ESðSÞ. Since T1 2 ESðSÞ, by Proposition 3.5, there existsa requirement r 2 R such that TestðrÞ � T1. By Proposition 3.6, TestðrÞ 2ESðSÞ. Since T1 is minimal, we have T1 ¼ TestðrÞ. Thus, T1 2 TSðSÞ. The resultfollows. �

Despite the fact that TSðSÞ contains all minimal elements of ESðSÞ, not everyTestðrÞ is a minimal essential subset. This is also illustrated in Example 3.2.

Example 3.2. Suppose T ¼ ft1; . . . ; t5g and R ¼ fr1; . . . ; r7g with the satisfiabil-ity relation S given in Table 2. It is easy to see that Testðr1Þ ¼ ft1; t2g, Testðr2Þ ¼ft3; t4g, Testðr3Þ ¼ ft2; t5g, Testðr4Þ ¼ ft1; t3g, Testðr5Þ ¼ ft2; t4; t5g, Testðr6Þ ¼ft1; t3g, Testðr7Þ ¼ ft5g, and MIN ESðSÞ ¼ fft1; t2g; ft1; t3g, ft3; t4g; ft5gg. Thus,Testðr5Þ 62 MIN ESðSÞ. Moreover, ft1; t2; t3g 2 ESðSÞ. Hence, MIN ESðSÞ$TSðSÞ$ESðSÞ.

We have shown thatMIN ESðSÞ is a subset of TSðSÞ, which in turn is a subsetof ESðSÞ. Moreover, we are going to prove that MIN ESðSÞ ¼ MIN TSðSÞ


whereMIN TSðSÞ denotes the set of all minimal elements of TSðSÞ. This result isimportant because it provides an efficient way to compute MIN ESðSÞ. In orderto prove this property, we need the following lemma.

Lemma 3.2. Let S be the satisfiability relation from T to R. For anyTestðrÞ 2 TSðSÞ, there exists T1 2 MIN ESðSÞ such that T1 � TestðrÞ.

Proof. For any TestðrÞ 2 TSðSÞ, two cases arise.1. Suppose TestðrÞ 2 MIN ESðSÞ. Simply let T1 ¼ TestðrÞ.2. Suppose TestðrÞ 62 MIN ESðSÞ. By Proposition 3.6, TestðrÞ 2 ESðSÞ. Hence,there exists T1 2 MIN ESðSÞ such that T1 � TestðrÞ.

Thus, the result follows. �

Proposition 3.8. Let S be the satisfiability relation from T to R. Then,MIN ESðSÞ ¼ MIN TSðSÞ.

Proof (� part). For any T1 2 MIN ESðSÞ, by Proposition 3.7, T1 ¼ TestðrÞ forsome r 2 R. Suppose T1 62 MIN TSðSÞ. Then, there exists r0ð6¼ rÞ 2 R such thatTestðr0Þ � TestðrÞ and Testðr0Þ 6¼ TestðrÞ. Since Testðr0Þ 2 ESðSÞ by Proposition3.6, this contradicts the minimality of T1. Thus, MIN ESðSÞ � MIN TSðSÞ.(� part) For any T1 2 MIN TSðSÞ, T1 ¼ TestðrÞ for some r 2 R. By

Lemma 3.2, there exists T2 2 MIN ESðSÞ such that T2 � TestðrÞ. SupposeT2 6¼ TestðrÞ. Then, by Proposition 3.7, there exists r0 2 R such that T2 ¼Testðr0Þ. Thus, we have Testðr0Þ � TestðrÞ and Testðr0Þ 6¼ TestðrÞ, which contra-dicts TestðrÞ 2 MIN TSðSÞ. Therefore, T2 ¼ TestðrÞ. Thus, MIN TSðSÞ �MIN ESðSÞ. �

Since computations of TSðSÞ and MIN TSðSÞ are straightforward fromdefinitions, Proposition 3.8 suggests that MIN ESðSÞ can be computed withoutknowing all essential subsets of S. In summary, the relationships between ESðSÞand TSðSÞ are illustrated in the schematic diagram in Fig. 1.

Table 2


r1 r2 r3 r4 r5 r6 r7

t1 X X X

t2 X X X

t3 X X X

t4 X X

t5 X X X


There are two complementary aspects of MAX RSðSÞ and MIN ESðSÞ,namely (1) a subset is essential if and only if it is not redundant; and (2) thenotion of maximality and minimality. Thus, it is very natural to ask whether

the set complement of a maximal redundant subset is a minimal essential

subset, and vice versa (that is, T1 2 MAX RSðSÞ if and only if ðT n T1Þ 2MIN ESðSÞ). However, the following example shows that this is not the case.Instead, we have T1 2 MAX RSðSÞ if and only if ðT n T1Þ 2 MIN REP ðSÞ asstated in Proposition 3.9 in Section 3.4.

Example 3.3. Suppose R ¼ fr1; . . . ; r4g and T ¼ ft1; . . . ; t4g. The satisfiabilityrelation S from T to R is given in Table 3. It is easy to see that

MAX RSðSÞ ¼ fft1; t4g; ft2; t4g; ft3gg, MIN ESðSÞ ¼ fft1; t2g; ft2; t3g; ft3; t4g; ft1;t3gg and MIN REP ðSÞ ¼ fft2; t3g; ft1; t3g; ft1; t2; t4gg. ft3g is a maximal redun-dant subset, but T n ft3g ¼ ft1; t2; t4g is not a minimal essential subset. Moreover,

ES(S)

TS(S)

ES(S)= MIN_

MIN_ TS(S)

Fig. 1. MIN ESðSÞ ¼ MIN TSðSÞ � TSðSÞ � ESðSÞ.

Table 3


r1 r2 r3 r4

t1 X X

t2 X X

t3 X X X

t4 X


ft1; t2g is a minimal essential subset, but T n ft1; t2g ¼ ft3; t4g is not a maximalredundant subset.

3.4. Structural properties of REP(S)

In this section, we investigate the structural properties of the set of repre-

sentative sets. Since essential subsets should be preserved and redundant sub-sets should be removed in the test suite reduction process, we are interested

in knowing the relationships among ESðSÞ, RSðSÞ and REP ðSÞ. When thetesting overheads of all test cases are the same, the most cost effective repre-

sentative sets are the optimal representative sets; otherwise, the minimal rep-

resentative sets. Therefore, we are interested in investigating the characterisitics

of the minimal and optimal representative sets.

First, let us discuss the relationships between REP ðSÞ and RSðSÞ. It is clearfrom the definition that T1 is a redundant subset of S if and only if T n T1 is arepresentative set of S. Hence, there is a one-to-one correspondence betweenRSðSÞ and REP ðSÞ.

Lemma 3.3. Let S be the satisfiability relation from T to R. T1 2 RSðSÞ if andonly if ðT n T1Þ 2 REP ðSÞ.

Proof. Immediate from definitions. �

Proposition 3.9. Let S be the satisfiability relation from T to R. T1 2 MAX RSðSÞif and only if ðT n T1Þ 2 MIN REP ðSÞ.

Proof. Immediate from definitions and Lemma 3.3. �

An implication of Proposition 3.9 is that finding all maximal redundant

subsets with respect to S is equivalent to finding all minimal representative setsof S. Further relationships between MAX RSðSÞ and MIN REP ðSÞ are stated inthe following proposition. These relationships are immediate from Proposition

3.9 and De Morgan�s Law.

Proposition 3.10. Let S be the satisfiability relation from T to R. Then, we have1. glbMAX RSðSÞ ¼ T n lubMIN REP ðSÞ.2. lubMAX RSðSÞ ¼ T n glbMIN REP ðSÞ.

Proposition 3.10 implies that test cases which appear in every maximal re-dundant subset will never appear in any of the minimal representative set;

whereas test cases which appear in every minimal representative set will never

appear in any of the maxmial redundant subset.


4. Dividing strategies based on essential test cases

This section investigates the properties of those dividing strategies based on

essential test cases. We begin with a discussion on the properties of essentialtest cases. The completeness properties of the essentials dividing strategy (to be

defined later) are then studied. Finally, we discuss further dividing strategies

based on essential test cases.

In Fig. 2, we summarise the relationships between the set Ess of all essen-

tial test cases and the greatest lower bounds of various sets of representative

sets.

Lemma 4.1. Let S be the satisfiability relation from T to R. Then, we have1. Ess ¼ glbMIN REP ðSÞ ¼ glbREP ðSÞ.2. Ess � glbOPT REP ðSÞ.

Proof

(1) Immediate from Propositions 3.4 and 3.10(2), and glbMIN REP ðSÞ ¼glbREP ðSÞ.

(2) Since OPT REP ðSÞ � MIN REP ðSÞ, glbMIN REP ðSÞ � glbOPT REP ðSÞ.Therefore, it follows from part (1) that Ess � glbOPT REP ðSÞ. �

The following example shows that Ess and glbOPT REP ðSÞ may be different.

glb REP(S)

glb MIN_ REP(S)

glb OPT_REP(S)

OPT_REP(S)

T

P(T)

REP(S)

=

= Essφ

Fig. 2. Relationships among various greatest lower bounds.


Example 4.1. Suppose R ¼ fr1; r2; r3; r4g, T ¼ ft1; t2; t3; t4g, and the satisfiabilityrelation S from T to R is given in Table 4. It is easy to see that Ess ¼ ; andOPT REP ðSÞ ¼ fft1; t2g; ft2; t3g; ft2; t4gg. Thus, Ess 6¼ glb OPT REP ðSÞ ¼ ft2g.

The essentials dividing strategy decomposes the problem of finding repre-

sentative sets for a satisfiability relation SðT ;RÞ into two subproblems, namelyfinding representative sets for SðEss;ReqðEssÞÞ and for SðT n Ess;R n ReqðEssÞÞ.Obviously, the only representative set for SðEss;ReqðEssÞÞ is Ess. Since SðT nEss;R n ReqðEssÞÞ is a subset of SðT ;RÞ, it is then interesting to investigatewhether REP ðSÞ can be constructed from Ess and REP ðSðT n Ess;R nReqðEssÞÞÞ.Normally, we have Ess 6¼ T and ReqðEssÞ 6¼ R. However, two special situ-

ations, namely Ess ¼ T or ReqðEssÞ ¼ R, need to be considered. Obviously,Ess ¼ T implies ReqðEssÞ ¼ R, but not vice versa. The condition Ess ¼ T meansthat every test case in T is essential. Therefore, Ess ¼ T is the only represen-tative set, that is, REP ðSÞ ¼ fEssg. As a result, MIN REP ðSÞ ¼ OPT REP ðSÞ ¼fEssg. On the other hand, ReqðEssÞ ¼ R means that all essential test cases cancollectively satisfy R. Thus, Ess is the unique minimal representative set. As aresult, we have MIN REP ðSÞ ¼ OPT REP ðSÞ ¼ fEssg. In the rest of this sec-tion, the completeness properties of the essentials dividing strategy are inves-

tigated.

Lemma 4.2. Let S be the satisfiability relation from T to R, U ¼ T n Ess andRU ¼ R n ReqðEssÞ. Suppose U 6¼ ; and RU 6¼ ;.1. For any T1 2 REP ðSÞ, ðT1 n EssÞ 2 REP ðSðU ;RU ÞÞ.2. For any U1 2 REP ðSðU ;RUÞÞ, ðU1 [ EssÞ 2 REP ðSÞ.

Proof. Immediate from definition and Lemma 2.1. �

Lemma 4.3. Let S be the satisfiability relation from T to R, U ¼ T n Ess andRU ¼ R n ReqðEssÞ. Suppose U 6¼ ; and RU 6¼ ;. If U1 2 MIN REP ðSðU ;RU ÞÞ,then ðU1 [ EssÞ 2 MIN REP ðSÞ.

Table 4


r1 r2 r3 r4

t1 X X

t2 X X X

t3 X X

t4 X X


Proof. Let T1 ¼ U1 [ Ess. By Lemma 4.2(2), T1 2 REP ðSÞ. Suppose

T1 62 MIN REP ðSÞ. Then, there exists T2 2 REP ðSÞ such that T2 � T1 andT2 6¼ T1. It follows immediately from Lemma 4.2(1) that T2 n Ess 2REP ðSðU ;RU ÞÞ. By Proposition 3.3, Ess � T1; T2. Thus, T2 n Ess � T1 n Ess ¼ U1and T2 n Ess 6¼ U1 which contradicts U1 2 MIN REP ðSðU ;RU ÞÞ. The resultfollows. �

Lemma 4.4. Let S be the satisfiability relation from T to R, U ¼ T n Ess andRU ¼ R n ReqðEssÞ. Suppose U 6¼ ; and RU 6¼ ;. If T1 2 MIN REP ðSÞ, then(T1 n EssÞ 2 MIN REP ðSðU ;RU ÞÞ.

Proof. Let U1 ¼ T1 n Ess � U . It follows immediately from Lemma 4.2(1)that U1 2 REP ðSðU ;RU ÞÞ. Suppose U1 62 MIN REP ðSðU ;RU ÞÞ. Therefore,there exists U2 2 REP ðSðU ;RU ÞÞ such that U2 � U1 and U2 6¼ U1. ByLemma 4.2(2), U2 [ Ess 2 REP ðSÞ. Since U2 \ Ess ¼ ;, U2 [ Ess � U1 [ Ess ¼T1 and U2 [ Ess 6¼ T1 which contradicts T1 2 MIN REP ðSÞ. The result fol-lows. �

Proposition 4.1. Let S be the satisfiability relation from T to R, U ¼ T n Ess andRU ¼ R n ReqðEssÞ. Suppose U 6¼ ; and RU 6¼ ;. Then,1. REP ðSÞ ¼ REP ðSðU ;RUÞÞ � fEssg.2. MIN REP ðSÞ ¼ MIN REP ðSðU ;RU ÞÞ � fEssg.3. OPT REP ðSÞ ¼ OPT REP ðSðU ;RUÞÞ � fEssg.

Proof

(1) Immediate from definition.

(2) Immediate from Lemmas 4.3 and 4.4.

(3) Immediate from (2) and Lemma 3.1. �

Since fEssg is a singleton set, it follows from Proposition 4.1 that when

T n Ess and R n ReqðEssÞ are non-empty, there is a one-to-one correspondencebetween the following pairs of sets

• REP ðSÞ and REP ðSðT n Ess;R n ReqðEssÞÞÞ.• MIN REP ðSÞ and MIN REP ðSðT n Ess;R n ReqðEssÞÞÞ.• OPT REP ðSÞ and OPT REP ðSðT n Ess;R n ReqðEssÞÞÞ.as illustrated in Fig. 3. Thus, a representative set of SðT ;RÞ can be easilyconstructed by taking the union of Ess and a representative set of the sub-problem, SðT n Ess;R n ReqðEssÞÞ. Similarly, if a minimal or optimal repre-sentative set of SðT n Ess;R n ReqðEssÞÞ is used, a minimal or optimal

representative set of SðT ;RÞ will be constructed, respectively. Moreover, Prop-osition 4.1 guarantees that all representative sets, all minimal representative

sets, and all optimal representative sets of the original problem can be


constructed from their counterparts of the subproblems. Example 4.2 dem-onstrates an application of the essentials dividing strategy.

Example 4.2. Suppose A½i�:key6A½2� i�:key and A½i�:key6A½ð2� iÞ þ 1�:keyfor i ¼ 2; . . . ; bn=2c where A is an array of n elements. Consider the problem ofrearranging A such that the above property holds for i ¼ 1; . . . ; bn=2c.A possible solution of the problem is adapted from Aho et al. [2, p. 273]. The

pseudo-code and the control flow graph are shown in Figs. 4 and 5, respec-

tively.Suppose that the testing objective is branch coverage. As shown in Fig. 5,

there are 17 branches, namely, b1; . . . ; b17. Hence, the set of requirements R isfb1; . . . ; b17g.A possible test suite T consisting of seven test cases, ft1; t2; . . . ; t7g, is given in

Table 5, with the corresponding satisfiability relation Sðft1; . . . ; t7g; fb1; . . . ;b17gÞ given in Table 6.It is easy to check that Ess ¼ ft1g and ReqðEssÞ ¼ fb1; b2; b3; b4; b5;

b6; b12; b13; b16; b17g. The essential dividing strategy suggests to divide theoriginal satisfiability relation S to SðEss;ReqðEssÞÞ and Sðft2; t3; t4; t5; t6; t7g;fb7; b8; b9; b10; b11; b14; b15gÞ.As a result, the reduced satisfiability relation Sðft2; t3; t4; t5; t6; t7g;

fb7; b8; b9; b10; b11; b14; b15gÞ consists of 6 test cases and 7 requirements. In otherwords, after the reduction, we only need to find representative sets that satisfy 7

requirements rather than the original 17 requirements. Once the representative

sets, the minimal representative sets, and the optimal representative sets of the

REP(S)

REP(S)MIN_

REP(S)

REP(S(U,R ))U

OPT_

REP(S(U,R ))U

MIN_

REP(S(U,R ))U

correspondencef is a one-to-one

f(T’) = T’ UEss

OPT_

Fig. 3. Relationships between REPðSÞ and REPðSðU ;RU ÞÞ where U ¼ T n Ess and RU ¼R n ReqðEssÞ.


latter satisfiability relation are found, their counterparts of the original satis-fiability relation S can be constructed as suggested by Proposition 4.1.

Fig. 4. Pseudo-code for Example 4.2.


The following two propositions (Propositions 4.2 and 4.3) suggest that both

the essentials dividing strategy and the construction of minimal and optimalrepresentative sets of the original satisfiability relation from their counterparts

of the reduced satisfiability relations are not computationally expensive.

Proposition 4.2. Let S be the satisfiability relation from T to R. The time com-plexity for applying the essentials dividing strategy is OðjT jjRjÞ.

b11and

A[2r+1].key<A[2r].key

A[r].key>A[2r+1].key

r = 1

START

r <= size div 2 size == 2rA[r].key >

A[2r].key

r = size

swap(A[r],A[2r])

swap(A[r],A[2r])r = 2r

andA[r].key>A[2r].key

A[2r].key<=A[2r+1].key

swap(A[r],A[2r+1])r = 2r+1

r = size

b4

b5

b6

b3

Y

N

Y

N

YY

N

N

b16

b15

b14

b13

b12

STOP

b1

b2

b17

b7

b8 Y

b9

N

b10

Fig. 5. Control flow graph for Example 4.2.


Proof. From Proposition 3.2, a test case t is essential if and only if there exists arequirement r in R such that r can only be satisfied by t. Thus, Ess can beidentified by checking the set TestðrÞ for every requirement r in R. This processcan be implemented in OðjRjjT jÞ. Thus, the process of constructing

SðEss;ReqðEssÞÞ and SðT n Ess;R n ReqðEssÞÞ can be implemented in OðjT jjRjÞ.In other words, the essentials dividing strategy can be implemented in

OðjT jjRjÞ. �

Proposition 4.3. Let S be the satisfiability relation from T to R and Ess be the setof all essential test cases in T . The construction of all minimal (optimal) repre-sentative sets of SðT ;RÞ from all minimal (optimal) representative sets ofSðT n Ess;R n ReqðEssÞÞ can be implemented in OðjT jjMIN REP ðSðT ;RÞÞjÞ(OðjT jjOPT REP ðSðT ;RÞÞjÞ).

Proof. Let S1 ¼ SðT n Ess;R n ReqðEssÞÞ. The construction of all minimal rep-resentative sets of SðT ;RÞ from all minimal representative sets of S1 involves theunion of every minimal representative set of S1 and Ess. The union operationcan be implemented in OðjT jÞ [2]. The whole process can be implemented inOðjT jjMIN REP ðSðT ;RÞÞjÞ because jMIN REP ðSðT ;RÞÞj ¼ jMIN REP ðS1Þj.

Table 5

Test cases for Example 4.2

Test case A½i�:keya Size

t1 2, 1, 3, 4 4

t2 3, 2, 1, 4, 5, 6, 7 7

t3 6, 2, 1, 4, 5, 3, 6

t4 10, 1, 3, 4, 2, 6, 7, 8, 9, 5 10

t5 1, 2, 3 3

t6 5, 1, 3, 4, 2 5

t7 2, 1, 3 3

a The index i starts from 1 to size. For example, in t1, A½1�:key ¼ 2, A½2�:key ¼ 1, and so on.

Table 6


b1 b2 b3 b4 b5 b6 b7 b8 b9 b10 b11 b12 b13 b14 b15 b16 b17

t1 X X X X X X X X X X


t3 X X X X X X X X X X X X

t4 X X X X X X X X X X X X X X

t5 X X X X X X X X


t7 X X X X X X X


Similarly, all optimal representative sets of SðT ;RÞ can be constructed from alloptimal representative sets of S1 in OðjT jjOPT REP ðSðT ;RÞÞjÞ. �

After applying the essentials dividing strategy, we still need to solve thesubproblem SðT n Ess;R n ReqðEssÞÞ. If T1 is a set of redundant test cases (thatis, T1 � ðT n EssÞ) such that ReqðT1Þ � ReqðEssÞ, T1 will never be part of anyminimal representative set. Hence, T1 is not in any optimal representative set.These results are stated in Lemma 4.5.

Lemma 4.5. Let S be the satisfiability relation from T to R. Suppose Ess 6¼ ; andEss 6¼ T . For any T1 � ðT n EssÞ such that ReqðT1Þ � ReqðEssÞ, we have1. T1 \ ð

SU2MIN REP ðSÞ UÞ ¼ ;.

2. T1 \ ðS

U2OPT REP ðSÞ UÞ ¼ ;.

Proof

(1) For any t 2 T1 � ðT n EssÞ, t is not essential. Since ReqðtÞ � ReqðEssÞ andEss is included in every minimal representative set after Lemma 4.1,t 62 T2 for any T2 2 MIN REP ðSÞ. Therefore, t 62 lubMIN REP ðSÞ. Hence,T1 \ ð


(2) Immediate from (1) becauseS

U2OPT REP ðSÞ U �S

U2MIN REP ðSÞ U . �

The essentials dividing strategy based on Proposition 4.1 can be further

enhanced by Lemma 4.5. Suppose T1 is a subset of T n Ess that satisfiesReqðT1Þ � ReqðEssÞ. We can simply discard T1 in the construction of minimalor optimal representative sets of S. Further relationship between REP ðSÞ andREP ðSðT n T1;RÞÞ is given in Proposition 4.4.

Proposition 4.4. Let S be the satisfiability relation from T to R. Suppose thatEss 6¼ ; and Ess 6¼ T . For any T1 � ðT n EssÞ such that ReqðT1Þ � ReqðEssÞ, wehave1. REP ðSÞ ¼ REP ðSðT n T1;RÞÞ � P ðT1Þ.2. MIN REP ðSÞ ¼ MIN REP ðSðT n T1;RÞÞ.3. OPT REP ðSÞ ¼ OPT REP ðSðT n T1;RÞÞ.

Proof(1) (� part) Let T2 2 REP ðSÞ. Since T2 ¼ ðT2 n T1Þ [ ðT2 \ T1Þ and T2 \ T1 � T1,

it suffices to prove that T2 n T1 2 REP ðSðT n T1;RÞÞ. Since Ess � ðT n T1Þ andEss � T2 by Proposition 3.3, we have Ess � ðT2 n T1Þ. Thus, ReqðEssÞ �ReqðT2nT1Þ. Moreover, since ReqðT2\T1Þ�ReqðT1Þ and ReqðT1Þ�ReqðEssÞ,we have ReqðT2 \ T1Þ � ReqðT2 n T1Þ. Hence, ReqðT2 n T1Þ ¼ ReqðT2Þ ¼ Ras T2 ¼ ðT2 n T1Þ [ ðT2 \ T1Þ. Thus, T2 n T1 2 REP ðSðTn T1;RÞÞ. Therefore,the result follows.


(� part) For any T2 2 REP ðSðT n T1;RÞÞ, T2 [ T 0 2 REP ðSÞ for any T 0 � T1.Hence, the result follows.

(2) (� part) Let T2 be any minimal representative set of SðT ;RÞ. That is,T2 2 MIN REP ðSÞ. By Lemma 4.5(1), we have T1 \ ð


Therefore, T1 \ T2 � T1 \ ðS

U2MIN REP ðSÞ UÞ ¼ ;. This implies that T2 �ðT n T1Þ. It is then easy to see that T2 2 MIN REP ðSðT n T1;RÞÞ. Thus, wehave MIN REP ðSÞ � MIN REP ðSðT n T1;RÞÞ.(� part) Immediate from definitions.


5. Dividing strategy based on essential subsets

This section investigates the properties of a complete dividing strategy based

on essential subsets. One of the advantages of having complete dividing

strategies with respect to minimal representative sets is that we can choose the

minimal representative set that best suits further testing constraints or con-

siderations. It is then desirable to have a dividing strategy to keep all those

elements in T that belong to some minimal representative sets and discard therest. In other words, the dividing strategy reduces SðT ;RÞ to SðlubMIN REP ðSÞ;RÞ. Hence, the minimal representative sets of the subproblem are also minimalwith respect to the original problem.

However, there are two major problems. The first problem is how to find

lubMIN REP ðSÞ without constructing MIN REP ðSÞ first. It should be notedthat if we know MIN REP ðSÞ, we should have already solved the test suitereduction problem. The second problem is whether this dividing strategy is

complete with respect to minimal and optimal representative sets. In this sec-

tion, we use the properties of minimal essential subsets to develop the solutions

to these two problems.First, we will prove that lubMIN REP ðSÞ ¼ lubMIN ESðSÞ. This result and

the property MIN ESðSÞ ¼ MIN TSðSÞ (Proposition 3.8) suggest an efficientway to compute lubMIN REP ðSÞ through the computation of lubMIN TSðSÞ.Secondly, we will show that the dividing strategy that reduces SðT ;RÞ toSðlubMIN ESðSÞ;RÞ is complete with respect to minimal and optimal repre-sentative sets.

5.1. Further properties of minimal essential subsets

In this section, we investigate the relationships between MIN REP ðSÞ andMIN ESðSÞ. One of the main results is that lubMIN REP ðSÞ ¼ lubMIN ESðSÞ(to be stated as Proposition 5.3).

Obviously, a minimal essential subset is not necessarily a representative set,and a minimal representative set may not be an essential subset. Interestingly,


the following proposition shows that test cases from all minimal essential

subsets together form a representative set of S.

Proposition 5.1. Let S be the satisfiability relation from T to R. Then,lubMIN ESðSÞ 2 REP ðSÞ.

Proof. For every r 2 R, by Lemma 3.2, there exists T1 2 MIN ESðSÞ such thatT1 � TestðrÞ. Thus, r 2 ReqðT1Þ for some T1 2 MIN ESðSÞ. Therefore, r 2ReqðlubMIN ESðSÞÞ. The result follows. �

Now, lubMIN ESðSÞ is a representative set. It is interesting to further ob-serve that, the intersection of a representative set of S and lubMIN ESðSÞ is stilla representative set of S.

Proposition 5.2. Let S be the satisfiability relation from T to R. Then, we haveðT1 \ lubMIN ESðSÞÞ 2 REP ðSÞ for any T1 2 REP ðSÞ.

Proof. For any r 2 R, let t be a test case in T1 such that it satisfies r. Ift 2 lubMIN ESðSÞ, we are done. If t 62 lubMIN ESðSÞ, TestðrÞ is an essentialsubset but not a minimal essential subset after Propositions 3.6 and 3.8. Hence,there exists a minimal essential subset in the form of Testðr1Þ such that Testðr1Þis a proper subset of TestðrÞ and r1 6¼ r. Since T1 is a representative set, thereexists t1 2 T1 such that t1 satisfies r1, that is, t1 2 Testðr1Þ. Therefore,t1 2 lubMIN ESðSÞ because Testðr1Þ 2 MIN ESðSÞ. Since Testðr1Þ � TestðrÞ, t1satisfies r. Hence, the result follows. �

An important implication of Proposition 5.2 is that, if T1 contains elementsother than lubMIN ESðSÞ, T1 is not minimal as stated in Corollary 5.1. Thisfurther suggests that it may be more effective to use lubMIN ESðSÞ instead of Tto compute the minimal representative sets of S.

Corollary 5.1. Let S be the satisfiability relation from T to R. For anyT1 2 REP ðSÞ, if T1 contains elements in T n lubMIN ESðSÞ, T1 62 MIN REP ðSÞ.

Proof. Since T1 2 REP ðSÞ, ðT1 \ lubMIN ESðSÞÞ 2 REP ðSÞ by Proposition 5.2.Since T1 contains elements in T n ðlubMIN ESðSÞÞ, ðT1 \ lubMIN ESðSÞÞ 6¼ T1.Hence, T1 62 MIN REP ðSÞ. �

Now, we are going to prove the following lemma, which is part of our major

result in this section.

Lemma 5.1. Let S be the satisfiability relation from T to R. Then, we havelubMIN REP ðSÞ � lubMIN ESðSÞ.


Proof. For any T1 2 MIN REP ðSÞ, it follows from Corollary 5.1 that T1 doesnot contain elements in T n lubMIN ESðSÞ. Thus, T1 � lubMIN ESðSÞ. In otherwords, lubMIN ESðSÞ is an upper bound of MIN REP ðSÞ. Therefore,lubMIN REP ðSÞ � lubMIN ESðSÞ. �

Lemma 5.1 means that test cases in a minimal representative set must appear

in some of the minimal essential subsets as lubMIN REP ðSÞ ¼S

U2MIN REP ðSÞ Uand lubMIN ESðSÞ ¼

SV 2MIN ESðSÞ V . Now, we are going to show that test cases

in a minimal essential subset also appear in some minimal representative sets,

that is, lubMIN ESðSÞ � lubMIN REP ðSÞ. We need the following lemma toestablish this result.

Lemma 5.2. Let S be the satisfiability relation from T to R and t 2 T1 such thatReqðtÞ 6¼ R, where T1 2 MIN ESðSÞ. Then, there exists T2 2 MIN REP ðSðT n ftg;R n ReqðtÞÞÞ such that ReqðtÞ 6� ReqðT2Þ.

Proof. Given T1 2 MIN ESðSÞ. By Propositions 3.7 and 3.8, there exists a r 2 Rsuch that T1 ¼ TestðrÞ where TestðrÞ 2 MIN TSðSÞ. Since R 6¼ ReqðtÞ and T is atest suite for R, T n ftg 6¼ ;. Obviously, ðT n ftgÞ 2 REP ðSðT n ftg;R n ReqðtÞÞÞwhich is, thus, non-empty.

We proceed the proof by constructing an A 2 MIN REP ðSðT n ftg;R n ReqðtÞÞÞ such that r 62 ReqðAÞ.Since REP ðSðT n ftg;R n ReqðtÞÞÞ is non-empty, MIN REP ðSðT n ftg;

R n ReqðtÞÞÞ is also non-empty. Hence, there exists A1 2 MIN REP ðSðT n ftg;R n ReqðtÞÞÞ. Two cases arise.1. If ReqðtÞ 6� ReqðA1Þ, the process is stopped and A1 is the required minimalrepresentative set.

2. If ReqðtÞ � ReqðA1Þ, there exists t0 2 A1 such that r 2 Reqðt0Þ.Since A1 2 MIN REP ðSðT n ftg;R n ReqðtÞÞÞ, ðR n ReqðtÞÞ \ Reqðt0Þ 6¼ ;.Let ðR n ReqðtÞÞ \ Reqðt0Þ ¼ fr01; . . . ; r0hg. Note that t 62 Testðr0iÞ for all i ¼1; . . . ; h. Since TestðrÞ 2 MIN TSðSÞ, there exists t0i 2 Testðr0iÞ such thatt0i 62 TestðrÞ for every r0i where i ¼ 1; . . . ; h. Hence, r 62 Reqðft01; . . . ; t0hgÞ andðR n ReqðtÞÞ \ Reqðt0Þ � Reqðft01; . . . ; t0hgÞ.Let B ¼ ðA1 n ft0gÞ [ ft01; . . . ; t0hg. Since all requirements in R n ReqðtÞ satis-fied by t0 are collectively satisfied by ft01; . . . ; t0hg, R n ReqðtÞ � ReqðBÞ. Hence,we successfully construct a set B 2 REP ðSðT n ftg;R n ReqðtÞÞÞ by replacingt0 with ft01; . . . ; t0hg in A1. In other words, the number of test cases that cansatisfy r in the set B is less than that in A1 by 1. Obviously, there exists anA2 2 MIN REP ðSðT n ftg;R n ReqðtÞÞÞ such that A2 � B. Thus, the numberof test cases satisfying r in A2 is less than or equal to that in B which, in turn,is less than that in A1 by at least 1.


By repeating the above steps, we obtain the following sequence of minimal

representative sets of SðT n ftg;R n ReqðtÞÞA1 ! � � � ! Ai ! Aiþ1 ! � � � ! Ak

Note that the number of test cases satisfying r in Aiþ1 is less than that in

Ai by at least 1. The process will eventually stop because there are only

finite number of test cases satisfying r in A1. When it stops at Ak, Ak 2MIN REP ðSðT n ftg;R n ReqðtÞÞÞ such that r 62 ReqðAkÞ. Now, since r 2ReqðtÞ and r 62 ReqðAkÞ, ReqðtÞ 6� ReqðAkÞ.

Hence, the result follows. �

We are now ready to prove the second part of the major result in the next

lemma, that is, elements in a minimal essential subset of S must appear in someminimal representative sets of S.

Lemma 5.3. Let S be the satisfiability relation from T to R. Then,lubMIN ESðSÞ � lubMIN REP ðSÞ.

Proof. For any t 2 lubMIN ESðSÞ ¼S

V 2MIN ESðSÞ V , t 2 V for some V 2MIN ESðSÞ. Two cases arise. For the case when R ¼ ReqðtÞ. Obviously,ftg 2 MIN REP ðSÞ and hence, t 2

SU2MIN REP ðSÞ U ¼ lubMIN REP ðSÞ.

Now consider the case when R n ReqðtÞ 6¼ ;. Since T is a test suite for R, wehave T n ftg 6¼ ; and T n ftg 2 REP ðSðT n ftg;R n ReqðtÞÞÞ. By Lemma 5.2,there exists T1 2 MIN REP ðSðT n ftg;R n ReqðtÞÞÞ such that ReqðtÞ 6� ReqðT1Þ.Thus, T1 [ ftg 2 REP ðSÞ. Moreover, T1 [ ftg 2 MIN REP ðSÞ because T1 is aminimal representative set of SðT n ftg;R n ReqðtÞÞ and t is the only test case tosatisfy those requirements in ReqðtÞ which are not satisfied by T1. Therefore,t 2

SU2MIN REP ðSÞ U ¼ lubMIN REP ðSÞ. Hence, the result follows. �

It follows immediately from Lemmas 5.1 and 5.3 that the least upper bounds

of MIN ESðSÞ and MIN REP ðSÞ are equal.

Proposition 5.3. Let S be the satisfiability relation from T to R. Then, we havelubMIN ESðSÞ ¼ lubMIN REP ðSÞ.

Proposition 5.3 describes a relationship between the minimal essential

subsets and minimal representative sets. It also provides a basis for a process to

find the maximal redundant subsets from the essential test cases. The rest of

this section is devoted to this discussion.

Proposition 5.4. Let S be the satisfiability relation from T to R. Then, we haveglbMAX RSðSÞ ¼ T n lubMIN ESðSÞ.

Proof. Immediate from Propositions 3.10 and 5.3. �


Corollary 5.2. Let S be the satisfiability relation from T to R. For any T1 2MAX RSðSÞ, ðT n lubMIN ESðSÞÞ � T1 � ðT n EssÞ.

Proof. Immediate from Propositions 5.4 and 3.4 because glbMAX RSðSÞ ¼TU2MAX RSðSÞ U and lubMAX RSðSÞ ¼

SV 2MAX RSðSÞ V . �

With Corollary 5.2, a recommended way to find a maximal redundant

subset T1 is as follows. First, we set T1 to T n lubMIN ESðSÞ. Then, we removeelements from T n Ess to T1 one by one until the remaining test cases in T n Essare no longer able to satisfy R.

5.2. A dividing strategy based on minimal essential subsets

As proved in Propositions 5.1 and 5.3, the least upper bound of the set of all

minimal essential subsets is a representative set as well as the least upper bound

of the set of all minimal representative sets. Hence, it is natural to consider the

dividing strategy that decomposes SðT ;RÞ to SðlubMIN ESðSÞ;RÞ. In fact, thisdividing strategy is complete with respect to minimal and optimal represen-tative sets as shown in the following proposition.

Proposition 5.5. Let S be the satisfiability relation from T to R and W ¼lubMIN ESðSÞ. Then,1. REP ðSÞ ¼ REP ðSðW ;RÞÞ � P ðT n W Þ.2. MIN REP ðSÞ ¼ MIN REP ðSðW ;RÞÞ.3. OPT REP ðSÞ ¼ OPT REP ðSðW ;RÞÞ.

Proof

(1) (� part) Immediate from definitions.(� part) Suppose that T1 is a representative set of SðT ;RÞ. Let T2 ¼ T1 \ W

and T3 ¼ T1 n W . Clearly, T1 ¼ T2 [ T3 and T2 \ T3 ¼ ;. By Proposition 5.2, T2 isa representative set of SðT ;RÞ. Moreover, since T2 is a subset of W , T2 is arepresentative set of SðW ;RÞ. That is, T2 2 REP ðSðW ;RÞÞ. Since T3 � ðT n W Þ,T1 ¼ T2 [ T3 2 ðREP ðSðW ;RÞÞ � PðT n W ÞÞ. The result follows.(2) (� part) Immediate from definitions.(� part) If T1 is a minimal representative set of SðT ;RÞ, T1 �

lubMIN REP ðSÞ. Therefore, by Proposition 5.3, T1 � lubMIN ESðSÞ ¼ W . Theresult follows.


Let RW denote the set of all requirements in R such that TestðrÞ is a mini-mal essential subset, that is, RW ¼ fr 2 R : TestðrÞ 2 MIN ESðSÞg. Since RW

is a subset of R, it is more intuitively appealing to reduce SðT ;RÞ to


SðlubMIN ESðSÞ;RW Þ instead of SðlubMIN ESðSÞ;RÞ. Nevertheless, since RW

may not be equal to R, we may wonder whether there is a subset of the test suitethat can collectively satisfy all requirements in RW but not all requirements in

R. The following lemma shows that this situation will never happen. In fact, ifa subset T1 of T satisfies all requirements in RW , T1 will satisfy all requirementsin R.

Lemma 5.4. Let S be the satisfiability relation from T to R, W ¼ lubMIN ESðSÞand RW ¼ fr 2 R : TestðrÞ 2 MIN ESðSÞg. If T1 is a subset of T such thatReqðT1Þ ¼ RW , then T1 2 REP ðSÞ.

Proof. Let T1 � T be such that ReqðT1Þ ¼ RW . By Proposition 3.8, RW ¼fr 2 R : TestðrÞ 2 MIN TSðSÞg. For any r0 2 R n RW , Testðr0Þ 62MIN TSðSÞ. Thatis, there exists r 2 RW such that TestðrÞ � Testðr0Þ and TestðrÞ 6¼ Testðr0Þ.Thus, test cases satisfying r also satisfy r0. Therefore, ReqðT1Þ ¼ R and T1 2REP ðSÞ. �

With Lemma 5.4, we are now ready to prove that the dividing strategy that

reduces SðT ;RÞ to SðlubMIN ESðSÞ;RW Þ is complete with respect to minimaland optimal representative sets.

Proposition 5.6. Let S be the satisfiability relation from T to R, W ¼lubMIN ESðSÞ and RW ¼ fr 2 R : TestðrÞ 2 MIN ESðSÞg. Then, we have1. REP ðSÞ ¼ REP ðSðW ;RW ÞÞ � P ðT n W Þ.2. MIN REP ðSÞ ¼ MIN REP ðSðW ;RW ÞÞ.3. OPT REP ðSÞ ¼ OPT REP ðSðW ;RW ÞÞ.

Proof

(1) (� part) Immediate from definitions and Lemma 5.4.

(� part) For any T1 2 REP ðSÞ, let T2 ¼ T1 \ W and T3 ¼ T1 n W . Clearly,T1 ¼ T2 [ T3 and T2 \ T3 ¼ ;. By Proposition 5.2, T2 2 REP ðSÞ. Moreover, sinceT2 � W , T2 2 REP ðSðW ;RW ÞÞ. Since T3 � ðT n W Þ, T1 ¼ T2 [ T3 2 ðREP ðSðW ;RW ÞÞ � PðT n W ÞÞ. The result follows.(2) (� part) Immediate from definitions and Lemma 5.4.

(� part) For any T1 2 MIN REP ðSÞ, T1 � lubMIN REP ðSÞ ¼lubMIN ESðSÞ ¼ W after Proposition 5.3. Therefore, T1 2 MIN REP ðSðW ;RW ÞÞ. The result follows.(3) Immediate from (2) and Lemma 3.1. �

The results of Proposition 5.6 are summarised in the schematic diagram in

Fig. 6. The dividing strategy that reduces SðT ;RÞ to SðlubMIN ESðSÞ;RW Þ isreferred to as the minimal essential subset dividing strategy. This dividing


strategy eliminates unnecessary test cases and requirements. Example 5.1 below

demonstrates an application of the minimal essential subset dividing strategy tothe problem presented in Example 4.2.

Example 5.1. Refer to Example 4.2. Then, we have MIN ESðSÞ ¼MIN TSðSÞ ¼ fft1g; ft2; t5g; ft3; t4gg, W ¼ lubMIN REP ðSÞ ¼ lubMIN ESðSÞ ¼lubMIN TSðSÞ ¼ ft1; . . . ; t5g, and RW ¼ fr 2 R : TestðrÞ 2 MIN ESðSÞg ¼ fb10;b11; b13; b14; b15g. The minimal essential subset dividing strategy will reduce theoriginal satisfiability relation to Sðft1; . . . ; t5g, fb10; b11; b13; b14; b15gÞ, whichhas 5 test cases and 5 requirements (note that the original satisfiability rela-tion has 7 test cases and 17 requirements). Hence, finding representative sets,

minimal representative sets, and optimal representative sets should be easier

for the reduced satisfiability relation than for the original satisfiability rela-

tion.

Proposition 5.6 suggests that the sets of all minimal (optimal) representative

sets of the original satisfiability relation S and the reduced satisfiability relationSðW ;RW Þ are the same. Hence, after the minimal (optimal) representative setsof the reduced satisfiability relation SðW ;RW Þ are found, their counterpartsof the original satisfiability relation SðT ;RÞ can be determined. In fact,MIN REP ðSÞ ¼ MIN REP ðSðW ; RW ÞÞ ¼ fft1; t2; t3g; ft1; t2; t4g; ft1; t3; t5g; ft1; t4;t5gg.

REP(S)

W

OPT_REP(S) =

REP(S(W,R ))WOPT_

REP(S(W,R ))WMIN_

MIN_ REP(S) =

REP(S(W,R ))

Fig. 6. Relationships between REPðSÞ and REPðSðW ;RW ÞÞ where W ¼ lubMIN ESðSÞ and RW ¼fr 2 R : TestðrÞ 2 MIN ESðSÞg.


The following two propositions (Propositions 5.7 and 5.8) show that both

the minimal essential subset dividing strategy and the construction of minimal

and optimal representative sets of the original satisfiability relation from their

counterparts of the subproblem are not computationally expensive.

Proposition 5.7. Let S be the satisfiability relation from T to R. The time com-plexity for applying the minimal essential subset dividing strategy is OðjRj2jT jÞ.

Proof. Computation of lubMIN ESðSÞ involves two steps: (1) the computationof MIN ESðSÞ and (2) the computation of the union of all elements ofMIN ESðSÞ. For the first step, it involves the computation of MIN TSðSÞ be-cause MIN ESðSÞ ¼ MIN TSðSÞ by Proposition 3.8. Computation of TSðSÞ canbe implemented in OðjRjjT jÞ. For any two elements A, B of TSðSÞ (A, B aresubsets of T ), the checking of A � B can be implemented in OðjT jÞ [2]. Hence,computation of the minimal elements of TSðSÞ can be implemented inOðjRj2jT jÞ because jTSðSÞj6 jRj. Thus, the first step can be implemented inOðjRj2jT jÞ. For the second step, the union of all elements of MIN ESðSÞ can beimplemented in OðjRjjT jÞ because (a) MIN ESðSÞ ¼ MIN TSðSÞ contains atmost jRj subsets of T and (b) the union of any two elements of MIN ESðSÞ canbe implemented in OðjT jÞ [2]. As a result, the computation of lubMIN ESðSÞcan be implemented in OðjRj2jT jÞ. Once MIN TSðSÞ is known, RW can be de-

termined in OðjRjÞ. Hence, the minimal essential subset dividing strategy can beimplemented in OðjRj2jT jÞ time. �

Proposition 5.8. Let S be the satisfiability relation from T to R, W ¼lubMIN ESðSÞ and RW ¼ fr 2 R : TestðrÞ 2 MIN ESðSÞg. The construction ofMIN REP ðSðT ;RÞÞ from MIN REP ðSðW ;RW ÞÞ (or, OPT REP ðSðT ;RÞÞ fromOPT REP ðSðW ;RW ÞÞ) can be implemented in constant time.

Proof. By Proposition 5.6, the set of all minimal (optimal) representative sets of

SðT ;RÞ and that of SðW ;RW Þ are equal. The computation ofMIN REP ðSðT ;RÞÞfrom MIN REP ðSðW ;RW ÞÞ (or, OPT REP ðSðT ;RÞÞ from OPT REP ðSðW ;RW ÞÞ)can be implemented in constant time. �

We have two major results about the completeness properties of two di-

viding strategies. First, Proposition 4.1 is about the essentials dividing strategythat decomposes SðT ;RÞ to SðEss;ReqðEssÞÞ and SðT n Ess;R n ReqðEssÞÞ.Secondly, Proposition 5.6 is about the minimal essential subset dividing

strategy that reduces SðT ;RÞ to SðlubMIN ESðSÞ;RW Þ, where RW is the set of

those requirements r 2 R such that TestðrÞ is a minimal representative set. SinceEss is a subset of lubMIN ESðSÞ, it is interesting to investigate the relationshipbetween these two dividing strategies. In fact, neither of them is a special case

of the other, as illustrated in Example 5.2.


Example 5.2. Suppose R ¼ fr1; . . . ; r6g and T ¼ ft1; . . . ; t6g. The satisfiabilityrelation S from T to R is given in Table 7. It is easy to see that Ess ¼ ft5g,MIN ESðSÞ ¼ fft1; t2g; ft2; t3g; ft3; t4g; ft5gg, lubMIN ESðSÞ ¼ ft1; . . . ; t5g, andRW ¼ fr1; r2; r3; r4g. Now, SðEss;ReqðEssÞÞ ¼ Sðft5g; fr4gÞ, SðT n Ess;R nReqðEssÞÞ ¼ SðT n ft5g;R n fr4gÞ, and SðlubMIN ESðSÞ;RW Þ ¼ Sðft1; . . . ; t5g;fr1; r2; r3; r4gÞ. Hence, these three subproblems, SðEss;ReqðEssÞÞ, SðT n Ess;R n ReqðEssÞÞ, and SðlubMIN ESðSÞ;RW Þ, are different from one another.

6. Conclusions

In this paper, we investigate the relationships between various sets of rep-

resentative sets. This includes the sets of all representative sets, all minimal

representative sets and all optimal representative sets. Our focus is on theminimal representative sets and the optimal representative sets because they are

the candidates for the most cost effective representative sets.

Some properties of essential test cases and redundant test cases are also

presented. Essential test cases are those that must be included in every repre-

sentative set, whereas redundant test cases are those that can be removed

without rendering the satisfaction of all requirements in R.Based on the concepts of essential test cases and redundant test cases, the

concepts of essential subsets and redundant subsets are developed. During theprocess of finding representative sets, essential subsets should be preserved but

redundant subsets should be removed from the original test suite. Since any set

which contains an essential subset is essential, minimal essential subsets are

studied. Similarly, the properties of the maximal redundant subsets are studied

because subsets of a redundant subset are redundant.

The theoretical results in this paper clarify and provide additional insights

into the structure of the set of representative sets. For example, the result

lubMIN REP ðSÞ ¼ lubMIN ESðSÞ (Proposition 5.3) means that elements foundin any minimal representative sets can also be found in lubMIN ESðSÞ but not

Table 7


r1 r2 r3 r4 r5 r6

t1 X X

t2 X X X X

t3 X X X X

t4 X X

t5 X

t6 X X


elsewhere. Hence, if software testers are interested in the minimal representa-

tive sets, they should consider lubMIN ESðSÞ instead of the test suite T .In addition, these theoretical results provide a basis for the development of

two complete dividing strategies for computing the minimal and optimal rep-resentative sets. First, Proposition 4.1 shows the completeness of the essentials

dividing strategy, which decomposes SðT ;RÞ to SðEss;ReqðEssÞÞ and

SðT n Ess;R n ReqðEssÞÞ. Secondly, Proposition 5.6 shows the completeness ofthe minimal essential subset dividing strategy, which reduces SðT ;RÞ toSðlubMIN ESðSÞ;RW Þ, where RW is the set of those requirements r 2 R such thatTestðrÞ is a minimal representative set. We also illustrate the applicability ofthese two complete dividing strategies in Examples 4.2 and 5.1.

This paper concentrates on the dividing strategies that are complete withrespect to the minimal and optimal representative sets, from the perspective of

essential test cases and essential subsets. On the other hand, the concepts of

redundant test cases and redundant subsets may give further insights for de-

veloping further complete dividing strategies for test suite reduction. It is

worthwhile to pursue further research along this direction.

References

[1] A.V. Aho, J.E. Hopcroft, J.D. Ullman, The Design and Analysis of Computer Algorithms,

Addison-Wesley Publishing Company, Reading, Massachusetts, USA, 1974.

[2] A.V. Aho, J.E. Hopcroft, J.D. Ullman, Data Structures and Algorithms, Addison-Wesley

Publishing Company, Reading, Massachusetts, USA, 1983.

[3] T.Y. Chen, M.F. Lau, Dividing strategies for the optimization of a test suite, Information

Processing Letters 60 (3) (1996) 135–141.

[4] T.Y. Chen, M.F. Lau, A new heuristic for test suite reduction, Information and Software

Technology 40 (5–6) (1998) 347–354.

[5] T.Y. Chen, M.F. Lau, A simulation study on some heuristics for test suite reduction,

Information and Software Technology 40 (13) (1998) 777–787.

[6] Y.-F. Chen, D.S. Rosenblum, K.-P. Vo, TestTube: A system for selective regression testing, in:

Proceedings of the 16th International Conference on Software Engineering, IEEE Computer

Society Press, 1994, pp. 211–220.

[7] R.A. DeMillo, A.J. Offutt, Constraint-based automatic test data generation, IEEE Transac-

tions on Software Engineering 17 (9) (1991) 900–910.

[8] T.L. Graves, M.J. Harrold, J.-M. Kim, A. Porter, G. Rothermel, An empirical study of

regression test selection techniques, ACM Transactions on Software Engineering and

Methodology 10 (2) (2001) 184–208.

[9] R. Gupta, M.L. Soffa, Employing static information in the generation of test cases, Software

Testing, Verification and Reliability 3 (1993) 29–48.

[10] M.J. Harrold, R. Gupta, M.L. Soffa, A methodology for controlling the size of a test suite,

ACM Transactions on Software Engineering and Methodology 2 (3) (1993) 270–285.

[11] D.S. Johnson, Approximation algorithms for combinatorial problems, Journal of Computer

and System Sciences 9 (3) (1974) 256–278.

[12] B. Korel, Automated software test data generation, IEEE Transactions on Software

Engineering 16 (8) (1990) 870–879.


[13] H.K.N. Leung, L. White, Insights into regression testing, in: Proceedings of International

Conference on Software Maintenance, IEEE Computer Society Press, 1989, pp. 60–69.

[14] A.K. Onoma, W.T. Tsai, M.H. Poonawala, H. Suganuma, Regression testing in an industrial

environment, Communications of the ACM 41 (5) (1998) 81–86.

[15] C.V. Ramamoorthy, S.-B.F. Ho, W.T. Chen, On the automated generation of program test

data, IEEE Transactions on Software Engineering SE-2 (4) (1976) 293–300.

[16] S. Rapps, E.J. Weyuker, Selecting software test data using data flow information, IEEE

Transactions on Software Engineering 11 (4) (1985) 367–375.

[17] G. Rothermel, M.J. Harrold, A safe, efficient regression test selection technique, ACM

Transactions on Software Engineering and Methodology 6 (2) (1997) 173–210.

[18] W.T. Tsai, D. Volovik, T.F. Keefe, Automated test case generation for programs specified by

relational algebra queries, IEEE Transactions on Software Engineering 16 (3) (1990) 316–324.

[19] W.E. Wong, J.R. Horgan, S. London, A.P. Mathur, Effect of test set minimization on fault

detection effectiveness, Software––Practice and Experience 28 (4) (1998) 347–369.

[20] W.E. Wong, J.R. Horgan, A.P. Mathur, A. Pasquini, Test set size minimization and fault

detection effectiveness: A case study in a space application, The Journal of Systems and

Software 48 (1999) 79–89.


Documents

On the divide-and-conquer approach towards test suite reduction