25
1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

  • View
    215

  • Download
    0

Embed Size (px)

Citation preview

Page 1: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

1

Software Testing and Quality Assurance

Lecture 25 – Testing Interactions

(Chapter 6)

Page 2: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

2

Lecture Outline Sampling Test Cases Orthogonal array testing

Page 3: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

3

Sampling test cases Population: all possible test cases that

can be executed A sample is a subset of a population

that has been selected based on some probability distribution.

Page 4: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

4

Sampling test cases Possibilities for determining which test cases

to select Based on probability distribution

A probability distribution defines, for each data value in a population, a set of allowable values and the probability that value will be selected.

Uniform probability distribution Each value in the population is assigned the same

selection probability For example, Probability distribution based on the

use profile

Page 5: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

5

Sampling test cases (cont...) A stratified sample is a set of samples

in which each sample represents a specific subpopulation Example: select a sample of test cases

from the use cases of each actor Stratified sample tests are selected from

a series of categories.

Page 6: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

6

Sampling test cases (cont...) Use of random number generator in

sampling Advantage: all values have equal

probability Disadvantage: test cases cannot be

reproduced (you may let the test driver record the generated values)

Class family: a set of classes related by inheritance.

Page 7: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

7

Sampling test cases: Orthogonal array testing (OATS)

Orthogonal arrays provide a specific sampling technique that seeks to limit the explosion by defining pair-wise combinations of a set of interacting objects.

An orthogonal array is an array of values in which each column represents a factor (a variable in an experiment).

Page 8: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

8

Sampling test cases: Orthogonal array testing (OATS)

It represents a specific class family (a class and its children)

E.g. 3 factors with 3 levels each (27 possibilities), with pair-wise only 9 possibilities

Page 9: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

9

Sampling test cases: OATS Example Decide how many independent variables will

be tested for interaction. This will map to the Factors of the array.

Decide the maximum number of values that each independent variable will take on. This will map to the Levels of the array.

Page 10: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

10

Sampling test cases: OATS Example Find a suitable orthogonal array. A

suitable array is one that has at least as many Factors as needed from Step 1 and has at least as many levels for each of those factors as decided in Step 2.

Map the Factors and values onto the array.

Transcribe the Runs into test cases.

Page 11: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

11

Sampling test cases: OATS example Number of states:

Class A: 2 Class B: 3 Class P: 1 Class C: 2 Class D: 3 Class E: 3

* State transition diagrams are not shown

Page 12: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

12

Sampling test cases: OATS example (cont...)

Step 1: Identify all factors: sending hierarchy, receiving hierarchy, parameter position in the message (six factors in the example: class A hierarchy, class P hierarchy, class C hierarchy and factors associated with each class hierarchy)

Page 13: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

13

Sampling test cases: OATS example (cont...)

Step 2: Determine levels for each factor by considering the set of possible values: One factor has one level: the parameter

class family only has one member: P Two factors have a maximum of two levels;

B and C. Three factors have a maximum of three

levels: A, D and E.

Page 14: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

14

Sampling test cases: OATS example (cont...)

Step 3: Locate a standard orthogonal array that fits the problem.

6 factors 3 levels

Page 15: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

15

Sampling test cases: OATS example (cont...)

Page 16: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

16

Sampling test cases: OATS example (cont...)

Step 4: Establish a mapping from each factor onto the integers in the array so that standard array can be interpreted The first column in L18 can be used to represent

the sender class family which has two classes: A and B (1 corresponds to A class and 2 corresponds to class B)

The second column in L18 when there is a difference in the number of levels (class A has 2 states and class B has 3 states).

Page 17: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

17

Sampling test cases: OATS example (cont...) The third column in L18 represents the

parameter hierarchy that only has one class, P. (any value in the third column represents P).

The fourth column represents the state of P (which there are two).

The fifth column represents the class C hierarchy, which has three members.

The sixth column represents the states of the C, D, and E classes.

Page 18: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

18

Sampling test cases: OATS example (cont...)

Step 5: Construct test cases based on the mapping and the rows in the table. Each row in the orthogonal array specifies

one specific test case.

Page 19: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

19

Sampling test cases: OATS example— result interpretation

The 10th column represents: An instance of B in

state 1 Is to send a

message by passing an instance of class P in state 3

To an instance of class E in state 2.

Page 20: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

20

Sampling test cases: OATS another example

Consider a web page with three distinct sections (Top, Middle, and Bottom) that can be individually shown or hidden by the user. You wish to test the interactions of the different sections.

Page 21: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

21

Sampling test cases: OATS another example (cont...)

There are three independent variables (the sections of the page).

Each variable can take on two values (hidden or visible).

An L4 orthogonal array will do the job — two

levels for the values and three factors for the variables.

Mapping the values onto the array where Hidden=0 and Visible=1

Page 22: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

22

Sampling test cases: OATS another example (cont...)

OA before mapping factors

Factor 1 Factor 2 Factor 3Run 1 0 0 0

Run 2 0 1 1

Run 3 1 0 1

Run 4 1 1 0

OA after mapping factors

Top Middle BottomTest case 1 Hidden Hidden Hidden

Test case 2 Hidden Visible Visible

Test case 3 Visible Hidden Visible

Test case 4 Visible Visible Hidden

Page 23: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

23

Sampling test cases: adequacy criteria for OATS

Exhaustive: all possible combinations of all factors are considered.

Minimal: only the interactions between the base classes from each hierarchy are tested.

Random: the tester haphazardly selects cases from several of the classes.

Page 24: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

24

Sampling test cases: adequacy criteria for OATS

Representative: a uniform sample that ensures that every class is tested to some level.

Weighted representative: Add cases to the representative approach based on relative importance or risk associated with the class.

Page 25: 1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

25

Key points Possibilities for determining which test cases to

select: based on probability distribution, uniform probability distribution and probability distribution based on the use profile.

A stratified sample is a set of samples in which each sample represents a specific subpopulation

An orthogonal array is an array of values in which each column represents a factor (a variable in an experiment).

Adequacy criteria for OATS: exhaustive, minimal, random, representative, and weighted representative.