Using Test Trees to get an Overview of Test Work

Use Test Trees to get an Overview of your Test

INTERNAL USE

Example of a test tree

Classification Internal Use Author AMQH Approved by N/A Version 1.12

FS-3. Person and Recidence Information paneRequirements: FR_182, FR_191

UI Spec: Section 5.4

User Story: none

Test casesNo. Class filter Ci filter Ci filter Ci 1

Person and Recidence Information, expanded

contents

1 topline: [-] Person and Recidence Information x

Mandatory labels

which label

2 Name: x

Gender:

which

3 female x

4 male

5 ID: x

6 Addresse: x

7 Telephone: x

Citizenship:

which

8 UK x

9 not UK

Dependable contents

which label or text

C/O:

relevant

10 yes x

11 no

Custodian:

info available

12 yes x

13 no

[!] Covert Name and Address

person subject to this

14 yes

15 no x

16 buttom line: [+] Other People at the same Address x

actions

17 press topline x

18 press buttom line

FS-3. Person and Recidence Information pane Coverage items 28

Requirements: FR_182, FR_191 40% 11


User Story: none

Ci No.

Classfilter

& Ci

filter

& Ci

filter

& Ci

filter

& Ci

Expected result

Person and Recidence Information, expanded only

P&RI contents

1 topline: [-] Person and Recidence Information present

Mandatory labels

which label

2 Name: shown with correct name

Gender:

which

3 female shown with text "Female"

4 male shown with text "Male"

5 ID: shown with correct ID

6 Addresse: shown with correct address

7 Telephone: shown with correct no.

Citizenship:

which

8 UK label shown, no info.

9 not UK label shown with correct citizenship

Dependable contents

which label or text

C/O:

relevant

10 yes C/O shown with correct info. before Address

11 no nothing shownshown

Custodian:

info available

12 yes label and info shown below Citizenship



Use Test TreesTest Coverage

Test Design and Implementation

Introduction to a Test Tree

Example


When are we done with the test

As much as we like testing, we cannot go on ‘for ever’ – this is why acceptance criteria are defined

When the acceptance criteria are reached we can stop the testing

How much do we have to test with this acceptance criteria:

No more than 10 unresolved defects


This is more like it

Acceptance criteria should always include a coverage measure

Example:

• a coverage of 100 % is achieved for the requirements on which the test is based

• a detailed coverage of 60 % (high risk areas) or 40 % (low risk areas) is achieved


Test coverage definitions

ISTQB:

the degree, expressed as a percentage, to which a specified coverage item has been exercised by a test suite.

ISO 29119:

degree, expressed as a percentage, to which specified test coverage items have been exercised by a test case or test cases


Specified test coverage items?

So we have to find out what specified test coverage items are.

ISO 29119: attribute or combination of attributes that is derived from one or more test conditions by using a test design technique that enables the measurement of the thoroughness of the test execution


Some test case design techniques


Boris Beizer• Control-flow testing• Data-flow testing• Domain testing• Finite-state testing• Loop testing• Syntax testing• Transaction flow t.

ISTQB• Boundary value analysis• Equivalence partitioning• Cause Effect graphing• Checklist• Classification tree• Decision table testing• Orthogonal arrays• Pairwise testing• State Transition testing• Structure based • Use case testing• User story testing• and more..

ISO 29119• Boundary value analysis• Equivalence partitioning• Cause Effect graphing• Classification tree• Combinatorial e.g.

• pairwise• orthogonal arrays

• Decision table testing• Random testing• State Transition testing• Scenario testing• Structure based • Syntax testing• and more…

Some coverage items

• Boundary value analysis Boundary + 1 or 2 nearest values

• Equivalence partitioning Equivalence partition

• Cause Effect graphing Decision rule

• Checklist Entry

• Classification tree Leaf

• Decision table testing Decision rule

• Orthogonal arrays Pair

• Pairwise testing Unique parameter/value set

• State Transition testing States and/or Chow’s n-switch

• Structure based Depends on which one

• Use case testing Unique scenario

• User story testing Acceptance criterion

• and more..Classification Internal Use Author AMQH Approved by N/A Version 1.19




Example


Test design and implementation

The test design and

implementation process

is a sub-process to the

entire test process

described by ISTQB and

in ISO 29119


Test design and implementation

items


Definitions

A feature set is a logical part of the system, which you can test in isolation, e.g.: • access to the system, including access right

• a user interface, e.g. a form or a pane

• a set of rules implemented in a component

• general error handling

A test condition is something within a feature set, which you can test in isolation: e.g. • a requirement

• a user story

• a rule





Example


The test tree principle

A test tree is

• a graphical way to present the test basis for a test item based on coverage items

• akin to classification trees – with coverage items described under independent classes

• a way to document the coverage obtained in a given test case

• a mean to calculate overall coverage of a test

• documented in Excel the most powerful test tool we have :-)


The contents of the tree




User Story: none

Test casesCi No.

Classfilter

& Ci

filter

& Ci

filter

& Ci

filter

& Ci 1 2


contents


Mandatory labels

which label

2 Name: x

Gender:

which

3 female x

4 male

Test tree can be very ‘tall’




User Story: none

Test casesCi No.

Classfilter

& Ci

filter

& Ci

filter

& Ci

filter

& Ci 1 2


contents


Mandatory labels

which label

2 Name: x

Gender:

which

3 female x

4 male

5 ID: x

6 Addresse: x

7 Telephone: x

Citizenship:

which

8 UK x

9 not UK

Dependable contents

which label or text

C/O:

relevant

10 yes x

11 no

Custodian:

info available

12 yes x

13 no



14 yes

15 no x


actions

17 press topline x


Person and Recidence Information, collapsed

contents

19 topline: [+] Person and Recidence Information

actions

20 press topline

Other People at the same Address,expanded

contents

21 Mandatory text: Other people at the Adress: <no.>

how many

22 0

more than 0

any subject to navne- og adressebeskyttelse

23 no

yes

all

24 yes

25 no

26 Sub-pane buttom: [-] Other people at the same address

actions

27 press topline

28 press Sub-panel topline

* <no.> lines with "-" <name> <id> of other people, sorted descending by ID

** <no.> lines with "-" <name> <id> of other people, sorted descending by ID

*** <no.> lines with "-" <name> <id> of other people, sorted descending by ID

This is a relatively small tree with only 3 classes and 28 coverage items

Sometimes a test tree can have 50 + coverage items

Use ‘divide and conquer’ where possible




Example


Example presentation

The example system is a case management system for case workers, who are responsible for the administration of payments to clients.


Result

Feature sets

The system is broken into the following feature sets (and many more):

• the framework for the overview of case information for manual handling containing a number of panes

• a specific pane within the framework of the overview

• rules for automatic handling

• etc.

•


Framework for overview of case info.


Case Overview for Case: nn

Family Relations

Person & ResidenceInfo

Result Details

Salary Info

Case History

Requirements for Person and Residence Info


Requirement FR_191: Show person information, family relations, history of the case (and more).

When a case worker types a person ID, the system shall present the following

information related to the citizen:

Salary Information

Case History

Person and Residence Information

(and more)

Requirement FR_182: Read and save person information.

Upon reception of person information, the system shall be able to store it with a

reference to the person’s ID.

Person and Residence Info

Design and notes for the pane with sub-pane


Note Description

1 This pane will be expanded as default. The following is shown for the

chosen person:

• Name

• Gender

• ID

• if relevant: C/O

• Postal address

• Country

• Telephone number

• Citizenship – if not UK

• if relevant: Name of custodian

2 If the persons’s name and address are covert, this is marked with a

colored text above the name and address, which are shown.

Text = “! Name and address are covert”

3 A click on ”[+] Other people at the same address” will show

information about other people living at the same address in a sub-

pane.

4 If one or more of the other person(s) at the address has covert

name and address this will be shown as a list of people without, if

any, sorted by ID, followed by a Text as above and a sorted list of

the people with, if any.

5 A click on ”[-] Person and Residence Information” will cause the

entire pane to collapse, so that only the top line is visible.

6 A click on "[+] Citizen and Residence Information" will cause the

pane to expand, with the sub-pane collapsed.

Coverage items are identified

Test case design techniques are used to find coverage items depending on the nature of the test basis, for example:


Equivalence class partitioning

Classification tree

Pane navigation

The navigation in the pane and sub-pane can be treated like a state machine with 3 states and 5 transitions


pri = “Person and Residence Information” pane

both = pri + “Other People at the same Address” sub-pane

Constructing the tree

Fill in the information about the test item and the test basis

Identify the highest level of classes for the test item from the test basis

For each class, identify the filters that will partition the class into further sub-classes

For each sub-class, identify the coverage items using appropriate test case design techniques

For each coverage item, describe expected result


Constructing the tree

Fill in the information about the test item and the test basis

Identify the highest level of classes for the test item from the test basis

For each class, identify the filters that will partition the class into further sub-classes

For each sub-class, identify the coverage items using appropriate test case design techniques

For each coverage item, describe expected result





User Story: none

Ci No.

Classfilter

& Ci

filter

& Ci

filter

& Ci

filter

& Ci

Expected result


P&RI contents


Mandatory labels

which label


Gender:

which






Citizenship:

which



Dependable contents

which label or text

C/O:

relevant



Custodian:

info available



[!] Covert Name and AddressClassification Internal Use Author AMQH Approved by N/A Version 1.128



14 yes text shown in red under topline

15 no no warning text shown

16 buttom line: [+] Other People at the same Address present

actions

17 press topline ([-]) P&RI pane collaps and topline with [+]

18 press buttom line ([+]) both panes expanded

P&RI (and sub-pane), collapsed

contents

19 topline: [+] Person and Recidence Information present

actions

20 press topline pane expands

P&RI and sub-pane, expanded

Sub-pane contents

21 Mandatory text: Other people at the Adress: <no.> present

how many with covert address

22 0 text and <no.> = 0 shown

more than 0

any subject to navne- og adressebeskyttelse

23 no *

yes

all

24 yes **

25 no ***

26 Sub-pane buttom: [-] Other people at the same addressshown

actions

27 press P&RI topline ([-]) both panes collapse

28 press Sub-panel topline sub-pane collapses

* <no.> = actual number; lines with "-" <name> <id> of other people, sorted descending by ID

** <no.> = actual number; warning text ; lines with "-" <name> <id> of other people, sorted descending by ID

*** <no.> = actual total number; lines for without; warning text ; lines for with

The rest of the tree


Select coverage items to test




User Story: none

Test casesNo. Class filter Ci filter Ci filter Ci 1


contents


Mandatory labels

which label

2 Name: x

Gender:

which

3 female x

4 male

5 ID: x

6 Addresse: x

7 Telephone: x

Citizenship:

which

8 UK x

9 not UK

Dependable contents

which label or text

C/O:

relevant

10 yes x

11 no

Custodian:

info available

12 yes x

13 no



14 yes

15 no x


actions

17 press topline x





User Story: none

Ci No.

Classfilter

& Ci

filter

& Ci

filter

& Ci

filter

& Ci

Expected result


P&RI contents


Mandatory labels

which label


Gender:

which






Citizenship:

which



Dependable contents

which label or text

C/O:

relevant



Custodian:

info available




Please try this at home


Thank you – and remember

• Test is difficult

• Test requires overview

• Test requires creativity

• Test requires systematic work

• Test requires imagination

• Test requires courage

• Test is fun32

[email protected]


Software

Using Test Trees to get an Overview of Test Work