12 Functional System Testing[1]

8/8/2019 12 Functional System Testing[1]

1/25

Functional System Testing

Written by

Adam Carmi


2/25

BlackBox Testing2

Outline Goalof testing

Test cases, test suites and test data

What is functional system testing?

Coverage

Functional testing techniques:

Functional analysis

Equivalence partitioning

Boundary value analysis


3/25

BlackBox Testing3

The goal of software testing The pr ocess ofuncovering evidence ofdefects in

software systems

Does not include efforts associated with tracking downbugs and fixing them

No amount of testing will improve the quality of acomputer program

The mo

re testing we do

of

a system, the mo

reconvinced we might be of its correctness

Testing cannot in general prove a system works 100%correctly


4/25

BlackBox Testing4

Test cases The basic component of testing is a Test Case

In its most generalform: (inputs, expected-result)

inputs include system state, user commands and data

values to be processed

expected resultincludes visible/audible interface

changes or changes in the system state

Test cases are organized intoTest Suites

functionality, security, performance,


5/25

BlackBox Testing5

Test case execution

A running of the software (under test) that

provides the inputs specified in the test case

and observing the results and comparing

them to those specified by the test case

If the actual result varies from the expected

result, then a failure has been detected


6/25

BlackBox Testing6

Test data An effective test strategy requires careful acquisition and

preparation oftest data prior to testing

Testing can suffer if test data is poor

Test data concerns: Depth: quantity and size of data

Breadth: variance of data values and data types

Scope: completeness, relevance and accuracy of data

Result of a query should be valid for the specific purpose of the

query, and not due to a missing or inappropriate value Conditions: data should reflect specific conditions in the domain

Data that would otherwise arrive after performing specific operationsover time

Test data and test results are expensive to construct


7/25

BlackBox Testing7

Example: Test data forTVRS

Name: test1.db

Description: Violation records designed forvalidating violation lookup

Violation ID Offenders first name Offenders last name Issuing policeman ID

243567 Rachel Josef 8700342

237812 Dan Levi 6386541

264683 Dan Porat 1346329

255245 Dina Josef 8245731

000345 longFirstN longLastN 8700342


8/25

BlackBox Testing8

Specification Vs. Implementation

The basic approaches to testing software are based on its

specification and implementation

White box testing test cases and data are constructedbased on the code that implements the software

Quality and correctness ofcomputations is validated

Will not be further discussed in this tutorial

Black box testing test cases and data are constructed

based solely on the softwares specification


9/25

Black

BoxTesting9

Functional System Testing

Testing ofa completed application to determine

that it provides allof the behaviors required of it

Testing ofcompleted increments that provide somedegree ofend-userfunctionality

Search for defects that are variances between the

actualoperation of the system and the

requirements for the system

System is treated as a black box


10/25

Black

BoxTesting10

How much testing is adequate?

Completely validating IEEE 754 floating-point division requires 264 test-cases!

float divide(float x, float y)

From practical and economic perspectives,

exhausti

ve testing is usua

lly n

ot p

ossib

le Which software pieces should we test?

Which test cases should we choose?


11/25

Black

BoxTesting11

Coverage

Coverage is a measure ofhow completely a test

suite exercises the capabilities ofa piece of

software Each line of code should be executed at least once

One test case should be constructed from each

specified requirement

It is necessary to use testing techniques thatnarrow down the numberof test cases allowing the

broadest testing coverage with the least effort


12/25

Black

BoxTesting12

Technique:F

unctional Analysis

Analyze the expected behaviorof the system

according to its functional specification

Generate a test procedure for each of the possibleusage scenarios

Corresponds to use case scenarios

Analyze how a change in one part of the system affects

other parts

Grand tour test cases: the result ofone test case

produces the data that is the input to the next test case


13/25

Black

BoxTesting13

Example:

Functional Analysis I

Use Case: Remove Traffic Violation

1. Super visor calls for deletion of the chosen Traffic Violation2. TVRS pr ompts Supervisorfor confirmation

3. Super visor confirms

4. TVRS requests OffendersDB to delete the Traffic Violation

from the offenders record

5. OffendersDB approves that the Traffic Violation has beendeleted

6. TVRS allows Supervisor tolook up a new Traffic Violation as

described in the Lookup Traffic Violation UC


14/25

Black

BoxTesting14

Example:

Functional Analysis IITest case ID: 134543

Pre-conditions: 1. TVRS initialized with test1.db database

2. Violation 243567 displayed in the Lookup Violation dialog

Related use cases: Lookup Traffic Violation, Remove Traffic Violation

Expected resultAction

Confirmation dialog is displayedPress Delete button

Lookup Violation dialog is displayedPress the Yes button

A message dialog stating that violation

243567 is not stored in TVRS

Enter 243567 at Violation ID text field

and press the Search button

Test results

Actual results:Passed

Failed

Defect diagnosis:


15/25

Black

BoxTesting15

Example:

Functional Analysis III

Verify effects

of

change

Filled when

the test case

is executed

How do we know that

violation 243567 is

stored in the system?

Can a tester

diagnose the cause

ofa defect?

In addition, a query

could be run on the

Offenders database


16/25

Black

BoxTesting16

Technique: Equivalence

Partitioning

Identifies ranges of input and initial conditionsthat are expected to produce the same result

A gr oup of test cases form an equivalence class if: They test the same feature/scenario

Ifone test reveals a fault, the otherones (probably) willtoo

If

a test do

es no

t rev

eal

afau

lt, the

other

ones(probably) will not either

It is adequate to use only a single representative ofthe equivalence class


17/25

Black

BoxTesting17

Example: Equivalence Partitioning I

Input value specification for Lookup Violation form:

Valid valuesField name

[0-9]{0, 9}Violation ID

[a-zA-Z]{0, 10}Offenders first name

[a-zA-Z]{0, 10}Offenders last name


18/25

Black

BoxTesting1

8

Example: Equivalence Partitioning II

Field Valid

equivalent

classes

Valid

representative

values

Invalid

equivalent

classes

Invalid

representative

values

Violation ID Known violation 00243567 ID < 0 or ID >

999999999

-1, 1234567890

Unknown violation 32456720 Non numeric ID 23ab@

Empty

Offenders

first name

Unknown violation David Character# > 10 Hasalongname

Single known

violation

Rachel Invalid character ad0@am

Many known

violations

Dan

Empty


19/25

Black

BoxTesting19

Example: Equivalence Partitioning III

The numberof test cases to choose from is reduced to(3 + 2) (4 + 2) (4 + 2) = 180

The actual number can be furtherlimited Single invalid field per test case (3 4 4 + 6 = 54)

Importance of use case

Resources available

Most frequent input

Life-critical software

Infeasible test cases

Randomly

...


20/25

Black

BoxTesting20

Technique: BoundaryValue

Analysis Based on experience / heuristics

Testing boundary conditions ofequivalence classes is

more effective Choose input boundary values as equivalence

classes representatives

Choose inputs that invoke outputboundary values

Examples: (0, 10] validate using 0, 1, 2, 9, 10, 11

Read up to 5 elements validate reading 0, 1, 4, 5, 6 elements


21/25

Black

BoxTesting21

BVA as an equivalence

partitioning extension

Choose one (or more) arbitrary value(s) in

each equivalence class

Choose valid values exactly on lower and

upper boundaries of equivalence class

Choose invalid values immediately below

and above each boundary (if applicable)


22/25

Black

BoxTesting22

General purpose test-suite

construction technique May be used toobtain reasonable coverage with little

effort Use cauti ously!

Unsuitable when

va

lues

ofdi

fferent

fie

lds are re

lated

1. While test cases can be added

Each new test case should include as many un-included valid non-boundary

equivalence class representatives as possible

2. While test cases can be added

Each new test case should include as many un-included valid boundary

equivalence class representatives as possible3. While test cases can be added

Each new test case should include a single invalid equivalence class

representative that has not been included before

4. Manually replace/remove redundant or infeasible test-cases


23/25

Black

BoxTesting23

Example:

Country

Club I

Day Sunday - Thursday Friday - Saturday

Gueststatus

Visitor Member Student Visitor Member Student

Age

(years)

Admission fee

[0, 16) 25 10 20 35 10 30

[16, 60) 50 25 45 70 25 65

[60, 120] 35 15 30 50 15 45

Specification


24/25

Black

BoxTesting24

Example:

Country

Club II

Field Valid

equivalent

classes

Valid

representative

values

Invalid

equivalent

classes

Invalid

representative

values

Day Sun - Thu Mon, Sun, Thu

Fri - Sat Fri, Sat

Guest

status

Visitor Visitor

Member Member

Student Student

Age [0, 16) 2, 0, 15 Non-numeric

value

14@a

[16, 60) 34, 16, 59 Age < 0 or

Age > 120

-1, 121

[60, 120] 100, 60, 120

A combo box is used

for choosing the day

and guest status


25/25

BlackBox Testing25

Example:

Country

Club IIITest case ID Day Guest status Age Result

1 Mon Visitor 2 25

2 Fri Member 34 25

3 Mon Student 100 30

4 Sun Visitor 0 25

5 Sat Member 16 10

6 Thu Student 60 30

7 Sun Member 15 10

8 Sat Student 120 45

9 Thu Visitor 59 50

10 Mon Member 14@a Invalid age

11 Fri Student -1 Invalid age

12 Fri Visitor 121 Invalid age

valid

valid

(boundary)

invalid

Documents

12 Functional System Testing[1]