Testing Software Solutions

Testing & Evaluating Software Solutions

Testing and Evaluation of Software Solutions

What is Testing? Testing is a process that checks to see

whether the solution meets the design specifications and is free from errors.

This chapter is concerned with more advanced testing techniques that prepare the product for release on the software market.

Testing & Evaluation of Software Solutions

Testing the Software Solutions

Reporting on the testing process

What is Testing?

Testing is done throughout the entire software development cycle:

Defining & Understanding the problem

Planning & Design of Software Solutions

Implementation of Software Solutions

Feasibility Study

Desk Checking Algorithms`

Debugging Tools

Comparison against original design specifications

The firsts step of testing a solution is to ensure that the program that you have created meets the design specifications outlined during the ‘defining & understanding the problem’ stage.

Design Specifications – are a list of criteria that must be met in the end product.

These specifications are no different to the specifications that you would have for buying a new car.

Comparison against original design specifications

Outline a list of criteria that you would use for purchasing a car?

Comparison against original design specifications

The following guidelines are used to ensure you have created measurable specifications: Clarify any vague terms (e.g. ‘most’, ‘some’ and

‘usually’) Avoid use of vague verbs (e.g. ‘processed’,

‘eliminated’ and ‘handled’) as they may be interpreted in more that one way.

Calculations should be accompanied by examples.

Comparison against original design specifications

The following guidelines are used to ensure you have created measurable specifications: Don not use incomplete statements such as ‘e.t.c’

and ‘so on’. Ensure the specification can be physically

measured. Incorporate pictures and diagrams to show

structure.

Comparison against original design specifications

Design specifications are often referred to as your ‘objectives’.

The aim of the project is to implement and solve each objective.

Objectives are written in such a way that a pass or fail can be allocated to them.

Comparison against original design specifications

Identify a tool that could be implemented to test the successful completion of objectives.

Comparison against original design specifications

Each objective is tested and if faults are encountered then the aspect of the product that failed is altered until the objective is realised.

This is why it is important to test your solution continually throughout the development process.

Testing Hierarchy

Testing

Alpha Beta

White-Box Black-Box

Module

Program

System

Live Data

Large Files

Response Times

Volume Data

Broad Categories of TestingAlpha TestingTesting by the personnelwithin the softwaredevelopment company.

Beta TestingTesting of the final solution by alimited number of users outsidethe company in real worldconditions.

What is Black-Box Testing?

Often referred to as ‘functional testing’ as it checks the inputs against expected and actual outputs.

Does not identify the source of the problem but rather to identify that a problem exists.

Once the problem has been identified it is necessary to move to ‘white-box’ testing to find and correct the error.

Black-Box Procedures

1. Range (Boundary) Checking Tests the upper and lower ranges of inputted

data. E.g. Numbers that may cause errors.

2. Equivalence Partitioning Testing the validation procedures of input

fields. E.g. Inputting 22a for the age field.

What is White-Box Testing?

Is called ‘structural testing’ as it concentrates on processes involved in the program rather than input-output.

This helps with the generation of test data as an explicit knowledge of the internal workings of the item being tested is needed.

White-Box Procedures

1. Statement Coverage Testing Test data would be chosen to test each

statement in a module. E.g. A flag (Test true and false)

2. Decision Condition Testing Test data selected to test each decision. E.g. Testing selection and repetition

statements.

Generating relevant test data for complex solutions

Often the process of generating test data is a expensive and time-consuming event due to the large numbers of variables used within a solution.

For this reason developers use a wide range of procedure given the situation which are organised into two main techniques: Black-Box Testing White-Box Testing

Black-Box - Question 1

Create a set of boundary (Range) test data items for an egg sorting machine. The machine works as follows:

“Eggs are graded as ‘SMALL’ if they weigh less than 50 grams, ‘MEDIUM’ if their weight is from 50 grams to less than 60 grams., ‘LARGE’ if their weight is from 60 grams to less than 70 grams and ‘EXTRA LARGE’ if their weight is 70 grams or more.”

Answers

49

50

59

60

69

70

71

Small

Medium

Large

Extra Large

White-Box - Question 2

Create a set of test data for the module below employing the ‘statement coverage’ method?

BEGIN

Get Wage

Get Tax

CaseWhere Wage

<$20,000 : Module A

<$40,000 : Module B

Otherwise : Module C

EndCase

IF Tax = True THEN

Module D

ELSE

Module E

END

Answers

$19,999, True

$19,999, False

$39,999, True

$39,999, False

$40,001, True

$40,001, False

White-Box - Question 3

Create a set of test data for the module below employing the ‘Decision coverage’ method?

BEGIN

Get Wage

Get Tax

CaseWhere Wage

<$20,000 : Module A

<$40,000 : Module B

Otherwise : Module C

EndCase

IF Tax = True THEN

Module D

ELSE

Module E

END

Levels of Testing

There are various levels of testing a system. Each is progressive and must be undertaken to ensure success.

Testing will commence at the lowest level.

3. System Testing

2. Program Testing

1. Module Testing

Module (Unit) Testing

Each individual module is tested to make sure that it is free from errors.

Treats the modules as stand-alone applications that do not require any other components of the program to function.

Normally, the use of libraries of code ensures that most modules used are appropriate.

Module (Unit) Testing

Identify the tool that is used to substitute the mainline of the program so that the module can be tested.

Program Testing

Ensures that all the modules work correctly together.

This level concentrates on the interface and the relationship of each module to the main program.

Uses both bottom-up and top-down testing of the program to ensure that the program meets the criteria set out in the design specifications.

System Testing

Involves testing the program in environments outside development.

Different processors, operating systems, software configurations and peripheral devices may cause errors to occur.

This testing is done by those outside the development team.

The System

Identify the names of the five (5) elements in a computer system.

Question 4

PaintPro is a new graphic manipulation software that allows users to scan images or input saved images and modify them via a variety of different filters.

Identify what area system testing would concentrate and the reasons why.

Question 4 - Answer

Software

If scanned images are to be inputted the program must be able to collaborate with appropriate scanner drivers.

Use of Live Test Data

Live test data is real test data that the program will work with.

Up to this point the program has been tested with data created by the development team for the purpose of finding errors.

Like testing a new car on a test track. Results will be obtained regarding how the

car performs within the environment in which it will be used.

Use of Live Test Data

Use of Live Test Data

For most products live test data should be created to test each of the following conditions: Large file sizes Mix of transaction times Response times Volume data Interfaces between modules Comparison with program test data

Large File Sizes

During testing small data files are used. It is appropriate to test the program with large

data sets to highlight problems with data access.

For example a program that is created process transactions within a video store will now have to process 10,000 customers and access a video stock list of 15,000 videos.

Large File Sizes

The limits of the program need to be tested to reveal any inefficiencies such as: Slow processing times. Slow network transfer rates. Effect on other users or programs.

Question 5

Identify TWO different types of errors that

could be found when testing large file sizes.

Mix of Transaction Types

Highlighting any errors that may occur during transactions being completed.

This may include: Multiple users using the program. Manipulation of data in different ways.

Response Times

Are the times required to complete a process. Any processes that are likely to take more

than one second should provide user feedback.

Both heavy and light loads should be checked.

Volume Data

Stress testing to see how the program performs with volumes of data beyond normal expectations is a way of checking efficiency and robustness.

Multiple users or a large number of processes occurring simultaneously may be required.

CASE tools can also be used.

Question 6

Banking software programs are faced with a multitude of customers trying to access records continuously because of the introduction of online-banking, EFTPOS and ATM’s.

Identify the areas that you would target regarding live testing such a system.

Question 6 - Answer

1. Large File Sizes – Trying to see the maximum withdrawals, deposits ect.

2. Mix of transaction types – Changing different applications i.e. BPAY, EFTPOS, different computers.

3. Response Times – Measure the time taken on different Internet Connections.

4. Volume Test Data – 50+ users accessing one account.

Interfaces Between Modules

In this section we are interested in testing the performance of interfaces that connect modules and programs.

Tests should examine the accuracy of data being passed as well as response times and the ability to cope with large volumes of data.

Comparison with test data

This involves the use of test data that checks the expected outputs of processing.

Because the outputs are already known, this testing process ensures the final product is able to perform processing correctly.

Benching Marking

Bench marking is the process of evaluating a product through a series of tests to meet a standard of excellence.

This process involves comparing competitors products to measure quality assurance.

Question 7

Identify the areas that a company may

employee in which to base its benchmarking

tests.

i.e. Overall Performance.

Question 7 - Answer

Overall Performance Quality Usability Robustness Maintainability Productivity

Quality Assurance

The quality of a product is measured against how well the product meets or exceeds users’ expectations.

The following factors are considered: Correctness: Does it do what it is supposed to do? Reliability: Does it do it all the time? Efficiency: Does it do it the best way possible? Useability: Is it designed for the end-user. Flexibility: Can it be modified?

Quality Assurance

Portability:Will it work with other hardware? Re-useability: Can it be used again? Integrity: Is it secure?

It is used to assure customers of the quality of the product.

Employs a set of measurable standards set out by the ISO (International Standards Organisation)

Question 8

A software application has been written to control the

operation of a digital video camera. The camera performs

all the usual video camera functions and some new ones.

For example, the camera can be set to commence

filming once it detects movement. This is useful for wildlife

photographers.

Describe the types of tests that could be conducted by

The quality assurance company to determine

reliability, efficiency and useability of this product.

Question 8 - Answers

1. Reliability – Beta test using live test data to test how the camera performs under large volumes of data. i.e multiple animals or long sequences.

2. Efficiency – Benchmarking could take place by giving a competitors product to compare.

3. Useability – Beta testing and surveying users regrading useability of the camera and its functions.