Download ppt - CEN 5070 – Software V&V What is Software Testing © 2001-8, Dr. E.L. Jones

CEN 5070 – Software V&V

What is Software Testing

© 2001-8, Dr. E.L. Jones

9/2003 Testing Concepts 2

Purpose

This module presents the basic concepts of software testing. We present several aspects of software testing, and a give a generic lifecycle.

"… arbitrarily selected test set ... results in inefficient testing, leaving

some functions untested while performing redundant testing of others."Darlene Mackay, Quality Consultants Unlimited


Agenda

• What is Testing

• Testing Lifecycle

• A Framework for Testing

• A Complete Example


WHAT IS TESTING?

• Executing software to determine whether it meets its specification

• Executing a software element to discover defects or anomalies

• Executing software to evaluate its behavior in different environments

• Inspecting software element code to discover defects or anomalies.


TESTING IS HARD

• Exhaustive testing is impossible

• Software is complex

• Specifying requirements is hard

• Testing is a fallible, human process

• Consequences of failure not always understood in advance


WORKING SMART

• Test small pieces

• Use proven techniques

• Use tools to advantage

• Start early

• Cost of errors grows exponentially with time they go undetected


Agenda

• What is Testing





A TESTING LIFECYCLE

Analysis

Design

Implementation

Execution

Evaluation

Specification

Test Strategy/Plan

Test Script, Data, Driver

Defect DataProblem Reports

Test Results

Test Cases


TESTING vs DEVELOPMENT

• Work broken into stages

• Require similar skills

• Require management oversight

• Produce artifacts used to assess quality of the work

• Test artifacts are overhead, not part of delivered system.


TEST ARTIFACTS

• Strategy/Plan (must know techniques)

• Test cases – central artifacts

• Test machinery to apply test cases

• Data showing results

• Reports that interpret test results

• List of problems/defects to be fixed.


TESTING IS HIERARCHICAL

• Unit testing – smallest components

• Aggregate testing – assemblies of units

• Integration testing – combinations of assemblies

• System testing – system built from assemblies


TESTING OBJECTIVES

• To demonstrate conformance

• To find defects

• To measure behavior in specific environments• Load / stress testing

• Performance testing

• To measure trustworthiness/reliability


Agenda

• What is Testing





REQUIRED ELEMENTS OF A TESTING METHODOLOGY?

• Specification for the software

• Process for designing test cases

• Repeatable process for designing, running and evaluating tests

• Accountable artifacts of testing activities

• Economical use of human, time and computing resources


Agenda

• What is Testing





BLACK-BOX TESTING

• Testing based on the specification rather than the implementation.

• Specification defines the expected response(s) to stimuli

SoftwareunderTest

Stimuli Response(s)


BLACK-BOX TECHNIQUES

• Functional testing -- tests the behavior of the software.

• Boundary testing -- tests behavior at the lower/upper bounds of input values

• Random testing -- tests using randomly generated stimuli

• Intuitive (ad hoc) testing -- error guessing


FUNCTIONAL TEST DESIGN METHODOLOGY

• Specification

• Identify behaviors

• Develop test cases

• Write test script


EXAMPLE A(1) Specification

• Compute pay for an hourly employee, given the number of hours worked and the hourly pay rate. Compute overtime at 1.5 times hourly rate for hours in excess of 40.

SoftwareunderTest

Hours Pay

Rate


EXAMPLE A(2) Identify Behaviors

• Case 1: No overtime (Hours <= 40)• Expect Pay = Hours * Rate

• Case 2: Overtime (Hours > 40)• Expect Pay = 40*Rate+1.5*Rate*(Hours - 40)


EXAMPLE A(3) Create Test Cases

• Case 1: No overtime (Hours <= 40)• Use Rate = 10, Hours = 30

• Expect Pay = Hours * Rate = 300

• Case 2: Overtime (Hours > 40)• Use Rate = 10, Hours = 50

• Expect Pay = 40*Rate+1.5*Rate*(Hours - 40)

= 550


EXAMPLE A(4) Write Test Script

StepStimuli Expected Response

Hours Rate Pay =

1

2

30

50 10

10 300

550


A MORE COMPLEX EXAMPLE (B)

• Increased number of behaviors

• Use of decision table to document

behaviors

• Test case generation from decision table


EXAMPLE B(1) Specification

• Compute pay for employee, given the number of hours worked and the hourly pay rate. For hourly employees (rate < 30), compute overtime at 1.5 times hourly rate for hours in excess of 40. Salaried employees (rate >= 30) are paid for exactly 40 hours.


EXAMPLE B(2) Identify Behaviors

• Case 1: Hourly AND No overtime • (Rate < 30) & (Hours <= 40)• Expect Pay = Hours * Rate

• Case 2: Hourly AND Overtime • (Rate < 30) & (Hours > 40)• Expect Pay = 40*Rate+1.5*Rate*(Hours - 40)

• Case 3: Salaried (Rate >= 30) • Expect Pay = 40 * Rate


DECISION TABLE

Condition

c1: Rate < 30 | Y Y N N

c2: Hours <= 40 | Y N Y N

Action

a1: Pay = Straight time | X

a2: Pay = Overtime | X

a3: Pay = Professional | X X

Columns defineBehaviors


EXAMPLE B(3) Create Test Cases

• One test case per column of decision table

• Case 1: Hourly, No Overtime

• Case 2: Hourly, Overtime

• Case 3: Salaried, No Extra Hours

• Case 4: Salaried, Extra Hours

• Order the test cases by column


EXAMPLE B(4) Write Test Script

StepStimuli Expected Response

Hours Rate Pay =

1

2

30

50 10

10 300

550

3 30 40 1600

4 50 40 1600


RULES -- DECISION TABLES

Condition

c1: Rate < 30 | Y Y N N

c2: Hours <= 40 | Y N Y N

Action

a1: Pay = Straight time | X

a2: Pay = Overtime | X

a3: Pay = Professional | X X Use X to select action(s)

Elementaryconditions

Use 'Y', 'N', '-' or space


NEXT TIME

• Test Case Generation Strategies

• Documenting the Test

• Specification Techniques