Introduction to

Software Testing

Definition of Software Testing Previous

Testing is a part of coding workMake sure that a system does what it is supposed to do

PresentTesting is the process of analyzing a software item to detect the differences between

existing and required conditions and to evaluate the features of the software items. [IEEE/ANSI, Std 829-1998]

Testing is a concurrent lifecycle process of engineering, using and maintaining testware in order to measure and improve the quality of the software being tested. [Rick D Craig, 2002, “Systematic Software Testing”]

Software Testing To find and prevent defect To have confidence that the system

performs adequately To provide information about

quality To provide the insight of product

risk and quality To ensure a program corresponds to

its specification Defect Detection Mechanism

Software Test Process Test Strategy Test Plan Test Design Setup Test Environment Test Execution Defect Tracking Testing Tools Evaluate Test Exit Criteria Reporting Improve Test Process

Some terminologies

Error

Defect, Fault, Bug

Incident, Issue

Failure

Risk

Related to human=Mistake

Related to productBug= slang / dialect

Related to eventAccident

Risk = likelihood of failure x impact

Some terminologiesTest

DesignTest Case : a set of input values and expected results, developed for a particular objective or test condition

Test Case 1 to be executedTest Case 2 to be executedTest Case 3 to be executed

Test Case n to be executed

Order, story

included

Test Suite

Test Procedure= Test script

Test Scenario

Test Objectives (Why)Primary

•To find defects•To gain confidence of software•To prevent defects•To see software matching its requirement specifications•To reduce the business risk•To provide well-informed advice (quantitative metric)

Others•To check development process•To detect issues•To check logical design•To confirm that the system and software work properly

Characteristic in Software Testing

Exhausting testing is impossible There would be infinity path in a program There would be infinity combination of all inputs values There would be infinity timing or moment of occurring the

GUI event Testing cannot prove that there are no

defects It is hard to find defects if you produced

the product Practical Goal of Testing

“To maximize possibility of finding defects at the high risk area within the given resource (time, people) -> to minimize risk”

Why Software need to be tested as early as possible?

General Testing Principles Testing Shows presence of defects Exhaustive testing is impossible

(why? – 2 on/off button ,tab option, calculator)

Early Testing Defect Clustering Pesticide paradox Testing s context dependent Absence-of-error fallacy

The V Model

Integration test

Requirements

DetailedDesign

Moduleimplementation

Unit test

System test

Acceptancetest

Run

test

sW

rite tests

CustomerDeveloper

Functional(BB)

Structural(WB)

ArchitecturalDesign

Req. analysis

Logical spec.

Physical spec.

Program source

V model matrixTest Levels Goal Perform

er Env Test Basis Difficulty levels

Acceptance Req. User Real SRS User

System Non-functional Prof. Tester

Testing SDD + SRS

Integration Functional Developer Dev. SDD

Unit Find Defect Developer-peer Dev. Functions Technical

Validation vs. Verification

Verification ValidationDefinition Am I building the product

right?The review of interim work steps and interim deliverables during a project to ensure they are acceptable.

Am I building the right product?Determining if the system complies with the requirements and performs functions for which it is intended and meets the orgainization’s goals and user needs. It is traditional and is performed at the end of the project.

Characteristic Human-Based Testing- To find defects, read documentations

Computer-Based Testing- To find defects, execute the software

Type Manual review•inspection/walkthrough/peer review•Static analysis

Low level testing•Unit testing / integration testing

High Level testing• usability testing, Function testing, System testing, acceptance testing.

Classification of Testing Static Testing

Technical Review

Inspection

Requirement Testing

Design Testing

Static Analysis

Dynamic Testing Black box testing Risk Based Testing Spec-based testing Exploratory based testing Experience based testing Structure based testing (white box) Statistical testing CTM (Classification Tree Method) Use Case Testing State Transition Testing Boundary Analysis & Equivalent Partitioning Control Flow Testing Basis Path Testing Pairwise Elementary Comparison Test Structure Testing Acceptance Testing System Testing Integration Testing Componen (Unit) Testing Structure Testing Regression / Confirm Testing Functional / Non-Functional Testing

Testing Approach

Testing Design

Testing Levels

Testing Types

Effective and Efficient Testing What is effective testing ?

To make test result which is decided or desired Effective tester decide what test result to be drawn from the

testing effort What is efficient testing?

Desired test result can be achieved as soon as possible (performance)

Efficient tester allocate time, cost and resource properly

Effective testing vs. efficient testing?

When to stop? Where to test more? The best way for effective and

efficient = Risk based testing

Effective and Efficient Testing Effective testing vs. efficient testing?

The best way for effective and efficient testing => Risk based testing

Efficient

Effective

Risk MatrixHi Chances of FailureLow Business Impact

Hi Chances of FailureHi Business Impact

Low Chances of FailureLow Business Impact

Low Chances of FailureHi Business Impact

Business Impact

Chances of Failure(likelihood)

Standards ISO/IEC29119 – Standard for Softwar

e Testing Process IEEE829 – Standard for Software Test

ing Document

Testing ArchitectureMaster Test Plan

Test Policy

Test Process Improvement

StakeholdersEnd User manageme

nt

Test Basis

Test organization

Test Strategy Test Tech. Test

ItemsTest

Environment

Test Level

Lifecycle

Plan/Control

AnalysisDesign

Imple.Exec.

Exit / Reporting

ClosurePlan/Control

AnalysisDesign

Imple.Exec.

Exit / Reporting

Closure

Test Plan Test Spec.Test Case

Status Report

Test Procedur

eTest ResultResult Report

ReleaseFinal

Report

Test Process

evaluation and advice

Test Process

Improvement Plan

Testware

Defects Reporting Test

Reporting

Plan/Control

AnalysisDesign

Imple.Exec.

Exit / Reporting

Closure

Organization

Test OrganizationSteering

CommitteeBusiness

Testing (permanent / project base)

InstallCodingAnalysis / Design

Project Staff & QA

Project Management

Support•Strategy, tech, training•Infra & Tools•Subject Matter

Testing A

Test Management

Testing B Testing C

1. Tested by Developer2. Independent Tester inside Project Team

3. Project Testing Group in organization

4. Business OrganizationUser CommunityIndependent IT org.

6. Outsourcing

5. Independent Test specialistUsability / Security/ CertificateTesting