Quality Assessment Handbook

1. INTRODUCTION ............................................................................................................................. 4

1.1. TESTING APPROACH .......................................................................................................................... 5

1.2. “V” MODEL ....................................................................................................................................... 6

1.3. STLC THREADS ................................................................................................................................. 7

2. THREAD: TEST DEFINITION ......................................................................................................... 8

2.1. ACTIVITY: PERFORM TEST ANALYSIS ........................................................................................................ 9

2.2. ACTIVITY: DEFINE TESTING TECHNIQUES ................................................................................................ 10

2.3. ACTIVITY: VALIDATE REQUIREMENTS ..................................................................................................... 11

2.4. ACTIVITY: DEVELOP TESTING STRATEGY ................................................................................................. 12

3. THREAD: TEST DESIGN ............................................................. ERROR! BOOKMARK NOT DEFINED.

3.1. ACTIVITY: DEFINE TRACEABILITY TRIAGE ..................................................... ERROR! BOOKMARK NOT DEFINED.

3.2. ACTIVITY: INFRASTRUCTURE REQUIREMENTS ................................................ ERROR! BOOKMARK NOT DEFINED.

3.3. ACTIVITY: TEST DATA REQUIREMENTS ........................................................ ERROR! BOOKMARK NOT DEFINED.

4. THREAD: TEST DEVELOPMENT ............................................... ERROR! BOOKMARK NOT DEFINED.

4.1. ACTIVITY: DEVELOP TEST SUITES ................................................................ ERROR! BOOKMARK NOT DEFINED.

4.2. ACTIVITY: GENERATE TEST DATA ............................................................... ERROR! BOOKMARK NOT DEFINED.

4.3. ACTIVITY: SET UP TEST SYSTEM ................................................................. ERROR! BOOKMARK NOT DEFINED.

5. THREAD: TEST EXECUTION ...................................................... ERROR! BOOKMARK NOT DEFINED.

5.1. ACTIVITY: EXECUTE TEST SCRIPTS ............................................................... ERROR! BOOKMARK NOT DEFINED.

5.2. ACTIVITY: ITERATE TEST EXECUTION ........................................................... ERROR! BOOKMARK NOT DEFINED.

5.3. ACTIVITY: TEST REPORTING AND METRICS ................................................... ERROR! BOOKMARK NOT DEFINED.

6. THREAD: TEST EXIT .................................................................. ERROR! BOOKMARK NOT DEFINED.

6.1. ACTIVITY: CREATE EXIT REPORT ................................................................. ERROR! BOOKMARK NOT DEFINED.

6.2. ACTIVITY: DISTRIBUTE EXIT REPORT ........................................................... ERROR! BOOKMARK NOT DEFINED.

7. THREAD: DEFECT MANAGEMENT ............................................ ERROR! BOOKMARK NOT DEFINED.

7.1. ACTIVITY: DEFECT PREVENTION ................................................................. ERROR! BOOKMARK NOT DEFINED.

7.2. ACTIVITY: DEFECT DETECTION ................................................................... ERROR! BOOKMARK NOT DEFINED.

7.3. ACTIVITY: ROOT CAUSE ANALYSIS .............................................................. ERROR! BOOKMARK NOT DEFINED.

7.4. ACTIVITY: DEFECT RESOLUTION ................................................................. ERROR! BOOKMARK NOT DEFINED.

7.5. ACTIVITY: PROCESS IMPROVEMENT ............................................................ ERROR! BOOKMARK NOT DEFINED.

8. TEST MANAGEMENT ................................................................. ERROR! BOOKMARK NOT DEFINED.

8.1. ESTABLISH AND DISTRIBUTE TEST PLAN ....................................................... ERROR! BOOKMARK NOT DEFINED.

8.2. OBTAIN COMMITMENT TO TEST PLAN ........................................................ ERROR! BOOKMARK NOT DEFINED.

8.3. RECONCILE WORK AND STAFF .................................................................... ERROR! BOOKMARK NOT DEFINED.

8.4. MONITOR PROGRESS AGAINST PLAN .......................................................... ERROR! BOOKMARK NOT DEFINED.

8.5. MANAGE DEFECTS TO CLOSURE ................................................................ ERROR! BOOKMARK NOT DEFINED.

9. SERVICE OFFERING ................................................................... ERROR! BOOKMARK NOT DEFINED.

9.1. FUNCTIONAL TESTING .............................................................................. ERROR! BOOKMARK NOT DEFINED.

9.2. PERFORMANCE TESTING .......................................................................... ERROR! BOOKMARK NOT DEFINED.

9.3. AUTOMATION TESTING ............................................................................ ERROR! BOOKMARK NOT DEFINED.

9.4. INTEGRATION TESTING ............................................................................. ERROR! BOOKMARK NOT DEFINED.

9.5. SYSTEM TESTING .................................................................................... ERROR! BOOKMARK NOT DEFINED.

9.6. REGRESSION TESTING .............................................................................. ERROR! BOOKMARK NOT DEFINED.

9.7. USABILITY TESTING ................................................................................. ERROR! BOOKMARK NOT DEFINED.

10. TOOL BOX ............................................................................... ERROR! BOOKMARK NOT DEFINED.

1. INTRODUCTION

Software testing is a core quality control component of any software development lifecycle and essential

to effective software quality assurance. Testing is the means of achieving two major objectives –

Verification (You built it right) and Validation (You built the right thing). By themselves, Verification and

Validation do not guarantee software quality; various test management threads, and other aspects of

software engineering are required.

Quality Assessment Hand Book focuses on defining QAT’s testing approach and reflects its years of

experience and resulting best practices in this area. Like any other framework, this Hand Book is designed

to be flexible and customizable so that it can be tailored for use on a variety of testing initiatives.

Specifically, this is designed to support testing on the following types of initiatives:

IT Managed Development Projects

Both production support changes and enhancement requests on IT Managed Applications

Stand-alone Testing Service for any software developed by teams outside IT or other third-party

1.1. TESTING APPROACH

This document identifies the QAT’s testing methodology as implemented across all projects. Core

elements of QAT’s approach to testing are -

Adaptable to all project types and methodologies by carefully and thoughtfully integrating with project management

Support of Process Improvement Goals as defined by various functional teams in IT

Risk Based Testing to detect the software’s technical risks, and to control critical path failures

Defect Management to include collection of defect measures and causal analysis to identify, remove and prevent defects as close to injection as possible

1.2. “V” MODEL

The V-Model is a structured testing framework emphasizing quality from the initial requirements stage

through the final testing stage. The framework can be used with any project management or system

development methodology. It focuses on testing throughout the development life-cycle, early

development of test requirements, and early detection of errors. Each major deliverable in the

development process is assessed, verified, validated and tested.

“V” - Model

1.3. STLC THREADS

STLC Threads

Our STLC can be classified into threads of work performed across the phases of SDLC. The goal of this

process is to depict the relationship and integration between testing and software development activities.

The SDLC phases shown on the diagram are for illustrative purposes and are not defined within this

document. A quick look at our STLC threads -

Test Definition– Defines the overall scope, strategy and performance objectives of the testing to

be performed

Test Design – Decide environment and set up, testing techniques to be used, test data, and test

sets & scenarios for automation and manual testing

Test Development –Build test scripts, and create test data; Create and configure test systems

Test Execution –Execute scripts, log defects, report metrics

Test Exit –Deliver Test Exit Report to confirm the software is ready to be deployed

Defect Management – Manage defect prevention, tracking, from detection till closure in

compliance to the approach defined in this document

2. THREAD: TEST DEFINITION

This thread defines overall test scope, Testing Strategy and performance objectives of the tests to be

performed. A Testing Strategy driven by the risk analysis is delivered at the end of the thread that, details

on planning and executing the testing phase in perfection.

The Testing Strategy provides the roadmap for the overall testing effort. Its primary goal is to provide a

management-level view of all testing to be accomplished and the sequencing and dependency between

the various types and iterations of testing.

This thread contains the following activities:

Perform test analysis

Define testing techniques

Validate requirements

Develop Testing Strategy

2.1. ACTIVITY: PERFORM TEST ANALYSIS

The purpose of this activity is to identify and assess the software’s technical risks and overall impact on

the business context so the optimal testing scope can be determined and planned.

Ultimately the question of how much testing is enough is a business decision to be made by the business

owner that owns the application software. The optimal level of testing cannot be decided without the

assistance of the business area owners.

2.2. ACTIVITY: DEFINE TESTING TECHNIQUES

The purpose of this activity is to assess how every item on scope would be tested and what technique

could be applied to test a particular feature of the item. Test scope and specifications (from the Test

Analysis) grouped by levels, modules etc. are required to focus on testing. Document the test structure

with applicable testing techniques, test data, and approach for each area. Ensure the structure allows the

specification coverage to be measured.

From your own experience, or other experience you have access to, identify existing techniques that will

either meet the requirements of the test approach, or can be adapted to meet them.

2.3. ACTIVITY: VALIDATE REQUIREMENTS

Defects introduced early in the effort to engineer a system due to poorly identified requirements are generally seen as a major factor leading to high system and software costs, especially if the defective requirements are undetected until later development phases in the lifecycle. The ultimate savings due to error detection, diagnosis, and correction before a trial system is produced are generally great.

Traceability matrix is created by associating requirements with the work products that satisfy them and is used to ensure all the requirements are met. The traceability matrix helps in locating affected system components when there is a requirement change or a defect identified. This thread addresses requirements validation through Requirements Traceability Matrix ensuring that the

set of requirements is correct, complete, and consistent

a system can be created that satisfies the requirements

and the solution can be tested to prove that it satisfies the requirements

2.4. ACTIVITY: DEVELOP TESTING STRATEGY

Develop a Testing Strategy to provide a roadmap for testing that defines the testing scope, objective, process and plan. Other areas that will be addressed in the Testing Strategy document include:

Test Infrastructure & Tool Requirements

Test Data Development & Management Process

Defect Management Process - Refer the thread Defect Management for additional information

on this topic.

Software Risk Profile