Project Documentation and its use in Testing

JTALKS

Sco

pe

•Requirements and Specifications

•Test Plan

•Test Case

•Check List

•Traceability Matrix

•Report Documents

Req

uir

em

ents

According to ISTQB: “Requirement: A condition or capability needed by a user to solve a problem or achieve an objective that must be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed document.”

Requirements can be divided into two major groups:

-Functional requirementsA requirement that specifies a function that a component or system

must perform.

The functional requirements include:

√ Business requirements.

√ Functional (system) requirements

-Non-functional requirementsA requirement that does not relate to functionality, but to attributes

of it such as reliability, efficiency, usability, maintainability, and portability.R

eq

uir

em

ents

And what about Agile?In Agile commonly used User Story - a high-level

user or business requirement, typically consisting of one or more sentences in the everyday or business language capturing what functionality a user needs, any non-functional criteria, and also includes acceptance criteria.

Req

uir

em

ents

And what about Agile?R

eq

uir

em

ents

Specification: A document that specifies, ideally in a complete, precise and verifiable manner, the requirements, design, behavior, or other characteristics of a component or system, and, often, the procedures for determining whether these provisions have been satisfied. [After IEEE 610]

Sp

eci

fica

tion

s

What is it? Why we need it?Test Plan: A document describing the scope, approach, resources and schedule of

intended ATM test activities. It identifies amongst others test items, the features to be tested, the testing tasks, who will do each task, degree of tester independence, the test environment, the test design techniques and entry and exit criteria to be used, and the rationale for their choice, and any risks requiring contingency planning. It is a record of the test planning process. [After IEEE 829]

The Test Plan document is created during the planning phase.

Test

Pla

n

Test Plan contains:High-Level Expectations: What’s the purpose of the test planning process and the

software test plan? What product is being tested? What are the quality and reliability goals of the product?

People, Places, and Things: This topic is best described as “pointers to everything that a new tester would ask about.” Who working on the project? What they do? How to contact them? Where documents are stored ? Where the software can be downloaded from? Where the test tools are located?

Definitions

Inter-Group Responsibilities: The inter-group responsibilities discussed earlier dealt with what functional group (management, test, programmers, and so on) is responsible for what high-level tasks.

What Will and Won’t Be Tested

Test Phases and their entrance and exit criteria: In a code-and-fix model, there’s probably only one test phase—test until someone yells stop. In the waterfall and spiral models, there can be several test phases from examining the product spec to acceptance testing. Yes, test planning is one of the test phases.

Test Strategy: What typen of testing will you use? When will you apply each and to which parts of the software? If tools will be used, do they need to be developed or can existing commercial solutions be purchased? If so, which ones?

Resource Requirements: People, Equipment, Office and lab space, Software, etc

Risks and Issues

Testing Deliverables

Test

Pla

n

Test

Pla

n

http://wiki.jtalks.org/display/jcommune/Test+plan

According to ISTQB syllabulus:High level test case: A test case without concrete (implementation level)

values for input data and expected results. Logical operators are used; instances of the actual values are not yet defined and/or available.

Low level test case: A test case with concrete (implementation level) values for input data and expected results. Logical operators from high level test cases are replaced by actual values that correspond to the objectives of the logical operators

But what’s a Test Case?IEEE Standard 610 (1990) defines test case as follows:

“(1) A set of test inputs, execution conditions, and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement.

“(2) (IEEE Std 829-1983) Documentation specifying inputs, predicted results, and a set of execution conditions for a test item.”

According to Ron Patton (2001, p. 65), “Test cases are the specific inputs that you’ll try and the procedures that you’ll follow when you test the software.”

Boris Beizer (1995, p. 3) defines a test as “A sequence of one or more subtests executed as a sequence because the outcome and/or final state of one subtest is the input and/or initial state of the next. The word ‘test’ is used to include subtests, tests proper, and test suites.

Test

Case

Most useful items of test case: Test Case Number

Test Case Priority

Test Case Name

Description

Pre-Conditions

Step (Action)

Expected Result

Remarks or Post-conditions.

Test

Case

Checklist-based testing: An experience-based test design technique whereby the experienced tester uses a high-level list of items to be noted, checked, or remembered, or a set of rules or criteria against which a product has to be verified.

Check

Lis

t

“When used right, a Traceability Matrix can be your GPS for your QA journey”.

Requirement Traceability Matrix - RTM is a table which is used to trace the requirements during the Software development life Cycle. It can be used for forward tracing (i.e. from Requirements to Design or Coding) or backward (i.e. from Coding to Requirements). There are many user defined templates for RTM.

Each requirement in the RTM document is linked with its associated test case, so that testing can be done as per the mentioned requirements. Furthermore, Bug ID is also include and linked with its associated requirements and test case.

Tra

ceab

ility

Mat

rix

Tra

ceab

ility

Mat

rix

Report

s Bug Reports: A document reporting on any flaw in a component or

system that can cause the component or system to fail to perform its required function. [After IEEE 829]

Test Progress Report: A document summarizing testing activities and results, produced at regular intervals, to report progress of testing activities against a baseline (such as the original test plan) and to communicate risks and alternatives requiring a decision to management. (for ex: Daily Status Report, Weekly Status Report)

Test Summary Report : A document summarizing testing activities and results. It also contains an evaluation of the corresponding test items against exit criteria. [After IEEE 829]

Test Evaluation Report: A document produced at the end of the test process summarizing all testing activities and results. It also contains an evaluation of the test process and lessons learned.