1

The Components of Testing

Kenneth J. SweetmanKenSweetman@verizon.net

972 578 9031

Methodology

Tools Infras

tructu

re

2

Testing is the operation of a system or application under controlled conditions and evaluating the results (e.g., 'if the user is in interface A of the application while using hardware B, and does C, then D should happen'). The controlled conditions should include both normal and abnormal conditions. Testing should insure that functionally works correctly (positive path) and intentionally attempt to make things go wrong in an application (negative path).

Testing Overview

3

People Process Infrastructure Testing

Architecture

Data Model

Network

Topology

ConfigurationManagementStandards

Planning

Execution

Defects

Metrics

Documentation

Test Standards

Organization

Communication

Process Flows

Standards

Special Events

Hiring

Training

Development

Evaluation

Compensation

Skills Inventory

Bench Strengt h

Core Elements of Testing

4

Testing ProcessesTest

Management

Test Execution

Test Panning

Defect Tracking

RequirementsTraceability

ApplicationReadiness

EnvironmentManagement

Software Version Control

Configuration Management

Test Script Version

Data Management Schedule Management

Test Automation

Load Test Management

Network Configuration

Test Script Version

Environment Monitor

Documentation Control

Metrics

5

Test ManagementTest Approach – Strategy and PlanTemplates Defined TermsStandards (Test Folder, Fields, Field

Values, Required/Optional Fields, etc.)

Test Build and MaintenanceBuild/MaintainScheduleValidationMetrics

Test Execution Standards WorkflowMetrics Configuration Management

Defect ManagementStandards (Priority, Severity, etc) WorkflowMetrics

Testing Standards CommitteeStandards Management Test EvolutionMetrics ManagementConfiguration Management

Requirements TraceabilityDocumentation Version ControlRequirement/Test Case LinkMetrics

Activities

6

SDLC

Set Test Objectives

Develop Test Plan

Build Test Scripts

Execute Test

Summarize and Report Results

Design Construction & Test

Test ReportAnalysis

Roadmap

8

Metrics – Test Execution

9

Metrics – Defects

10

Cost of a Defect

11

Infrastructure: Configuration Management

Configuration ManagementSoftware

- Version Control- Software Inventory- Runtim e Control- Coverage Analyzer- Technical Assistance

D ata- DASD Managem ent- Data Migration and Aging- Intraday and N ightly Runs- Refresh and Maintenance Jobs- On Dem and Jobs

In frastructure- Connections to other Regions/Applications- Environm ent Maintenance and Monitoring- P roblem D iagnosis and Resolution- E rror Tracking(Abends, System Dum ps, etc)- Table Maintenance- Library S tructures

12

Infrastructure: Topology

Test Production Environments

TestManagement

Mainframe

Intranet/Internet

Unix

Web Applications

Testers

Testers

Testers

Intranet

Irvine

Mumbai

Remote

Testing Infrastructure

Mercury Servers

13

Infrastructure: Environment Schedule

TE 4

TE 4

TE 3

Kenneth J. Sweetman Environment

Environments "At a Glance"June July Aug SeptMay

7 14 21 28 2 9 16 23 30 2 9 17 24 31 6 13 20 273 10 17 24 31

TE 3MQA&B Dev

TE 1 13 States Q&R Backend

TE 2 DP Tri-state Backend

Project ARelease 4.2Release 4.3.2Release 4.4

TE 4TE 2

Migration Tools File Delivery

Project B

Project C Release 2.8

TE 1

TE 3TE 1TE 3 TE 1

PTE 2

TE 1

Release 1.1Release 1.2 PTE 2

PTE2SS2ACA Backend

TE 4MQ Q&R Backend

PTE1SS2ZQ Backend

PTE1

GCX Change 5/13EAB Conflict -- ZQ set to 4/1

GCX Change 5/15

GCX Change 5/15

E24

14

Testing Tools: Configuration Management

HardwareServer (Tool Server, Web Server, etc)Disk SpaceNetworkDisaster Recovery

SoftwareTool Set (Defect tracker, Test Script Management,

Test Execution, Document Control, DB Tools)Software Management ( Licenses, Upgrades, Patches, etc)DatabaseMonitors

RolesTesting - Test manager, test lead, tester, defect manager,

test automation.Environment - Software version control, Data Manager,

Configuration manager, Website Manager.Tool Set - Test tool manager, DBA, Security administrator,

Tool administrator.

15

Change Policy Points and Register the Loan

Business Workflow

16

Commitme

nt

Time

Contact

Awareness

Understanding

Evaluation

Trial Use

Deployment

Institutionalization

Testing Tools: Deployment

17

Regression Test The re-testing of fixes or modifications to the software or its environment after

they have occurred. Automated testing tools can be especially useful for this type of testing.

Test critical functions of an application to insure that these functions were not negatively impacted.

Black Box Test Tests are based on requirements and functionality Tests are not based on business processes and not

based on any knowledge of internal design or code.

White Box Test Tests are based on knowledge of the internal logic of

an application's code. Tests are based on coverage of code statements, branches, logic paths, conditions and data paths.

ProcessKnown Inputs Known Outputs

ProcessKnown Inputs Known Outputs

Testing – Main Concepts

18

Unit Test A preliminary test; to test particular functions or code modules. Typically

done by the programmer and not by testers, as it requires detailed knowledge of the internal program design and code. Not always easily done unless the application has a well-designed architecture with tight code; may require developing test driver modules or test harnesses.

Testing - Main Concepts

Integration Test This is the joint test of each system module. In this phase the components

are integrated and interact in a controled environment. This test is designed to discover failures and incompatibility between components. The Integration Test completes when all components have been tested

19

Functionality Test The functional specification is used to construct test cases designed to verify

each specified function point (or a “what”) from the user’s point of view. A combination of positive and negative should be built to reasonably explore the program’s condition set boundaries.

Usability Test Is intended to identify usability deficiencies existing in the software. It primarily

explores user interfaces for ease of use and understanding, and for compliance to any specified usability requirements.

Volume Test Volume testing involves subjecting the system to heavy volumes of data or work.

Specifically, the system should be able handle the volume of work specified in it’s objectives. If, for example, one of the objectives is the ability to maintain 100 open files simultaneously, a test case is generated that opens 100 files and verifies they can be read from and written to.

Testing - Main Concepts