End-to-End Testing

Helping our Customers improve quality and reduce costs

2Introduction

Concept Reply focuses on End-To-End (E2E) Testing, which means Testing, Validation and Quality Assurance (QA), specifically addressing several markets:

Telco & Media Banking Automotive Extended Enterprise

The areas of intervention cover the full QA chain: from test methodology and process to design and execution via testing management, automation and tools.

The course of action can vary from professional services to turn-key projects, and even an innovative Test-as-a-Service (TaaS) approach.

A formal approach to QA does not allow for certain key factors: Cost Of Poor Quality Identifying defects early Phase Containment Effectiveness metrics Phase Screening Effectiveness metrics Maturity Level of the organisation

A pragmatic approach to QA

Quality Metric

0% 100%(Perfect)

C

o

s

t

Conformance Costs

Non Conformance

Costs

TimeCost

Quality

CMM Levels

C

o

s

t

O

f

Q

u

a

l

i

t

y

Phase

C

o

s

t

p

e

r

d

e

f

e

c

t

Prevention

Shipment

Field

But, based on our experience, the pursuit of Quality must be pragmatic: each organisation needs to find its own trade-off between Quality, Cost & Time-to-market.

QA Services

Concept Reply E2E Test unit

Test Design

Traceability Requirement

Definition of Acceptance Criteria

Definition of Test Architecture

Test Execution

Manual

Automated

Defect Analysis

First Level Troubleshooting

Defect Triage

Test coverage optimisation

Process & Tools

Q.A. Business Process Consultancy Test Management & Automation

Concept E2E Test: market intervention areas

Test Process

Test Tools

Test Design/Execution/Reporting

Embedded System & M2M

Solutions

Applications & Services

Telco & Multimedia

Devices

Telco & Media Banking

Automotive Enterprise

Positioning of Concept Reply E2E Testing

Turn-key Projects/Services

Professional Services

TechnologyBusinessProcesses

Test Engineering

Tool Integration & Development

ProcessImprovement

Test Factory

Process Improvement: Methodological Approach

Observe Identify clients target validation objectives, not only in the test area but also

in terms of business impacts (e.g.: customer satisfaction/loyalty, liability in the event of critical incidents)

Understand existing test process Analyse gathered data to identify most effective way for Reply to approach

intervention (full process coverage, single/multi area improvement) Share and discuss proposed solution with the customer

Improve Define an optimised test process, according to customers needs and

priorities Implement the process Deploy the process, and verify the application

Check Monitor improvement Perform lessons learned and corrective actions to drive continual process

and quality improvements

Check

Improve

Observe

Wide range of solutions: Integration of commercial tools Customisation of open source tools Development of ad-hoc solutions:

E.g.: Integration of different standalone customer tools in a unique dashboard

In different areas: Full test process management

Requirement management Test management Fault management

Configuration management

Test automation

Tool integration & development

Test Factory Model

Commercial, proprietary, open source customisation

Requirement & Test Management

REQUIREMENTS

OBJECTUNDER TEST

REQUIREMENTANALYSIS

TEST DEVELOPMENT

A

D

A

P

T

E

R

TEST EXECUTION RE

P

O

R

T

M

A

N

A

G

E

M

E

N

T

TEST RESULT & FAULT MNGMT.

TEST TOOLS

GOVERNANCE

REPORTS

Service, Turn-Key projects On-Site/Off-Site

Project Management Logistic Management Vendor Management

Client

Vendor

Test Reporting, issue tracking Test Automation

Test as a ServiceThe Test-as-a-Service implements an innovative approach to Testing (TaaS) : Full Customer control over process and plan with clear and optimised interfaces (Service Manager, Project Managers) A pay-per-use model within a general framework agreement A flexible and scalable level of service with guaranteed Quality of Service, milestones and Key Performance Indicators. Workload peak management 3rd party management

Project Assignment& Planning Request

Project Endorsement& Planning Response

Project Management& Operations

Project ClosureConfirmation

Project Manager

Test Team

Reply Test House

Integrated Planning & Logistic

Client

ServiceGovernance

Service Manager

3

r

d

p

a

r

t

y

Project Managers

Technical Experts \ Test Team

Test Specification Support to definition of requirements Risk and business impact analysis Definition of Acceptance criteria

Test Design Design of test architecture Definition of test environment Traceability of requirements Development of test cases and automation scripts

Test Execution Validation of test environment Scheduling of tests Test Execution (Manual or Automated) Defect/Incident management (Triage) First level troubleshooting

Test Engineering: Test Design & Execution

Functional Testing

Interaction Testing

Regression Testing

Boundary Testing

Compatibility Testing

Esploratory Testing

Testing from Escaped Defect Analysis

Stress Testing

Performance Testing

Negative Testing

Perceived Quality Testing

Comparative Testing

References

Thanks

www.reply.eu

Slide Number 1IntroductionA pragmatic approach to QAConcept Reply E2E Test unitConcept E2E Test: market intervention areasPositioning of Concept Reply E2E TestingProcess Improvement: Methodological ApproachTool integration & developmentTest Factory ModelTest as a ServiceTest Engineering: Test Design & Execution ReferencesSlide Number 13