Final Paper

Right “Fit for Purpose” Testing Process through Test Process Assessment and Improvement

Author: Karankumar Wadhwani, Test Solutions Consultant

Co – Author 1: Vishesh Kumar Gupta, Test Consultant

Co – Author 2: Rahul Sharma, Test Tooling Consultant

SOPRA STERIA

AbstractIT organisations generally believe that they have world class testing teams delivering

to the SLAs’ within the mandated cost and quality paradigm. In the rapidly changing IT

space, as SDLC methodologies continue to mature towards a nonstop delivery model,

does the belief still hold?

In a bid to ‘do more for less’, we usually cut down testing efforts as that’s most easy

thing to do!! What is the effect of this? It’s anybody’s guess, poor quality.

Instead, institutionalising and continually improving the testing process in line with the

best practices best suited to today’s project methodologies (Agile/DevOps) can

provide the much needed impetus, confidence and the assurance in software quality.

The pursuit for establishing strong testing processes must begin with an unbiased

evaluation of the current test processes using benchmark industry standards and

accepting the shortcomings.

Though the identified gaps with respect to the industry standards gives a view of areas

to be focused, there is still a need to have a well-defined path for test process

improvement which can instil industrialised set of testing practices eventually leading

to tangible value to business.

In the book “The High Performance Entrepreneur”, Subroto Bagchi shares a wonderful

story that highlights the importance of having strong process. The story leads to the

conclusion “Be like Michelangelo or else, you must follow process”.

This whitepaper will detail about the whys and how of test process improvement

keeping in sight the TMMi mandated ways of working.

Test Process ImprovementWhyIn an ever-shifting software world, as the SDLC matures from the Waterfall to the Agile

and DevOps, there is increasing demand for enhanced level of efficiency and

effectiveness in testing. In order to bring about a semblance of conformity with the

emerging trends and needs, it has become all the more important to continually review

the existing processes and to assess their applicability against the changed or newer

benchmarks.

A robust Test Process Improvement process must be defined to aid the alignment and

refinement of the current Testing Process whilst making best use and reuse of the

existing test assets across the organisation. The improvements must be in line with

the product quality goals and testing objectives, which are themselves derived from

the organisations business objectives and growth drivers.

Even the topmost of the quality conscious organisations with the best of class test

process must apply Test Process Improvement initiatives, supported by organisation

wide cooperation, to adopt the Test Maturity Model into their test organisation.

The organisations should consider and value Process Improvement as a structured

and a measured activity, which enables their Testing services to achieve the highest

level of efficiency and effectiveness.

The key fundamental principles include quantifying the test process and further fine-

tuning it to ensure adherence best practices and industry standards, enhance

efficiency and effectiveness of testing and bring in the culture of continuous

improvement and innovation.

The vital objective of the Test Process Improvement process is to develop a system

which continuously improves testing. Test process optimisation entails the following:

Instituting test process assessment against best practices and industry

standards

Identifying the weak and the strong process areas, suggesting areas from

where process assets can be extracted and reused

Deploying incremental and innovative improvements across test processes and

technology

Selecting and providing the best practices to the organisation

Continuously evaluating new testing related tools for adaptation

Reuse of high quality test assets across the organisation

Continuously improving the testing process involves proactively and systematically

identifying, evaluating and implementing improvements to the standard test process

and project’s defined processes on a continuous basis.

Where

The Test process improvement initiatives across most of the organizations seemingly

have higher focus and returns around the core testing functions like Test Policy and

Strategy, Test Planning, Design, Execution and Control activities. Nevertheless, the

other supporting functions also have an equivalent impact on the success of testing

organizations. Now, depending upon the specific need or level of test process

improvement required, the organisation can seek to assess and improve upon the

core or/and the supporting testing functions.

The level of scope of the assessment will be driven by the factors such as:

Size of the testing organisation

Current and targeted maturity level

Timelines to bring the change

Budget constraints

Cost / revenue benefits

How

The first step towards any improvement is realization of the necessity to improve, be it

Test Process or any other area. The problem signs concerning testing need to be

interpreted consequently providing an impetus to improve the testing process.

To carry out the Test Process Improvement, an individual / group must collaborate

with relevant stakeholders to assess the current testing process, to introduce the

newer testing practices, and to manage the existing ones.

A gap analysis must be carried out to determine the delta between the current set of

organisation testing practices including tooling landscape against the best available in

the industry. This gap analysis should result in qualitative and quantitative

identification of the current state of play. This will serve as the basis to identify the

required changes.

The assessment scope and methodology could be customised to suit the exact needs

of the organisation depending upon the factors described before. Typically, the

assessments could range from basic informal surveys to more formal and structured

interviews with the testing stakeholders, and thorough analysis of testing process

enablers such as guidelines, templates, checklists, tools. A fast track approach, which

is a smaller subset of a full-fledged assessment, could be employed if time is a

constraint.

The assessment would result in a set of ideas and recommendations. These ideas

could range from setting consistent testing guidelines and practices for the

organization, to implementing the right set of tools to enable the automation and

enforcement of these guidelines and practices. These improvement ideas must be

captured in a Test Process Improvement log to allow further planning, tracking and

reporting. The implementation of these ideas must be managed using the

organisation’s Change Management and Configuration Management processes.

The following process should be followed to sift through the idea logs and implement

the fit for purpose ideas:

Ideateo Triage the ideas and assess the suitability of these with the stakeholders

before finally deciding to approve / reject these proposals

o The approved proposals must undergo a change impact assessment to

gather details such as: Cost benefit, efficiency benefits and other

intangible benefits

Incubateo Basis the Change Impact analysis, a refined cut down list of the potential

improvements need to be agreed

o Group and Prioritise requirements according to the scope, benefits that

they offer, the type of change

o Identify the necessary resources required to implement requirements

o Create a detailed plan to implement the requirements

o A schedule and high level milestones must be listed in the plan

Executeo Carry out the implementation of the process improvement to ascertain

the benefits that it produces

o Capture the process improvement metrics to measure the scores from

before and after the implementation. These metrics are presented to the

relevant stakeholders along with the benefit analysis

o Once the relevant stakeholders are satisfied with the improvement

initiatives, they must provide a sign off

Train

o People who will be responsible to work with the improved test process

must be trained to increase familiarity with the process

Key Success Factors

Below are the key success factors which will play a crucial role in realising the true

potential of test process improvement:

Clear vision of the objectives

Intent to bring in change

Willingness to accept the shortcomings

Overcoming apprehensions

Awareness of the latest industry standards and innovations in testing

Applying the right methodology (there is no one size fits all)

Benefits

A business outcome driven approach towards test process improvement delivers

several benefits and adds value to business:

High Quality: Industry best practices strengthen testing to deliver a high quality

product/application

Improved Productivity: Test process optimisation and incubation of robust

techniques (E.g. Risk based Testing) to enhance productivity

Cost Optimisation: Identify bottlenecks and recommend ways to optimise

testing efforts leading to cost reduction

Reduced risk levels: Predictable quality and increased level of confidence

leading to reduced risks

References & Appendix

World Quality Report: 2016-17

https://www.tmmi.org/

“The High Performance Entrepreneur”, Book by Subroto

Bagchi

Author Biography

Karankumar Wadhwani works as a Test Solutions Consultant in Sopra

Steria Noida, and having 8 years of experience in Test Solutioning,

Presales, Consulting and Testing delivery. He has expertise in creating

Testing proposals, conducting test assessments and implementing QA

process changes. He holds a Master’s degree in Business and

Management from University of Strathclyde Business School, Glasgow,

U.K.

Vishesh Kumar Gupta works as a Test Consultant in Sopra Steria Noida.

He has over 12 years of industry experience gained through multiple roles

covering Test Transformation Consulting, Test Management and

Business Analysis.

Rahul Sharma works are a Test Tooling Consultant in Sopra Steria Noida,

with over 20 years of experience in Testing.

THANK YOU!