8/8/2019 POC - RisK Based Testing

1/2

What is Risk Based Testing?- By Susheel Vellanki, Sr QAEngineer, Symphony Corporation

Before understanding risk based testing, we must understand the importance of risk based testing by answering following

questions.1)Can we achieve 100% test coverage always?Do we always have enough time to do full regression testing? How much of

regression testing has been automated or can be automated?

2)How much % of compatibility testing (browsers, databases, OSs, application servers, networks, hardware platforms) is typicallycovered during regression testing?

3)How can we assure that the most frequently used modules/components are working fine? Do you use tools to analyze codecoverage from manual or automated regression testing?

4)What is the $ value of the time spent in trying to cover 100% regressio n and compatibility testing vs. risks in releasing thesoftware to custom ers early due to internal and external factors in the project?

Lets explore the evolution of risk based testing and what makes it different from o ther types of testing by answering the above

questions. Ideally, no project, and especially no software project, is executed without adverse external influences. We plan to have

100% test coverage on the software p roducts that need to be delivered to the end users. But in a practical world this i s seldom

possible due to potential problems like delays in requirements and development, non-availability of required test environments on

time, budgetary constraints etc. Why do we need so much time for testing, Why do we need to do so many types of testing,Reduce the testing time, I will take the risk, let us release the software- are the first reactions by bus iness to IT teams and

reduction in testing time is the m ost common solution adopted by mos t companies during delays in early phases of s oftware

development lifecycles. However, IT teams especially testing teams must learn how to operate and still delivery high quality

software under such c ircumstances.Testing teams can start by prioritizing the tests that need to be performed and this is

determined by a set of key people working on the project like the Test Manager, Test Lead with the help of Business Analyst.

Though it is a problem based testing tec hnique, it is applied in the case of looming deadlines and limited resou rces. When

combined with the traditional ways to test the application, it limits the scope of the testing efforts. This technique is used to

prioritize the development and execution o f tests based on the two factors, impact and likelihood of failure of the functionality

or the aspect being tested. How and Why Risk Based Testing Formulating a risk based testing approach needs an understanding of

the risks involved before you start testing the product. There are going to be som e residual risks that cannot be eliminated. Risk-

Based testing helps to find the right quality (No high severity defects and very less low severity defects) level that can be delivered

within limited schedule and resources. This is done by identifying business and technical requirements for an application and

prioritizing these requirements on the basis o f factors likelihood and impact of failure. Risk o f a requirement can be assessedbased on:

A)Which requirements and attributes are critical for the success of the application/product and the business value of the same

based on the time to market. Business Analyst or Project Manager in consultation with the customer can come o ut with this

information.

B)How often is a feature/requirement used?

C)Is there is work around for the user to be able to use the application effectively for this requirement/feature?

D)How visible is a problem in a feature or attribute (for customers, users and people outside)?

E)Can we deliver without this requirement/feature?

Each requirement may result in m any test scenarios and hundreds of tes t cases. Testing team with their deep understanding of

business knowledge and end-user perspective must prioritize the test scenarios and test cases, based on risk analysis performed

on the requirements and identify the tests that need to be executed. Testing teams should analyze the requirement and test

scenarios and score them on following factors based on the business knowledge the team has acquired working on the same

project or business area: -Consequence of failure (CF) (For e.g., business impact of failure)

-Probability of failure (PF) (For e.g., size and complexity of a requirement, use of open source components)

-Vulnerability of failure (VF) (for e.g., features/requirements with lot of defects from past test execution cycles)

Scores the requirement on following factors: Functionality,Reliability,Usability,Efficiency,Maintainability and Portability.

8/8/2019 POC - RisK Based Testing

2/2

Risk based testing is only effective in teams with good business knowledge and end-user perspective. This will determine the risk

value of the requirement/test scenario. Even when testing teams do not have sufficient time to write detailed test cases, test

scenarios must be documented in detail with key validations to be performed during execution and prioritized. Formalizing this

process and incorporating this strategy in your projects will help the testing team deliver the right business deliverables on tim e

and with adequate test coverage. This technique makes it easy for the organizations to deliver the right level of quality in the

context of changing business requirements. It provides a critical level of objectivity in determining what things to test by using a

combination of business and technical requirements to prioritize the testing process. The c hallenging part though is how tomeasure success o f risk based testing strategy on a project. There are certain ways in which it can be achieved Return on

Investment (ROI), Delivered defect density, Code co verage analysis and Testing effectiveness.

Why is it Important?

The risk based approach su pports the strategy that can expose all the risks and eliminate them. Additionally, it also aims to reduce

the impact of residual risks that w ere not anticipated or eliminated. A com plete understanding of the system and its o perational

environment helps to understand the potential risks thereby establishing a w ell-defined scope of risk identification and

assessment thus enabling the testing team to prioritize the test cases to be executed in the order of the high risk to low riskrequirements and high risk test cases to low risk test cases for each of the requirements.

To Conclude, Risk Based Testing is an approach that req uires skill and experience to isolate the most impo rtant tests on the basis of

technical and business risks and constraints. Risk based testing is a powerful testing technique t hat enables the testing teams to

streamline their testing efforts. This knowledge helps in mitigating the risk and m inimizing the testing efforts, thus, bringing an

objectivity to test designing and test ma nagement activities. It enables testing teams to m ake informed decisions while defining exit

criteria. It is debatable if the Risk Based Testing is an art or science. It is science because, testing teams follow a technique/struc tured

process to arrive at the test cases to be executed to ensure acceptable quality of the product or application within the constrained

time lines. It is an art because deriving the test cases to be executed to obtain acceptable quality depends o n the knowledge, skill,

experience and judgment of the person prioritizing the tests. For me risk based testing is 50% science and 50% art. What do you think?

Please share your thoughts and e xperiences on risk based testing and whether it is a n art or science or com bination of both.