Assess with your blinkers off widen your scope for tool evaluation

INDIUM SOFTWAREAn Independent Software Testing Firm

Assess with your blinkers off - Widen your scope for Tool Evaluation

“Business Assurance through Quality Assurance”

© 2014 Indium Software

ENHANCING SOFTWARE QUALITY

2

Agenda

• Introduction

• Growing Trends

• Are we doing it Right?

• Refined Tool Evaluation

• Case Studies and Inferences

• Benefits of the Refined Approach

• Conclusion



3

Introduction

For reasons well known,

automation is important in

today’s world of testing

The focus should be on a

reliable automation solution that can last the

entire life span of a product

With varying platforms and new age n-tier applications it

is not advisable to stop with the life span of a

project

To beat the race process tweaks are

necessary to make an

automation project a success



4

Growing Trends



5

Are we doing it Right?

Steps involved in general tool evaluation• A few common parameters that seem to play a vital role in evaluating a tool are picked• A POC is conducted on the selected tool• Automation is kicked off

External sources that can impact tool evaluation• Personal influence of peers – Remember two experiences are never the same. Tool

Evaluation should not biased.• Resources’ skill set – Picking the wrong tool may end up more expensive than training a

resource.• Closed mind – Be open and up to date on what is available in the market.



6

Refined Tool Evaluation

Select the Best

PoC

Pick top 4

Determine tool capabilitiesUnderstanding Application

• Technologies and techniques used

• Third party components used

• Architecture

• Feature knowledge

• Review all test cases

• Interacting interfaces

Refined Approach



7


Select the Best

PoC

Pick top 4


• Features supported

• Skill required

• Cost

• Support



8


Select the Best

PoC

Pick top 4




9


Select the Best

PoC

Pick top 4


• Real device scenarios

• Web services, message queues and database absorption layers

• Interface interaction

• Synchronization between client and server

• Plug in and devices

• Database / ODBC

• Communication protocol

• Client / server response time



10


Select the Best

PoC

Pick top 4




11

Case Study 1

Challenges

• Selecting a tool that supports all the three platforms to ensure:

• Coverage of device integrated test cases• Reduce cost

• Some of the licensed tools had limited or no support to the browsers like Chrome and Firefox

Benefit of Refined Process

• Promised support for current / upcoming platforms• Complete automation test coverage• Significant cost savings on license

Client

• Leading windshield provider in US

Domain

• Retail POS

Platforms

•Desktop – VB / SQL

•Web – Application in development (Firefox, Chrome and IE)

•Mobile–Sync data between mobile and desktop (Android)



12

Inferences

The number of automated candidates has drastically increased by ~ 25%

The increase in automation test coverage has reduced manual effort by 50%

Defects found in critical areas have increased by 80%



13

Case Study 2

Challenges• Selecting a tool that supports UI and UI less layers testing• Testing all the service layers in the application such as:

• Web Services• Metadata Services• Integration Services• Workflow Services

Benefit of Refined Process• 100 % automation coverage and solution• Reducing the risk of service layer failures• Early defect detection• Significant cost savings on license• Increased test coverage and shorten testing cycles• Improving the overall quality by testing services and its consumers

Client

•Leading Automobile Reseller

Domain

•CRM

Platforms

•Web – via only IE•Plug-in – via Outlook



14

Inferences

Knowing the application before hand and picking the right tool does have an impact on the UI test coverage ~ by 10%

Test coverage on Service layers have increased by 55% therefore gaining confidence in delivery as not just the UI has been tested but also the Service layers

Automation defects detection has increased by 50% with our refined approach



15

Benefits of the Refined Approach

Key Levers With Refined Approach Value Additions

Test Coverage Increased by 30%

Ensuring test coverage on interacting devices across platforms

Reducing the risk of service layer failures

Test Execution Faster by 40% With more automated test cases, manual test time is saved

Early Defect Detection Faster by 10%

Early defect detection in critical performance areas, service layer defects help reduce UI defects

Very easy root cause analysis of defects

Cost Increased bottom line No need to invest on a different tool for

different devices and Service layers Significant cost saving on license cost



16

Conclusion

Tools and Technologies growing at a rapid speed, you can be sure to beat the competition if you just open up to

assess with your blinkers off!



17

Thank You

USA: [email protected]

EMEA & APAC: [email protected]@indiumsoft.com

INDIA: [email protected]

SunnyvaleSuite 210, 1250 Oakmead ParkwaySunnyvale, CA – 94085.Phone: +1(408) 501-8844Fax: +1(408) 501-8808

Atlanta Crown Office Suites1870 The Exchange, Suite 100 Atlanta, GA 30339Phone: +1 (678) 608-0809, +1(770) 989-7302

New DelhiF-1/5 Okhla Industrial Area Phase-I, New Delhi – 110 020.Phone: +91-11-6613 0400

BengaluruSRS Plaza, No. 246, 2nd & 3rd Floor13th Cross, Indira Nagar 2nd StageBengaluru – 560 038.Phone: +91-80-4914 7777

ChennaiVDS House41, 2nd Floor, Cathedral RoadChennai – 600 086.Phone: +91-44-2811 6330Fax: +91-44-4210 4033

mailto:[email protected]




https://www.facebook.com/pages/Indium-Software/290116636922

http://www.linkedin.com/company/82503

https://twitter.com/indiumsoft

Technology

Assess with your blinkers off widen your scope for tool evaluation