A more successful model for multi-shore testing

12008, Perficient Confidential

A more successful model for multi-shore testing

Ken McCorkellJanuary 28, 2010

The contents of this presentation are the sole copyrighted property of Perficient Inc and may not be reproduced in whole or in part without written permission from Perficient Inc.


Agenda

Introduction When should you build more code? Test Automation Strategies Building custom automation frameworks Agile Testing Strategies for testing in a multi-shore environment Q&A


Fast Facts

Founded in 1997

Public, NASDAQ: PRFT

~$250 million in annualized revenues

Locations in 19 major North American markets

Global Delivery Centers in Europe and China

1400+ technology consultants

Dedicated solution practices

Served 600+ clients in 2007

Alliance partnerships with major technology vendors

Multiple vendor/industry technology and growth awards


Perficient’s Global Delivery Center – Key Facts

Our Strengths Fully owned and operated since 2004 Operating at SEI CMMI Level 5 Worldwide Leader! – One of the first

to achieve CMMI using an Agile Methodology

20-40% average organic growth strategy

Located in Hangzhou, China•About 2.5 hours from Shanghai•Home to prestigious universities•Large City - Pop. 6.5M+•#1 city to do business in China – Forbes (5 years running!)

•Excellent talent pool- home to over 1200 technology enterprises

•All business conducted in EnglishChina Global Delivery Center

Best Business City in ChinaBest Business City in China


Perficient’s ‘3D’ Testing Approach

Breadth / Depth

Lif

ecyc

le

Inn

ova

tio

n

AgileApproach

ClassicWaterfall

ComprehensiveTesting

2 DimensionalTesting

Testing asa Necessity

Testing asa strategic

differentiator

Functional testing Usability testing Accessibility testing Compatibility testing Load testing Stress testing Compliance testing Internationalization testing Localization verification API / interface testing Installation validation Test data generation / masking

Elimination of traceability matrices Continuous build, integration and testing Test first principles (development best practice) Planned regression testing Early integration testing (steel threads) Polymorphic development team roles

High degree of automation Automation framework Flexible COTS / open source toolkit Multi-shore as a strategic advantage

Characteristics:

• Based on Agile best practices where testing is integrated into the full lifecycle

• Covers the ‘big-picture’ of testing; not just relegating most testing to an ‘after-thought’

• A high degree of leading edge innovation in how testing is approached and how it can be leveraged in the organization as a critical practice

Characteristics:

• Based on Agile best practices where testing is integrated into the full lifecycle

• Covers the ‘big-picture’ of testing; not just relegating most testing to an ‘after-thought’

• A high degree of leading edge innovation in how testing is approached and how it can be leveraged in the organization as a critical practice


Agenda



When should you build more code?

The answer is – when you have successfully validated the delivered code

The key to a successful project is being able to quickly and effectively identify the quality of the application under test

For a multi-shore project this can be achieved with:Automation and test frameworks, an agile integrated testing model, and visibility and communication across the process

In an agile testing model code is verified at least every iterationIn an agile testing model code is verified at least every iteration


Technical debt and untested code

Technical Debt of untested featuresStacks up

# of Code modules without tests

Code dev without testing

“Technical debt occurs incurs when a team chooses an approach that's expedient in the short term but that increases complexity and is more costly in the long term.”

Technical Debt


Agenda



Why should you use test automation?

1.Automation is essential to ensure constant communication between testing and development efforts

2.Quick visibility into code quality3.Main reason: Automation is often the only way to

generate the velocity for testing needed to release fully tested builds and potentially shippable projects in short iterations


Automated testing success

Did you know more than 50% of test automation projects fail to meet their goals?

Reasons for this low success rate include:1.Expensive tools and technology are seen as the

answer, but the processes are actually the critical elements

2.Automation projects are not planned as an integrated part of the development process

3.Automation coding is not planned with the same rigor as application coding

4.The proper resources and time are not allocated to building up a robust automation framework


How to best use Automated Testing

Automated tests should be used for: automating the most repeated tests replacing tests that are tedious to execute

manually testing on multiple OS or environment

configurations creating regression for legacy applications by

implementing a set of tests each iteration

The actual goal is usually not to automate 100% of all possible tests, but the most high value tests

The actual goal is usually not to automate 100% of all possible tests, but the most high value tests


Open Source as a Test Strategy

Consider using open source testing tools or a combination of open source and COTS

Selenium IDE WebTest

Mercury LoadRunner, Mercury QuickTest Pro, Mercury Quality Center, Mercury WinRunner

SOAPUI WebService Testing

Apache JMeter Performance Test

Junit, HTTPUnit, Nunit

TestNG


Agenda



Automated Test Frameworks

Automated Test Frameworks are essential to providing efficient automation scripts based on the specific testing environment

Keep these principles in mind: Do not simply record and playback tests Parameterize data use for tests – Data-Driven Testing Parameterize field names for easy maintenance Write reusable modules for common test functions Make tests atomic – tests should not depend on other

tests

Remember: automated tests are code, and should be planned just like application code

Remember: automated tests are code, and should be planned just like application code


Agenda



Agile Testing: Traditional Testing vs Agile Testing

Traditional Testing Model: Separate test group from development Tests are derived from detailed requirements instead of

being part of requirements Testing is conducted after development is complete Defects are written to communicate with developers Developers and Testers are working against each other Automation is a nice-to-have instead of a must-have

This traditional approach can work if there is sufficient time and budget for re-testing and coding cycles

This traditional approach can work if there is sufficient time and budget for re-testing and coding cycles


Agile Testing: Traditional Testing vs Agile Testing

Agile Testing Model: Testers are part of the development team Team works closely with customers to define acceptance

tests for each requirement Testers are included from the beginning of the

requirements and design Test each feature early as it is completed Paired testing between testers and developers Provides continuous feedback to development in person

and with up-to-date test metrics and automation results


Combining Agile with Automation

Use continuous integration (CI) to automatically build and test

Test-driven development creates failing unit tests first then writes code that makes the test pass

Task tracking and burn-down reports can show progress on coding and automation tasks in one system

Popular CI tools include Hudson, Cruise Control, Maven, and antPopular CI tools include Hudson, Cruise Control, Maven, and ant


Agenda



Multi-shore testing

What does all of this fit in with multi-shore testing?

Multi-shore testing should pay even more attention to agile testing and automation, because they are essential for communication and visibility across the testing process

Visibility provided by automation and agile approaches closes gap between onshore and offshore teams

Maximize efficiency by practicing the “follow-the-sun” pattern with shared areas of responsibility


Multi-shore communication

Communication with multi-shore teams is difficult to integrate with a truly agile process

Reduce time between checkpoints to avoid communication gaps

Communication gap

offshore

onshore

time

last checkpoint

Cross-shore communication


Visibility and communication

• Daily Level- Daily stand-ups- Daily or hourly build of the software including running

tests- Team self-organizes and discusses problems as they

happen- Daily question and answer on the project Wiki page- Q&A sessions on instant messenger for off-hours times- Daily posting of knowledge learned to the knowledge-

sharing space on the wiki


Visibility and Communication (cont.)

• Weekly Level- Weekly status report to the Project Management Organization

(PMO) to escalate and communicate issues- Weekly test reports are sent out with testing progress and bug

reports

• Iteration Level (1- 4 weeks )- Demo to all interested parties including customer (real working

code demo). - This gives real visibility about what the team has actually

produced- Retrospectives (these are important!)

• Project Level- Project retrospective with lessons learned and feedback to the

process patterns- New process patterns that worked well get rolled back into the

process pattern templates

Keep to the fixed schedule for communication checkpoints

Keep to the fixed schedule for communication checkpoints


Summary

Best Practices for multi-shore testing Test Automation should be used as a communication tool for

product quality Testers and developers should be paired on areas of

functionality to share knowledge across shores Code repositories and dev/test environments should be

shared by cross-shore teams Communication tools such as: wikis, task management

systems, daily Q&A sessions, IM, and sync-up calls are essential to visibility across the team

Consider a first step of starting an offshore test initiative to maintain test automation scripts

Most importantly, treat the resources all as one team!Most importantly, treat the resources all as one team!



Provider Healthcare Analytics and Driving Quality Outcome Measurements

Thursday, February 25, 201012:00 - 1:00PM CST

Next Month:

Healthcare analytics are of great importance in terms of both providing quality of care and in meeting increasing regulatory reporting requirements. Consider the following:

• Healthcare insurance organizations are implementing pay-for-performance measures, including CMS, to change or create better healthcare outcomes• ARRA stimulus dollars require providers to report on "meaningful use" of electronic health records and to report HHS quality measures• Physician alignment necessitates transparency in reporting outcomes and performance expectation• Healthcare providers must provide outcome results as part of re-inventing service delivery

Join Perficient as we discuss how healthcare analytics can improve your outcomes:

• The business case for why healthcare analytics matters to progressive health systems• A look at some of the market-leading healthcare analytics products and their associated architectures• Case studies of organizations that have begun to address the value of healthcare information

www.perficient.com/webinars

Technology

A more successful model for multi-shore testing