Thursday, 9:45 AM April 10:45 7th 2011 AM QUEST Boston 2011 · • “Test coverage” – Supports assessing the business risk associated to a release o Derived from test requirements

2011

Transforming Outsourcing Relationships

QUEST Boston

Thursday, April 7th, 2011

9:45 AM – 10:45 AM

PRESENTER:

Yann Gloaguen

COMPANY:

SQS Group

This was page intentionally left blank

2008-01-20

© SQS Group Limited | Presentation title | July 2008 | page ‹Nr.›

SQS Group Limited

Yann GloaguenApril 2011

Transforming Outsourcing Relationships

QUEST 2011 Boston

Presentation Brief

Objectives:When testing functions are outsourced, the various partners and

stakeholders often perceive success differently. In this presentation we will: 1. Highlight common engagement challenges 2. Cover ways of turning around such engagements3. Outline what is shaping up to be the next generation of governance model

© SQS Group Ltd. | QUEST 2011 Boston | Transforming Outsoursing Relationships | Page 2

2008-01-20


Common engagement challenges

Service delivery is approached differently by buyers and suppliers

IT services buyer’s viewIT services buyer’s view IT services supplier’s viewIT services supplier’s view

Quality As high as possible As high as needed

Quantity As much as possible As much as needed

Resources Same team of hand-picked consultants

Career development, cost effectivness and attrition

Cost Fixed, predictable Risk adverse

Timing Need flexibility and responsiveness Need visibility


2008-01-20


Service delivery is often governed by traditional service levelagreements (SLA)

Identify discrete quantitative measures &

measurements

Define SLA agreeing target per performance indicator


Traditional SLAs made up of discrete quantitative “performance indicators” fuel inappropriate governance

• Fact #1: “performance indicators” drive behaviour

• Fact #2: “Staff retention”, “capacity buffer”, “productivity”, “defect detection”, “milestone met” and “test coverage” are common performance indicators

• Fact #3: “Common” performance indicators are defined differently from one organisation to another, from one IT services supplier to another

• Fact #4: IT services buyers often overlook performance indicators’measurements

• Fact #5: IT professionals are too often left on the side-line during contract negotiations

• Consequence #1: Performance of IT services suppliers is inconsistently perceived by stakeholders

• Consequence #2: Frustration, mother of stagnancy and ineffectiveness, kicks-in


2008-01-20


Example biased performance indicators – “Staff retention”

• “Staff retention” - Reflects knowledge retention from the supplier: o “planned / unplanned staff loss”o “rolling period”

• 75% staff retention? o Means 30, of the initial 40, still in the team at the end of the term?o Or a maximum of 6 testers to have left the team at the end of the term? (34

remaining from the initial team of 40, 6 new joiners: 73.9% staff retention)


Example biased performance indicators – “Productivity”

• “Productivity” – Defined as “quantity or quality of output to the inputs required to produce it”

o Often associated with the execution phase of a projecto Extrapolated to other testing phases

• Number of tests executed per person per day?o Wont testers create ever smaller tests as releases go on?o The smaller the tests, the least test points per test, the more tests to run per

release… A business case for a headcount increase?


2008-01-20


Example biased performance indicators – “Test coverage”

• “Test coverage” – Supports assessing the business risk associated to a release

o Derived from test requirements o Where test requirements are derived from functional specificationso Functional specification being written in-light of business requirements

• So what is it?o Number of tests executed over the number of tests in scope? o Ratio of test requirements covered by tests over the overall number of test

requirements?o Or could it be the number of test requirements covered by ran tests over the total

number of test requirements?


Example biased performance indicators – “Test coverage”

• “Percentage milestones met” – Impacts “time-to-market”o Measure responsiveness to business demand o Only report on the ones that will directly aversively impact on the business

• So what are they?o They aren’t tasks, don’t have a duration i.e. no “allowed deviance”o They are hand-picked and kept to a minimum; a high number will:

Dilute failuresPrevent root cause analysisFoil efficient process improvements

o They are programme specific


… And they all are “SMART” performance indicators..!

2008-01-20


When SMART is not smart enough

• Performance indicators cannot only be SMART:o “S” – Specifico “M” – Measurableo “A” – Attainableo “R” – Relevanto “T” – Time-bound

• Performance indicators should also be:o Understoodo Impact-assessedo Standard


The beauty of the beast

And what was shaped to be a roaring success…

… at best triggers dismissive shrugs


2008-01-20


4 steps to turning around endangered engagements

Step 1 – Confirm high-level expectations for the service delivered

IT services buyer’s viewIT services buyer’s view IT services supplier’s viewIT services supplier’s view

Quality Honest endeavour, taking in to account the agreed team profile (i.e. Skills and availability)

Quantity Honest endeavour, taking in to account dependencies and known limitations (e.g. Toolset, headcount)

Resources It is in the buyer and supplier’s interest to manage resources involved

Cost Both parties run a business, the contract must be mutually profitable

Timing Extra-flexibility shall be granted, at least at a premium, responsiveness derived from visibility


2008-01-20


Step 2 – Conduct a health-check & follow-up on recommendations


Engagement Management

Test Vendor Management Testing Personnel Resourcing Test Process

Improvement Test Metrics

Test Centre

Test Management and Functional Testing Services

Test Management Test QA Role Test Strategy/Approach Test Reporting Test Process and

Methodology Phase Containment

Functional Test Scripting Test Repository Test Estimation Test Planning Functional Test

Coverage Defect Management

Test Automation Unit TestingSystem and

Integration Test Execution

UAT BAU Testing

Strategy and Approach Scripting Test Scenarios Execution and

Analysis Reporting OAT

Performance & Non-Functional Testing

Environment Management

Configuration Management

Release Management

Test Environments Management

Functional Requirements

Non functional Requirements

Requirements

Step 3 - Combine discrete quantitative measures for an effective governance

Identify discrete quantitative measures &

measurements

Define performance indicators from quantitative measures

Define SLA agreeing target per performance indicator


2008-01-20


Step 4 – Confirm roles and responsibility


Business Analyst Project Manager Head of Testing Application Owner IT Security Account

ManagementOn-site Test

ManagerOff-site Test

Manager Off-site IT Sec

Application identification C C C C A R RUse OAR to select pilot applications(s) C C C C A R RMeet with Project stakeholders C R A R C C C C CAgree scope of pilots with application stakeholders C R R C I A C C ITrain consultants in applications and technologies C C C I A R RStart work on pilot application(s) I I I I I A R R IStructured reporting to application stakeholders C C C I A R RProgress/Status reporting I C C C I A R R CDetailed planning C C I I I A R R IOffshore testing fully established I I I I I A R R IProgress/status reporting I I I I I A R R CUse OAR to prioritise additional applications I I I I I A R R C

Pre-Initiation Establish connectivity approach I C C I C A C C RSetup connectivity I I I I R A I I RTest connectivity I I I I R A I I R

Transfer Monitor connectivity and enhance if required I I I I R A C C R

Steady State Monitor connectivity and enhance if required I I I I R A C C R

Pre-Initiation Agree high level SLAs I C R C C A R C CAgree metrics to measure KPIs I C R C C A R C CProduce plans C C C C C A R R RDefine governance and communication model C C R C C A R R RDevelop dashboards C C C C C A R R RDevelop reports C C C C C A R R RInitiate service reviews I I R I I A R R IReporting against SLAs I I I I I A R R RRegular service reviews I I I I I A R R RFeedback and process improvement C C C C C A R R R

Pre-Initiation Agree knowledge acquisition approach C C C C A R R I

Initiation Knowledge acquisition plan C C C C A R R IBring consultants onshore to work with project teams I C C I I A R R ISend domain experts offshore to train test team C C C I I A R R IWebinars I I I I A R R IApplication Test Approach Cheat Sheets creation C I I I A R RReadiness monitoring and reporting I I I I I A R R IShadowing programme I C I I A R R IPerform training for new resources C I A R R

Pre-Initiation Agree types of resources required C C A R RIdentify resources I A R RInitiate training I I A R RRamp up resources starting with the key roles I I A R R IAssign resources to target applications I I A R R

Steady State Retain knowledge through challenging roles C A R R

Pre-Initiation Agree target testing tools C C C A R R I

Initiation Select tools based on POC I I I A R R IDevelop tools’ frameworks C I I I I A R R ILicense and software procurement R A I C C IImplement tools I I A R R IMaintain tools I I A R R REnhance frameworks for efficiency C I I A R R I

Environment architecture hosting assessment C R C A C C RAgree hosting solution C C C A RAgree ownership C C C A RLicense and software procurement R A I C C IEnvironment set-up I C I A R I C C CEnvironment proving C I I A C I R R C

Transfer Maintenance processes defintion I C I R R A I I R

Steady State Monitoring I I A R I I I R

A - AccountableR - ResponsibleC - ConsultedI - Informed

Steady State

Initiation

Transfer

Managed Testing ServiceRACI Matrix

Managed Testing Service Activity

Initiation

Transfer

Client Organisation SQS

Testing Activities

Connectivity

Governance

Knowledge Management

Pre-Initiation

Initiation

Initiation

Transfer

Steady State

Transfer

Steady State

Transfer

Steady State

Pre-Initiation

InitiationEnvironment

Managed Testing Service Aspect

Managed Testing Service Phase

Resourcing

Tooling

• Document “RACI”o A - Accountabilityo R - Responsibilityo C - Consultedo I - Informed

The next generation of outsourced engagements

2008-01-20


IT services buyers need business value & performance

• The demand-led global IT market has been the spring under which Indian IT services company have placed their buckets.

• Market for such commoditized IT services is drying up; IT needs are shifting towards value led delivery

• Only a case for business performance or value measured by revenue, profit, customer satisfaction and market share justifies IT expenditure

• IT services buyers are contemplating de-risking IT investments shifting from commoditized services models aimed at operational cost-cutting, to business performance driven contracts

• IT services buyers need their partner to provide them with a referential depicting where their business is in their world

• Operational effectiveness still is crucial, business value through innovation & performance becomes paramount


Pricing models – some new front-liners

• Output basedo De-risk your IT servicingo Pay for what is deliveredo Control, plan and cost what needs to be delivered early

• Value Basedo Reward over-achievementso Penalized under-achievementso Promote innovation and operation effectiveness


2008-01-20


Innovation – Do you allow for some?


• Do you allow for the right skills on the service?o Starving off the engagement from leads and architects will prevent innovation

• Do you empower supplier towards change?o The end product of innovation is change

• Share roadmap & visiono Is your supplier bedded in your vertical (Organization & business line)

Thank you for your attention

SQS Group LimitedYann Gloaguen7 -11 Moorgate LondonEC2R 6AFUnited Kingdom

Phone: 0044 (0) 207 448 4620Email: [email protected]

Documents

Thursday, 9:45 AM April 10:45 7th 2011 AM QUEST Boston 2011 · • “Test coverage” – Supports assessing the business risk associated to a release o Derived from test requirements