Fonterra – Our Testing Journey - ANZTB Ludlam - Fonterra Our Testing... · Agenda • Introduction to Fonterra, Information Services and our system landscape • Preparing for the

Fonterra – Our Testing JourneySherry Ludlam – Test Capability Team Lead

Confidential to Fonterra Co-operative Group

16th March 2011

Agenda

• Introduction to Fonterra, Information Services and our system landscape

• Preparing for the journey

• Starting the journey

• Paving the way

Page 2


– Test strategy and capability – processes, team and tools

• Reaching our destination

• Questions

Introduction to Fonterra

• Dairy is our life's work. It's our passion and it's what we do best.

• “Fonterra” = “Flowing from the Earth”

• Fonterra Co-operative Group: Farmer suppliers are Shareholders

• World's No 1 Milk Processor by annual volume

• World’s largest Dairy Exporter

Page 3


• 96% of NZ dairy farmers are with Fonterra

• Exports 95% of New Zealand-made dairy products to 140 countries

• Account for 20% of NZ annual export earnings – turnover NZ$16b pa

• Responsible for more than a third of international dairy trade

• Employees – 15,600

• Shareholders 10,537

Introduction to Fonterra – Our ProductsConsumer Brands Ingredients

Page 4


FoodservicesEver wondered where that great tasting melted mozzarella on your order-in pizza comes from?

Foodservices work behind the scenes to meet the different needs of chefs, caterers and restaurateurs.

Introduction to Fonterra

Page 5


Fonterra Information Services

• Shared service into the Fonterra business units

• Employs 150 in Fonterra IS, with over 800 people working at any one time on projects

• Supports 46 countries

Page 6


• Multi-vendor organisation

– Application Development and Maintenance – outsourced to HCL

• Testing performed by HCL or project resources

– Infrastructure support – outsourced to various vendors

Fonterra System Landscape

• Core system – SAP

– Currently 35-40% of our business is using SAP, which is increasing to 85% over the next 3 years

• ECC

• Includes Sales, Logistics, Warehouse Management, Finance, Materials Management, Asset

Management, HR

• SCM

• Planning

Page 7


• SRM, CRM, BW, GTS

• Over 200 custom applications

– Spread across the business units

– Slowly being phased out

• 5,200 registered to use SAP, with another 500 coming with Australia SAP implementation

Preparing for the journey


Page 8

Preparing for the journey

• Legacy systems amalgamated at the inception of Fonterra

– Manufacturing, warehouse management, financials

• Implemented SAP in June 2004

• Have continued to grow our suite of modules and functionality, removing legacy systems along the way

Page 9


• No set processes or tools to manage testing in any of our environments

– Testing was manual

– No structured test management

– Recreating tests every time a change was made

• Attempted to use Solution Manager (badly) to manage SAP testing

Preparing for the journey continued

• In 2006 the need for better testing processes, standards, practices and management was identified

• Initial business case was put together to implement a full testing solution, including processes, tools, training, automation and performance testing

– Resources - $5m, 23 resources, 11 months

• Test manager and analyst were hired to put basic test processes in place, but full solution not

Page 10


• Test manager and analyst were hired to put basic test processes in place, but full solution not implemented

– Outsourcing occurred!

• 2008 – used a project implementing the SAP WMS solution as a pilot to trial HP Quality Center

• 2009 - Large push from business to put more and more of our footprint onto the SAP landscape, so IS needed to implement strategy for upgrading, protecting, maintaining and managing the SAP environment

The start of the journey


Page 11

The roadmap

• Pyramid

– Programme Pyramid started in August 2009 to create a more sustainable SAP environment, enable faster implementations and reduce TCO

• Re-architecture

– Put SAP onto a virtualised platform

• Solution Manager – upgrade to up-to-date version

Page 12


– Enable better management and monitoring

• Test Strategy & Capability

– Implement test vision

• Support & Enhancement pack

– Bring SAP software version up to n-1

• SAP Centre of Excellence

– Governance and management of SAP

Paving the way


Page 13

Testing strategy and capability - vision

• To be the go-to team for all things systems testing in Fonterra

• To provide a valuable service to Fonterra IS, it’s service providers and ultimately the business by improving testing standards and efficiency

• To define and implement efficient and effective management of testing in Fonterra, with supporting tools, skills and processes

Page 14


supporting tools, skills and processes

• To up skill Fonterra performers in testing roles and responsibilities

Process – how we did it

• Project team was setup to build Test Capability Team and Processes

– Built and defined charter, team processes, testing processes and documentation templates

– Metrics were designed to help us measure success

• Tools

– Implemented SAP Test Acceleration Optimisation in association with Solution Manager, HP Quality Center (HPQC) and HP Quick Test Professional (HP QTP)

Page 15


• Utilised specialist TAO resource from India to assist in implementation

• Implemented TAO version 2.0 due to addressing earlier issues with TAO 1.0.

• This also enabled better integration with HPQC

• We were the first in New Zealand to implement full suite

• Baseline Test Suite

– Identified core business processes that met automation criteria

– Team worked closely with TAO specialist to build automated test scripts

Test Capability Team - structure

TEST CAPABILITY TEAM LEAD

QUALITY MANAGEMENT

IS SERVICE DELIVERY

PROCESS OWNER

GOVERNANCE

Page 16


TEST PORTFOLIO MANAGERS TEST ENGINEERS TEST CONSULTANTS

Testing Management & Governance

Test Administration & Maintenance

Out Sourced Test Resources

• Test engagement with Test Managers• Resource and tool licence planning

• Test checkpoint sign offs• Reporting• Coaching

• Test design and build• Test automation

• Baseline test suite maintenance• Test training• Test tool administration

• Test automation • Test tool skills

RESPONSIBILITIES

Processes and documentation

Testing Methodology

EPO Project Methodology

Concept Business Case

Mobilisation Business Requirements

Analysis & Design

Build & Test Final Preparation

Support & Sustain

ClosureDesign Analysis

(including Build and Test)Requirements Analysis

- Solution Sizing and Approval

Enhancement Methodology

Page 17


Key

Testing check point gate

Test Strategy & Planning Test Close OutTest Design

& Preparation

Testing Deliverables

Biz CaseTest ImpactAssessment

TestEngagementAgreement

Test Strategy Test Plan

Test Cases(manual tests)

Test Scripts(automated tests)

Open Defectreport

Test Summaryreport

Test PIR

Test Execution & Evaluation

How do people know about us?

• TCT Sharepoint site

– Contains all our documentation, templates, metrics and training material

• Integration with Project Management Office

• Proactivity around projects – get in early to sell the message

Page 18


• Proactivity around projects – get in early to sell the message

• Computer Based Training

– Tools and processes

• Regular communications (newsletters), classroom training and test inductions

Tools – HP Quality Center

An application that supports test management

� A library of our test cases

Key features of HPQC

What is HPQC ?

Page 19


� A library of our test cases (manual) and test scripts (automated)

� Manages defects and test execution

� Enables a repeatable use of test cases and test scripts

� Reports the status of testing and defects

Tools – SAP Test Acceleration Optimisation

An SAP tool to build automated test scripts

� Records SAP transaction

Key features of SAP TAO

What is SAP TAO?

Page 20


� Records SAP transaction screens and keystrokes which can be replayed as a automated test.

� Integrated with HPQC which runs the automated TAO test scripts

� Can analyse what has changed in the SAP solution and flag which test scripts will be out of date and require amendment

Tools – Solution Manager

An SAP application to support our SAP asset

Features of Solution Manager

What is the Solution Manager?

Page 21


� Provides tools to support and monitor our SAP environment

� Provides a Business Process Repository (BPR)

� Delivers an assessment of the impact of a change to production systems

� Enables the scope of regression testing to be identified

Features of Solution Manager

The integrated solution

(HP QTP)

Page 22


HPQC structure

Baseline Test Suite

Projects Testing

Page 23


Project 1

Project 2

Enhancement 1

Enhancement 2

Enhancement 3

Enhancements Testing

Project 3

Project 4

Performance testing

• Mid 2010 – Load Runner purchased to complete performance testing

– Have executed 3 performance tests since implementation

– Very costly exercise due to resource cost and specialised SAP monitoring resource

MB31 Business Transaction (without think time)

Page 24


MB31 Businesstransaction with login andlogout

MB31 Businesstransaction without loginand logout

MB31 Businesstransaction. All 7 stepswith login and logout

Reaching our destination


Page 25

Outcomes

• Consistency of approach across all projects and enhancements

• Better definition of test scope for changes

• Reusable test cases (manual) and test scripts (automated)

• Better understanding of our business solution

Page 26


– Documented business scenarios that can be found easily, with related system transactions

• Beginning to gather data to allow more accurate estimates re testing time and effort required for testing

• Tool suite implemented that supports a large part of our system landscape and business

Celebrating success

• The new solution has already shown proven results:

– Training data creation

– Regression tests for multiple SAP projects

– Identification of issues on other projects (2 very significant!)

– Two rounds of regression testing across our Quarterly Releases

• Automated test scripts – 73

– Run 284 times since May 2010$82,558

$90,000

Cumulative Cost Savings

Page 27


– Run 284 times since May 2010

• Manual test cases – 1419

• Cost savings - $222, 352

– Only related to automated test

Execution

• Test management benefits

– Proactivity

– Visibility

– Reporting

$7,363

$49,071

$14,536

$-

$34,867 $33,956

$-

$10,000

$20,000

$30,000

$40,000

$50,000

$60,000

$70,000

$80,000

HR QTI Finance WM Planning MD FAM

NZ

D

Business Areas

Cumulative Cost Savings

Challenges

• Data maintenance and identification

• Script maintenance

• Resource availability for issue resolution and support

– Building of scripts

– Running of scripts

Page 28


• Automation requires business knowledge and automation tool knowledge – hard to get in one person

• “Selling the tool and team”

– Processes and tools implemented have been seen as increased cost, due to the effort required to produce proper documentation of test cases/scripts and maintaining the solution

– However, once it is used by a project once, we get huge buy-in

– Necessary to protect our solution and production environment

Continuing the journey

• Upgrade to ALM (Application Lifecycle Management) and Performance Center (from Load Runner)

• Expand scope from Test Capability to Test Services

– Move more towards total test control in the Fonterra environment

• Work alongside two major business projects to grow solution and automation

– Implementation of SAP in ANZ and remediation of NZ implementation

Page 29


– Implementation of SAP in ANZ and remediation of NZ implementation

• Improve Quality Control – Test Portfolio Manager’s more involved in projects, more detailed review and sign-off

– How do we justify and resource?

• Improve metrics to better track progress

• Identify opportunities for continuous improvement by quarterly surveys


Page 30

Any questions?

The End


Page 31

Documents

Fonterra – Our Testing Journey - ANZTB Ludlam - Fonterra Our Testing... · Agenda • Introduction to Fonterra, Information Services and our system landscape • Preparing for the