Pwc Systems Implementation Lessons Learned

ISACASystems Implementation Assurance – Lessons Learned

February 2009

Systems Implementation Assurance – Lessons LearnedPricewaterhouseCoopers

Agenda – Lessons Learned

1. Project Phase 1- Planning / Mobilization

2. Project Phase 2 – Design / Blueprint

3. Project Phase 3 – Realization / Build & Test

4. Project Phase 4 - Pre Go-live / Deliver Phase

5. Project Phase 5 - Post Go-live / Maintenance Phase

6. Example Project Discussion Document


Phase 1- Planning/Mobilization

Careful planning, particularly in the early stages of a project, is necessary to coordinate activities and manage project risks effectively. The depth and formality of project plans should be commensurate with the characteristics and risks of a given project.

Outline Project Plan

Define Roles and Responsibilities

Define Project Communication and Reporting Requirements

Define Deliverables and Expectations – Involvement of all Key Players

Outline Risk Acceptance - Manage Internal and External Risks

Define Project oversight activities – Definition of Standards

Define Tollgates and Requirements

Define Budget and estimated Project Costs

Define Project Change Procedures


Phase 1 Planning/ Mobilization – Lessons Learned

Putting a proper project governance structure in place with sufficient "checks and balances".

Proper Executive and Senior Management buy-in and involvement in project and milestones reached

Projects are often comprised of international teams and must consider both cultural issues and compliance with local laws and regulations

Broader industry and business issues must be taken into consideration


Phase 1 Planning/Mobilization – Lessons Learned cont.

Underlying Data Model Consideration (e.g. US GAAP versus

IFRS)

Downstream impact on support functions such as internal audit and security administration

Additional Considerations to be aware of during the planning stage:

41% of projects fail to meet management’s objectives

Only 28% of project fulfill management's expectations

Only 16% of IT projects hit all their targets

50% of projects end up late or over budget


Planning/Mobilization – Lessons Learned cont.

Reasons for project failure in the planning stage: Bad estimates

Scope changes

Change in environment

Insufficient resources

Change in strategy

Imprecise goals/ Insufficient budget

Poor communication

Insufficient support

Wrong project management

Insufficient motivation

Stakeholders not adequately defined

Poor quality of deliverables


Project Phase 2 - Design/Blueprint

The design phase involves converting the informational, functional, and network requirements identified during the initiation and planning phases into unified design specifications that developers use to script programs during the development phase

Application Control Standards Designing appropriate security, audit, and automated controls

Standards should be in place to ensure end users, network administrators, auditors, and security personnel are appropriately involved during initial project phases.

Application control standards enhance the security, integrity, and reliability of automated systems by ensuring input, processed, and output information is authorized, accurate, complete, and secure.

Automated input controls help ensure employees accurately input information, systems properly record input, and systems either reject, or accept and record, input errors for later review and correction (e.g. Check Digits, Completeness Checks, Duplication Checks, Validity Checks, Reasonableness Checks, etc.)


Project Phase 2 - Design/Blueprint cont.

Processing Controls - Automated processing controls help ensure systems accurately process and record information and either reject, or process and record, errors for later review and correction.

• Batch Controls

• Error Reporting

• Transaction Logs

• Run-to Run Totals

• Sequence Checks

Output Controls - Automated output controls help ensure systems securely maintain and properly distribute processed information


Phase 2 Design/Blueprint – Lessons Learned

Avoid excessive customization - companies desire to "re-invent the wheel"

Many key controls are application driven (e.g. controls which depend on system generated reports, configuration settings such as for the three-way match in the procurement cycle)

Effective process to prioritize all the business "wish-lists”

Decision Making from “Middle Management” – Timely Decisions


Project Phase 3 - Realization/Build & Test

Development

Development standards should be in place to address the responsibilities of application and system programmers. Application programmers are responsible for developing and maintaining end-user application.

Library Controls - Libraries are collections of stored documentation, programs, and data. Program libraries include reusable program routines or modules stored in source or object code formats.

Automated Password Controls – Management should establish logical access controls for all libraries or objects within libraries

Automated Library Applications – When feasible, management should implement automated library programs, which are available from equipment manufacturers and software vendors


Project Phase 3 - Realization/Build & Test – cont.

Version Controls Software Documentation

System Descriptions – System descriptions provide narrative explanations of operating environments and the interrelated input, processing, and output functions of integrated application systems

System Documentation – System documentation includes system flowcharts and models that identify the source and type of input information, processing and control actions (automated and manual), and the nature and location of output information.

System File Layouts – System file layouts describe collections of related records generated by individual processing applications

Naming Convention - critical part of program documentation

End-User Instructions – Organizations should establish end-user instructions that describe how to use an application.


Project Phase 3 - Realization/Build & Test

Build & Test

The testing phase requires organizations to complete various tests to ensure the accuracy of programmed code, the inclusion of expected functionality, and the interoperability of applications and other network components. Thorough testing is critical to ensuring systems meet organizational and end-user requirements.

Acceptance Testing – to assess the overall functionality and interoperability of an application

End-to-End Testing - to assess the interoperability of an application and other system components such as databases, hardware, software, or communication devices

Functional Testing - to assess the operability of a program against predefined requirements

Integration Testing - to assess the interfaces of integrated software components


Project Phase 3 - Realization/Build & Test – cont.

Parallel Testing - to compare the output of a new application against a similar, often the original, application

Regression Testing - to assess functionality after programmers make code changes to previously tested applications

Stress Testing - to assess the maximum limits of an application

String Testing - to assess the functionality of related code modules

System Testing - to assess the functionality of an entire system

Unit Testing - to assess the functionality of small modules of code


Phase 3 Realization/Build & Test – Lessons Learned

Project streams reporting 99% completion of tasks which, if subject to deeper analysis, does not hold water

Incomplete testing which can have a devastating post go-live impact when "too lightly" tested configurations fail and disrupt the business

Data conversion is a task which many times are under-estimated


Project Phase 4 - Pre Go-live/Deliver Phase

The implementation phase involves installing approved applications into production environments.

Primary tasks include…

announcing the implementation schedule,

training end users, and

installing the product.

Additionally, organizations should…

input and verify data,

configure and test system and security parameters

Management should circulate implementation schedules to all affected parties and should notify users of any implementation responsibilities.


Phase 4 Pre Go-live/Deliver Phase – Lessons Learned

Training is a key area where projects tend to cut corners:

Insufficient training can be disastrous for the morale of users, acceptance of the new application and company productivity which can seriously hamper the pre-go-live promises of more efficient post go-live environment.

Strong personalities, ego's, compensation structures and a mentality of "nothing will stop us from going live on x-date" can mean that pre-determined exit factors for the deliver phase such as successfully completed testing and completed cut-over activities can be compromised


Project Phase 5 - Post Go-live/ Maintenance Phase

Management should… conduct post-implementation reviews at the end of a project to validate the completion of project objectives and assess project management activities.

interview all personnel actively involved in the operational use of a product and document and address any identified problems.

analyze the effectiveness of project management activities by comparing, among other things, planned and actual costs, benefits, and development times.

document the results and present them to senior management.

The maintenance phase involves… making changes to hardware, software, and documentation to support its operational effectiveness.

making changes to improve a system’s performance, correct problems, enhance security, or address user requirements.


Phase 5 Post Go-live/Maintenance Phase – Lessons Learned

PwC was able to categorize post go-live issues in the following 35 buckets, sorted by number of incidents, highest number first:

Locked user/UID validity date required resetting

Abend related issues

Report generation

Authentication

Batch processing/upload issues


Phase 5 Post Go-live/Maintenance Phase – Lessons Learned cont.

Interface processing issues

Transaction Processing issues - mostly FI, FI-AP, SD

PO/EBP GR IR Processing issues

Access - General

SAP Mail/Inbox/Workflow Issues

Process Chain Issues

Authorization Issue

Shopping Cart PTP

Master Data issue



HR Transaction Processing Issue

Non - PROD access issue - to DEV,QA etc

ABAP Error

Miscellaneous

BW/BI/Related Reports Issues

Cannot access ESS

Missing Data/Unable to display issues



Backup Issues

Project Systems/WBS Issue

Data Entry / Update / Delete Request

Runtime Error

User error/Training Issue

Extracting/Downloading Data from SAP

SAP GUI Access Issues

Financial Period End Consolidation



File error/File copy requests

Network Issue

Foreign language/Unicode

MSS Data Display Issues

Transport request / issues

Operating System Issue

Draft

SDLC Selection Framework • IT Process Maturity

Independent Project AssuranceFebruary 2009

Draft


Realize the tangible and intangible business benefits outlined in the business case with the priority to increase customer satisfaction with billing and an enhanced ability to efficiently and effectively launch new products and services in the future.

Deliver the project on time, within budget, with agreed critical functionality for the business as quickly as possible.

Leverage standard SAP business process design and core infrastructure to reduce risk and cost. Provide a standard platform to allow for ease of integration and reporting. Deliver a compliant system that addresses key stakeholder requirements, including financial and

regulatory reporting requirements.

The company is making a significant investment to implement a single pricing, billing, invoicing, accounts receivable and cash management and collection system, utilizing SAP as the core technology. With Business Blueprint of Phase II of Project SAP complete, Executive Management would like to gain the appropriate assurance that the project achieves it’s stated objectives:

Understanding Your Objectives

Draft


Issue Possible Area of Assurance

Data Quality

• Billing data quality and accuracy

• Customer master conversion/migration

• Customer rate accuracy

• Interfacing of information to legacy systems

• Review controls around data cleansing and conversion for billing and customer master data.

• Share independent perspective on data conversion activities and provide recommendations throughout the process.

• Assess key interfaces identified and controls supporting completeness, accuracy, validity, and restricted access risks.

Customer First Focus

• Invoice Presentation Quality and Accuracy

• Shipment Rating Timeframe

• Review controls and system configurations associated with invoice generation and shipment rating and provide recommendations related to validity, completeness, accuracy, efficiency, and evidence of duplication.

• Share independent perspective on good practices associated with revenue cycle and billing/invoicing.

Financial Reporting

• Inaccurate Bad Debt Provision Calculation

• Excessive Unapplied Cash Balance

• Current system Upgrade

• Share other client experiences regarding security, internal control and risk management associated with SAP upgrade to ECC 6.0.

• Provide independent perspective on technical strategy for cash application.

• Assess process to define key financial and management reporting requirements and assess the effectiveness of the reporting designed to meet these requirements.

Issues on Your Mind

Draft


• Ongoing review of the project, control and business outcomes focusing on the stated Project SAP business objectives, risks, and priorities.

• Provide Executive Management with ongoing project assurance reporting.

• We would work along side the project identifying potential issues as early as possible and hence allowing Executive Management adequate time to consider, and if necessary address such issues. This is critical if the independent project assurance role is to add value to the project and help assist in its successful outcome. To this end we believe the independent assurance function should:

– Attend and provide input to key project team meetings– Provide a rolling progress report on issues identified

through our work– Brief key program stakeholders on the status of our work

and issues arising on a regular basis

ProjectOutcomes

ControlsOutcomes

BusinessOutcomes

ImplementationMethodology

Project Management

Project Governance

FunctionalReadiness

TechnicalReadiness

OrganizationalReadiness

BusinessCase

BenefitsRealizationPlan

Project Structure

DataQuality

Interfaces

ITGCs

BusinessProcesses

Project Assurance – A Suggested Approach

Draft


• Flexible, tailored approach to focus on management’s priorities for assurance regarding the achievement of Project SAP objectives.

– Efforts embedded in and integrated with overall Project SAP approach with a focus on value-add– “One touch” integration of effort with external audit requirements to minimize disruption to project and avoid

surprises– Evaluate and leverage work performed by others (e.g., Parent Company Internal Audit, SAP, etc.)

• “Hub and Spoke” deployment of world class functional and technical capabilities from PwC to the project:– SAP Risk Management, Security, and Control– Transportation & Logistics – Business Process– Data Assurance– Program/Project Management– Internal Control and Financial Reporting

• Distinguished history of providing independent project assurance services to the company and the parent company.– Experience navigating the Demand and Supply IT Model– Invested in relationships throughout the service center and the company.– Teams deployed alongside of the company in Houston, Scottsdale, and Plantation.

Our Value Proposition to the company

Draft


Integrating our Audit into Project SAP

Go-live & supportBlueprint

TIMETABLE

Realization

Business Process/ IT General Controls

Data Conversion/ Cleansing

Management Reporting

Security and Access Control

Testing Framework

Control Design/Gap Analysis

Agreement of expected key controls within the draft

documentation during the Blueprint and Realization

phases of the project allows maximum opportunity to

correct any issues within the design.

Data Conversion and Cleansing

Data integrity is a key risk within any environment;

this risk is increased during periods of

changes such as a system replacement.


Many key business process controls rely upon system

generated data. The requirement to manipulate this

data as part of its use adds additional risk. Effective design and implementation of system

reports maximises process efficiency and reduces the audit

risk.

Testing Framework

Our experience of large implementations has

found that the proving of the system is complex and

difficult to manage effectively. A key factor are the controls around

the remediation of issues reported during the testing

phase.

Security and Access Control

As greater use of system based controls are built into the control environment, the

reliance upon the proper allocation of access increases. Getting this right from day one both for business and support users reduces the risk that gaps are

found post live that affect our strategy.

Draft


Business Process/IT General Controls

Review proposed business process control documentation containing the following types of controls: configurable, reports, manual procedures, automated, and interfaces.

Evaluate key controls over financial reporting (selected by the company) for completeness, accuracy, validity, restricted access, efficiency, resilience, and evidence of duplication.

Review of SAP screens to confirm settings of configurable controls.

Walkthrough of business process controls to confirm existence/operation of the automated and manual controls.

Assess SAP ITGCs


Assess process to define key financial and management reporting requirements and assess the effectiveness of the reporting designed to meet these requirements.

Baseline key custom reports used to support the operation of manual controls for financial reporting (completeness, accuracy).

Testing Framework

Ensure requirements for unit testing, integration testing, system testing, UAT, interface and performance testing are adequately considered with a focus on testing of key controls.Assess whether an adequate testing monitoring system is in place.

Assess coordination of testing between business and IT.

Review configuration management and change control strategy and plan.

Review sample of testing scenarios and results focusing on consistency in approach and compliance with policy in relation to key controls.

Data Conversion/Cleansing

Review scope, approach, and requirements for data cleansing and conversion.

Assess quality controls within the conversion, setup and cleansing processes to ensure data integrity.

Review controls over the data cleansing and conversion process.

Review sample of data cleansing and conversion results.

Review strategy for master data maintenance.

Security/Access Controls

Review proposed SAP access related controls for sensitive access (SA) and Segregation of Duties (SOD) rule set; role maintenance; and user provisioning.

Assess SAP user roles against SA and SOD rule sets.

Walkthrough user provisioning and role maintenance process.

Assess existence of processes to manage access during implementation and during early stages of live operation.

Example Workplan

Draft


Questions

Contact Information

– Peter Harries, Partner 213 – 356 – 6760

– Charles Lewis, Partner 602 – 364 – 8290

– Pablo Hernandez, Senior Manager 602 – 364 – 8064

– JJ Marais, Senior Manager 602 – 364 – 8232

Documents

Pwc Systems Implementation Lessons Learned