Reining in project riskPredictive project analytics

How organizations increase the success rate of complex projects by predicting project performance

2

Contents

3 Introduction 4 The evolution of project management 5 Foreseeing the future 6 Predictive project analytics: A methodology 8 Scenarios in success 9 Moving forward 10 References

Reining in project risk Predictive project analytics 3

The management of major capital investments, such as reengineering business processes, building a new facility, or implementing new technology, is a major undertaking for any organization. Yet, despite their strategic importance, these very projects frequently experience delays, cost overruns, or fail to meet operational expectations entirely. Research shows that 63 percent of companies have experienced a recent project failure.1 The more complex the project, the greater the likelihood of failure. In fact, projects between $1 million and $3 million have a 34 percent chance of success, while those that are $10+ million have just a 7 percent likelihood of success.2 Even projects that reach completion experience challenges, with nearly half failing to meet time, budget, and/or quality goals.3

There are many reasons for project failures, ranging from poorly defined requirements to a lack of specialty resources to in-house teams with competing priorities. The organizational effects of project failure can be severe, including significant unforeseen costs, operational failures, regulatory non-compliance with potential accompanying penalties, customer dissatisfaction, and loss of competitive advantage or market share. While research studies provide insight into the factors that contribute to success and failure, they do not instruct businesses on how to predict the outcome of a particular project in total or at different phases of the project. Traditional risk assessment offers opinions, but cannot pinpoint specific control gaps that cause projects to degrade. Despite the inherent complexity of large projects, insufficient time is devoted to risk management until risk becomes an issue. During the early stages of most projects, potential risks are identified but are “filed away” and typically are not actively referenced as part of the project management.

As the project progresses and risks surface, teams scramble to build mitigation and remediation strategies and do not consistently consider the impact to either the project phase or the project as a whole. In addition, there are often project risks that are not readily visible, making it difficult to develop mitigation strategies. With global, multi-thread projects, risk management can be even more difficult, due to the sheer number of tasks, resources, deliverables, and stakeholders. Add to this conflicting objectives and priorities among the stakeholders, as well as pressure to deliver “quick and cheap,” and you have a scenario that, in all likelihood, will lead to a project with high expectations and little success — one that costs a great deal in time, resources, and capital. When a complex project loses, the business loses.

Introduction

4

Since the 1950s, project management (PM) has played an integral role in steering organizations through the intricacies of getting work done. First-generation project management approaches integrated project planning, control, and management with time management approaches. The National Aeronautics and Space Administration (NASA) was a pioneer in project management, driven by the organization’s pressing need for a framework to contain project risk and scope. In the mid-1980s, competitive pressures and a need to get products to market faster saw most manufacturers implement PM approaches and solutions for new product development. In this second generation of project management, the Work Breakdown Structure (WBS) helped project managers and systems engineers break projects into smaller, discrete work elements. The use of WBSs allowed businesses to better organize and define the work and scope of the project.

In the late 1980s, a need arose for project management in areas other than manufacturing and engineering. Global competition was heating up just as such project management organizations as the Project Management Body of Knowledge (PMBOK) and the Association for Project Management (APM) were emerging to put structure to the PM discipline. Project management processes became distinct and separate from product development processes, and PM methodologies underpinned the business management methods of the day, including business process reengineering and total quality management.

By the early 1990s, the discipline of project management had become a core competency for many organizations. PM manuals, procedures, tools, templates, and applications were widely disseminated through corporate intranets. PM practices were also used in change management, which officially arrived in 1994 with the publication of the first “State of the Change Management Industry” report in Consultants News.4

Over the past decade, third-generation, strategic project management has emerged to address the ongoing concern of value creation on the part of organizations. Inconsistency between strategy and project management and the need to address complex social, economic, and business issues has propelled project management to a new level of strategic importance and delivered a new set of challenges. In trying economic times, businesses have further sharpened their focus on reining in cost and extracting value from projects.

The evolution of project management

Reining in project risk Predictive project analytics 5

As businesses continue to experience pressure to grow revenue and improve margins in a struggling economy, many are looking for new ways to assess the health and viability of projects, programs, and business transformation initiatives. Predictive Project Analytics (PPA) is a project risk assessment methodology that provides foresight on potential risks and where immediate fixes for in-flight projects and programs (at any stage of the lifecycle) should be implemented to mitigate risk. PPA can predict the likely outcome of a project based on 28 common attributes. Using a quantitative, data-driven approach, rather than an intuitive approach, PPA predicts the likelihood of project success through predictive analysis of key project and organizational characteristics. PPA can help to identify a floundering project and avoid the costs of a late or poorly delivered project or outright failure.

PPA’s predictive analytics engine is built on the Navigator© analytics database developed by the Helmsman Institute.5 This database contains over 2,000 projects across multiple industry sectors. One of the key outcomes of the Helmsman work was the development of the Helmsman Complexity Scale, which measures the complexity of projects across multiple domains and industries in order to predict potential risk throughout the project lifecycle. PPA uses the Helmsman Complexity Scale as a key part of its analytics engine. To realize the full value of PPA, additional databases, methodologies, and benchmarking information are combined with business and analytics expertise to uncover project complexity and risk. PPA represents the next step in the evolution of project management because it is:•Research-centric: Early PPA research was carried

out over several years by the Helmsman Institute and enhanced with databases and industry expertise that are used when performing analysis.

•Project-agnostic: The PPA database includes a range of project types that have formed the basis of the attributes and complexity scale.

•Low-cost/high-value: PPA delivers value by reducing project risk and avoiding potential project failures and/or overruns.

•Focused on early value: The very early stages of a project benefit from the analysis of complexity and the level of control required for project success.

•Able to prioritize results: PPA can tell which characteristics, if improved, would be most closely correlated with an increase in the likelihood of success.

•Cost-efficient: Using predictive project analytics, organizations can realize efficiency gains by reducing or eliminating unnecessary project controls in areas not at risk.

Through predictive project analytics, organizations can identify and avoid failure by better understanding project complexity against the maturity of existing controls and governance models. As a result, financial, productivity, and reputation-based losses are minimized, while internal attributes that contribute to project success are strengthened.

Foreseeing the future

Crunchy questionsHow can you determine if your company would benefit from using predictive project analytics? The answer is simple, ask yourself the following questions and answer them honestly:1. How often do your key projects meet the

expectations of the stakeholders? Cost, timing, and performance are just three areas of expectations to consider.

2. What has been the impact on your company from projects that have failed to meet expectations or just plain failed? Impacts can be far-reaching across people, process, and technology.

3. Can you describe what your company would look like if your key projects were successfully completed? Would success attract the best talent, gain market share, or transform operations?

4. Based on the answers to the questions above, can your company afford not to use predictive project analytics? What would stop you from using it?

6

The practice of predictive project analytics requires a methodology to help ensure the proper management of risk and the highest value return from the project. The PPA methodology guides the organization through the project lifecycle. It also:•Providesanobjectiveandindependentassessmentof

program or project issues and risks to management, regulators, the board of directors, and other key stakeholders.

•Employsindustry-acceptedframeworksandstandards— such as Institute of Electrical and Electronics Engineers (IEEE), Capability Maturity Model Integration (CMMI), Control Objectives for Information and related Technology (COBIT), Projects In Controlled Environments (PRINCE2), and PMBOK.

•Protectsinvestmentbyidentifyinggapsingovernancemodels and delivery risks and by validating that the project is aligned with business requirements.

•Increasestheprobabilityofmeetingqualitygoalsandbusiness objectives.

•Identifiesprogramandprojectrisksandcontrolgapsearly.

•Providesongoingcostcontrolandmanagementforprojects that are at risk of overrun.

Research shows that project success hinges on a range of factors. Ensuring that they are addressed can help mitigate project risk, reduce the incidence of failure, and close gaps by accurately benchmarking the organization’s capabilities against similar projects.

A five-stage approachThe goal of the PPA methodology is to predict a project’s potential pitfalls and clearly identify ways to manage project risk and costs. This takes place across five stages, which progress from risk and complexity assessment to reporting. (See Figure 1.)1. Inherent risk and complexity assessment: A detailed

assessment of the project’s risk and complexity to determine the level of controls and governance required to deliver a successful project.

2. Interviews and structured document review: A series of interviews with key project team members and stakeholders, further bolstered by a review of core project artifacts, such as plans, reports, and logs.

3. Predictive analytic project review: The application of predictive analytics to key project attributes, which produces a correlation between project complexity, controls, and success. This stage relies on a database of over 2,000 projects and an effective assessment of softer factors, such as leadership and decision making, by analytics and business experts.

4. Analysis and synthesis: The aggregation of outputs from both the structured, experiential review and the application of predictive analytics, which delivers a broad, deep view of project risks. It also identifies specific control and governance improvements that can help improve project outcome.

5. Reporting: PPA findings, delivered in a format that makes sense for the project organization and management. Reports seek to provide practical recommendations and prioritized actions that help address or avert identified project risks.

Predictive project analytics: A methodology

Figure 1: Five-stage approach for managing project risk and costs

• Develop a deep understanding of the project and organization

• Interviews with core project team members

• Interviews with key stakeholders

• Detailed review of core documents, such as plans, reports, and logs

• Perform a project complexity assessment

• Analytic project review using the predictive analytic tool

• Analyze output from interviews, document review and predictive analytic tool

• Develop broad and deep view of the key unmitigated project risks

• Identify control improvements most correlated with a successful outcome

• Deliver findings of review• Identify prioritized

recommended actions

Inherent risk and complexity assessment

Interviews and structured document review

Predictive analytic project review Analysis and synthesis Reporting

1 2 3 4 5

Reining in project risk Predictive project analytics 7

Predictive analytics technologyDemonstrated quantitative methods leverage a database of completed projects to infer the required level of control to maximize the probability of your project’s success. The database contains detailed information on over 2,000 completed projects, categorized by product type, complexity, management approach, and outcomes. By combining these quantitative methods and database of empirical project data, PPA can provide an objective assessment of the inherent complexity and management characteristics of the project. By comparing current performance levels against required levels predicted, the PPA technology can help pinpoint specific gaps, hidden obstacles, and missing controls under such categories as budgeting, scheduling, risk management, and team capabilities. This allows the organization to determine the specific improvements or investments that can increase the likelihood of project success. It also identifies areas of over-investment, where less effort can be applied without affecting project results.

People + controls = maturityRecent research has identified a strong correlation between project complexity and project outcome. Not surprisingly, more complex projects fail more often. However, research has shown that complexity itself was not the factor influencing failure; having the right people and controls in place was ultimately more important. Findings indicate that a team succeeding on a low-complexity project may fail as project complexity rises. Research also revealed that better management of project risks, supported by internal audits, increases the likelihood of achieving project objectives. Simply put, more complex projects require higher degrees of project management expertise and controls in order to succeed.

Governance

Accountability

Issue Management

Role ManagementBenefits Management

Budget Management

Exec Support

Direction

DeliveryDesign Management

Planning

0% 25% 50% 75% 100%

Stakeholder ManagementResourcing

Risk Management

Contract Management

Vendor ManagementAcceptance

Expected level of controls given the complexity of the project using benchmarks in the predictive analytic tool

Actual level of controls assessed by Deloitte using the predictive analytic tool

Figure 2: Comparison of project control maturity against maturity needed for success

8

The most telling display of PPA’s potential in reducing project risk and increasing positive project outcome exists in real-world success stories. The following scenarios demonstrate the power of predictive project analytics in reducing failure in complex projects:

PPA scoring helps ensure ongoing project successA large health care organization’s multi-year, multi-phase clinical information project had experienced significant delays, using up a large portion of the overall contingency budget. Consequently, the board and executive sponsor sought to answer a number of key questions through project governance and review before moving forward with the project. These included:•Istheprojecttimelinereasonablefortheproject?•Dotheprojectstructures,controls,andreportingreflect

“leading practices”?•Isthelevelofboardandexecutivereportingappropriate?•Isthecontingencyreasonableforthistypeofproject

given its current stage?•Willtheproject,basedoncurrentexecutionstrategyand

governance, be successful?

PPA capabilities allowed for the comparison of the project’s current practices against other successful projects to answer these key questions. Analytics practitioners assessed the project’s complexity and compared the organization’s project practices against the predictive project analytics database. PPA tools quantified the degree of practices in use and helped identify areas for improvement across many aspects of the project. An independent and practices-based assessment was performed to determine the reasonableness of the project timeline.

By identifying gaps in the project structure, governance, and practices, the PPA team scored the project’s overall success and recommended actions to improve project outcome. As a result, the board and executives gained a greater awareness of project complexity and a measure of its probability for success. They now use the initial PPA score as a baseline and can accurately monitor improvements.

PPA streamlines consolidation and relocationA large technology company undertook a complex project to consolidate its operations from two locations into one. The project consisted of two phases: the first to develop a detailed plan and cost estimate; and the second to execute and complete project activities, including technology, processes, construction, and relocation. Given the complexity, scale, and cost of the project, the business sought the best way to plan for the efficient use of available resources. Executives understood the need for a robust project management plan that fully considered and outlined approaches for the management of project and business risks.

PPA experts were engaged to review various components of the project and verify that project and business risks were appropriately considered and mitigated. The team focused on ensuring compliance of project activities with key organizational policies. To ensure appropriate management of communication, risk, quality, cost, resources, scope, governance, and leadership support, the team reviewed the adequacy of project management controls. PPA capabilities allowed the team to assess project complexity and benchmark it against successful projects of similar complexity. This helped identify areas of potential risks and issues across 17 aspects of the project, including governance, risk, planning, and vendor management.

As a result of applying PPA to the project, management gained a strong understanding of the connection between project risk and key success factors and achieved a holistic view of the project and its complexity. Because PPA practitioners helped to identify risks and improvement areas, management was able to improve project practices and controls. For internal audit, PPA provided higher value than simply focusing on past activities or compliance. The organization now relies on PPA reports to establish clear risk mitigation strategies that are objective and controllable, and that offer a clear focus for project success as a final outcome.

Scenarios in success

Reining in project risk Predictive project analytics 9

For every $1,000 spent on a project, roughly 1.5 decisions are required. How can organizations make confident decisions? Predictive project analytics can help protect ROI and avoid costly overruns on complex projects by mitigating delivery risks and improving overall project performance. As a result, complex and risk-intensive projects can be delivered in a timely, effective manner. Regulatory and compliance challenges are reduced, while cost efficiencies improve with ongoing cost management and better control. By identifying project risks and lifecycle control gaps early in the process, businesses are better able to streamline the project lifecycle, meet quality goals, and achieve business objectives.

Moving forward

Benefits of predictive project analyticsProtects project investment•Projectreview–“eyesandears”formanagement•Independentassessmentofprojectplanning,

execution, and protects project investment•Independentmonitoringofemergingprojectrisks

helps control operational and reputational risks

Lowers and contains project costs•Independentassessmentofprojectbudgetadequacy•Independentassessmentofongoingbudgetneeds

and requests•Prioritizesprojectsotherightlevelofproject

management can be applied

Increases likelihood of project success•Projectionofwhatcouldgowrongbeforeitdoesgo

wrong•Adherencetoprojectplan

Gets projects back on track•Unbiased,project-agnosticexpertisealertsthe

project organization when timelines or objectives are threatened

•Interimauditrecommendationsfocusonprojectremediation

•Expertisetore-scopeandre-planasneeded

Improves the project organization and project practices •Employmentofgenerallyacceptedpracticesand

methodologies•Leverageprojectinvestmentbynotre-inventingthe

wheel

10

1 “Chaos Report,” Standish Group International, 2010.2 Ibid.3 “Making Change Work,” IBM Corporation, October 2008.4 Scott Whelehan, “Capturing a Moving Target: Change Management,” Consultants News. Retrieved from: http://www.

changemanagementresources.com/2011/10/04/the-beginning-of-change-management/5 “A Comparison of Project Complexity between Defence and Other Sectors,” The Helmsman Institute, Sidney, Australia.

www.helmsman-institute.com.

References

Reining in project risk Predictive project analytics 11

This publication contains general information only and Deloitte is not, by means of this publication, rendering accounting, business, financial, investment, legal, tax, or other professional advice or services. This publication is not a substitute for such professional advice or services, nor should it be used as a basis for any decision or action that may affect your business. Before making any decision or taking any action that may affect your business, you should consult a qualified professional advisor.

Deloitte, its affiliates, and related entities shall not be responsible for any loss sustained by any person who relies on this publication.

About DeloitteDeloittereferstooneormoreofDeloitteToucheTohmatsuLimited,aUKprivatecompanylimitedbyguarantee,anditsnetworkofmemberfirms,each of which is a legally separate and independent entity. Please see www.deloitte.com/about for a detailed description of the legal structure of DeloitteToucheTohmatsuLimitedanditsmemberfirms.Pleaseseewww.deloitte.com/us/aboutforadetaileddescriptionofthelegalstructureofDeloitteLLPanditssubsidiaries.Certainservicesmaynotbeavailabletoattestclientsundertherulesandregulationsofpublicaccounting.

Copyright©2012DeloitteDevelopmentLLC.Allrightsreserved.MemberofDeloitteToucheTohmatsuLimited

Contacts

John PeirsonPartner Deloitte&ToucheLLP+1 612 397 4714jpeirson@deloitte.com

Neil WhitePrincipalDeloitte&ToucheLLP+1 212 436 5822nwhite@deloitte.com

Vivek KatyalPrincipalDeloitte&ToucheLLP+1 612 397 4772vkatyal@deloitte.com

Scott WallaceDirectorDeloitte&ToucheLLP+1 612 659 2671scwallace@deloitte.com