Quantifying and Forecasting Uncertainty in Innovation Project Management - Dr. Jose A. Briones

QUANTIFYING UNCERTAINTY IN INNOVATION PROJECT

MANAGEMENT

Jose A. Briones, Ph.D.

SpyroTek Performance Solutions

Twitter: @Brioneja

http://www.palisade.com/2010Conf/Vegas/

Agenda

Problem Definition New Framework for Innovation

Management Discovery Driven Planning Reverse Income Statement The “Flaw of Averages” Probabilistic Decision Analysis Case Study: Forecast of iPad Sales

www.Brioneja.com Twitter: @Brioneja

Introduction Product innovation has been described as the

way out of today’s difficult business environment.

The rate of success of development projects, in particular disruptive innovation projects remains too low.

We believe that a reason for the low success rate is the erroneous application of analysis methods designed for incremental innovation like NPV and DCF to projects with high levels of uncertainty


Background

In this Chapter of the “Beyond Stage Gate” series we will discuss the use of Discovery Driven Planning and Probabilistic Decision Analysis in the management of innovation projects with high levels of uncertainty.

Probabilistic Decision Analysis, when combined with the right management processes like Discovery Driven Planning is a very effective approach to evaluate and manage the risk and potential of innovation projects


• “We keep rediscovering that the root reason for established companies’ failure to innovate is that managers don’t have good tools to help them understand markets, build brands, find customers, select employees, organize teams, and develop strategy”

• “Some of the tools typically used for financial analysis, and decision making about investments, distort the value, importance, and likelihood of success of investments in innovation”

• “There’s a better way for management teams to grow their companies. But they will need the courage to challenge some of the paradigms of financial analysis and the willingness to develop alternative methodologies”

Clayton M. Christensen


Classical Stage-Gate Process

Project is managed in a linear fashion

“The Stage-Gate system assumes that the proposed strategy is the right strategy, the problem is that except in the case of incremental innovations, the right strategy cannot be completely known in advance” – Clayton M. Christensen

Source: Product Development Institute, Inc.


Challenge

Create a unified framework for innovation projects/product development that 1. Provides the flexibility needed for

innovation to work

2. Still has the metrics needed for proper measurement of progress and resource allocation.


A New Innovation Project Categorization

Tech

nica

l Unc

erta

inty High

Medium

Low

Market UncertaintyLow Medium High

Source: Product Development Management Association

This categorization is based on the most critical variable for new innovation projects: Degree of uncertainty


Resources

Resources

TimeTime

Level

1

2

3

VOC

Idea Generation

Technology Assessment

PrototypeDevelopment

Customer Testing

BusinessCaseValue in Use

Analysis RegulatoryIP Strategy

Supply ChainAnalysis

Roadmap/Timeline

Risk Analysis

The Spiro-Level™ 3-D Approach to InnovationLaunch

Quadrant I

Quadrant IIQuadrant III

Quadrant IV


Overall Project Management

Level 1

• Discovery Driven Planning• Knowledge to Assumption Management

Level 2

• Product/Technology Roadmaps• Multigenerational Product Planning

Level 3

• Linear Stage-Gate• Agile/Lean Development• Rapid Prototyping


Financial Analysis

Level 1

• Reverse Income Statement• Real Options

Level 2

• Probabilistic Decision Analysis

Level 3

• NPV• DCF


Discovery Driven Planning (DDP)

Process for management of innovation and development projects with high levels of uncertainty such as disruptive innovations, new product/new market projects or game changers

1995: First Publication by Rita Gunther McGrath and Ian C. MacMillan (1)

2000: Popularity increases with publication of book “The Entrepreneurial Mindset” (2)

2008: Endorsed by Clayton M. Christensen as an alternative to Stage Gate for high uncertainty projects (3)

2009: Publication of book Discovery Driven Growth (4)


http://hbr.org/search/Rita+Gunther+McGrath/0/author

http://hbr.org/search/Ian+C.+MacMillan/0/author

DescriptionDiscovery Driven Planning turns the basic project

map on its head

Discovery Driven Planning is a plan to learn, not to show that you had all the answers when you wrote the plan

“Learn as you go”

DDP uses four basic documents ‘Reverse Income Statement’ models the business

economics. ‘Pro-Forma Operations Specs’ defines operations needed

to run the business. ‘Key Assumptions Checklist’ identifies the business

hurdles and assumptions for the initiative. ‘Milestone Planning Chart’ specifies when the assumption

needs to be tested.www.Brioneja.com Twitter: @Brioneja

Reverse Income Statement

Instead of estimating the market demand and calculating the potential revenue/profit, DDP starts by defining the desired financial target that will meet the company’s targets and strategy

The method then defines the set of assumptions, technical and commercial that MUST be proven true in order for the project to succeed


Generic Reverse Income Statement


Reverse Income StatementRequired Profits (MM$/yr) $20

Return On Sales 20%Target Revenue (MM$/yr) $100Market Share Penetration 30%

Market Size (MM$/yr) $333Unit Price, $ $500

Unit Sales M/yr 200Production Capacity Per Line, Units/Day 100

Production Lines Needed 6Capital Cost of a Production Line, MM$ $1Total Capital Investment Needed, MM$ $6


The Flaw of Averages

The Flaw of Averages states that plans based on the assumption that average conditions will occur are usually wrong

Can you drown in a river that is on average only three

feet deep?

Source: “The Flaw of Averages” Marketingnpv.com


Demand Forecast Problem

Forecast for annual demand: “Between 150,000 and 250,000 units”Average 200,000 units.Plant is sized for average demand estimateProfit is also calculated using average demand

Flaw of averages: Profit WILL be lower than forecastedNo upside to exceeding demand estimate due to plant

capacity limitationsAverage becomes upper limit


Solution: Ranges and Probabilities

Input data and results are expressed as ranges and probabilities80% Chance of number being between 570 and

689K

Source: “The Flaw of Averages” Marketingnpv.com


Probabilistic Decision Analysis Model uses @Risk probabilistic decision analysis

software Monte Carlo simulation Risk and opportunity analysis

Designed for complex projects with high levels of uncertainty Inputs contain high number of variables, either technical or

financial with a high degree of uncertainty, assumptions and dependencies ○ New product development assessment○ Capital spending decisions○ Value chain analysis○ Production and sales forecasting analysis

Eliminates use of “one at a time” cases Analyzes thousands of cases simultaneously Generates a range of outcomes Outcome charts are analyzed to make decisions on direction


Input Ranges Input values are entered in range format –

Width and shape of range are critical inputs Definition of the input ranges is the most critical

step Do not start with the typical value, start with the

range, define the shape of the function (10%, 50%, 90% probability).

There are multiple choices for the shape of the input range:Triangular: Most common for initial

assumptions Normal distribution: Used when more

accurate input data is availablePERT: When data is in form of probabilitiesGamma distribution: Good to model

pricing distributions in B-B cases

5.0% 90.0% 5.0%0.4342 0.6448

0.30

0. 35

0. 40

0. 45

0. 50

0. 55

0. 60

0. 65

0. 70

0. 75

Reverse Income Statement Combined with Probabilistic Decision Analysis


Reverse Income Statement Min Target MaxRequired Profits (MM$/yr) $15 $20 $25

Return On Sales 15% 20% 25%Market Share Penetration 25% 30% 35%

Unit Price, $ $400 $500 $600Production Capacity Per Line, Units/Day 75 100 125Capital Cost of a Production Line, MM$ $0.8 $1.0 $1.4

Define ranges for each input variable

Revenue vs. Market Size

Revenue range is projected w level of probability


8.0% 81.1% 10.8%

$81.00 $120.00

$60

$70

$80

$90

$100

$110

$120

$130

$140

$150

$160

0.000

0.005

0.010

0.015

0.020

0.025

0.030

Target Revenue (MM$/yr)

Target Revenue (MM$/yr)

Minimum $64.17Maximum $151.41Mean $101.06Std Dev $14.69Values 5000

5.0% 90.0% 5.0%

$254.36 $436.48

$150

$200

$250

$300

$350

$400

$450

$500

$550

$600

0.000

0.001

0.002

0.003

0.004

0.005

0.006

0.007

0.008

Market Size (MM$/yr)

Market Size (MM$/yr)


Unit Sales vs. Capital


10.5% 79.6% 9.9%

162.5 250.0

100

150

200

250

300

350

0.000

0.002

0.004

0.006

0.008

0.010

0.012

0.014

Unit Sales M/yr

Unit Sales M/yr

Minimum 114.11Maximum 338.25Mean 203.46Std Dev 34.00Values 5000

6.2% 83.8% 9.9%

$4.20 $8.00

$2 $4 $6 $8 $10

$12

$14

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

Total Capital Investment Needed, MM$

Tidal Capital Investment Needed, MM$


Capital investment is expressed a range vs. unit sales

Capital Investment Sensitivity Analysis


0.50

-0.45

-0.45

0.44

-0.35-0

.5

-0.4

-0.3

-0.2

-0.1 0.0

0.1

0.2

0.3

0.4

0.5

0.6

Coefficient Value

Unit Price, $

Required Profits (MM$/yr)

Return On Sales

Production Capacity Per Line, Units/Day

Capital Cost of a Production Line, MM$

Total Capital Investment Needed, MM$Regression Coefficients

Sensitivity analysis is a way to focus on the variables that are most important to reduce uncertainty


iPad 2010 Unit Sales (MM) 7.1iPad 2011 Unit Sales (MM) 14.4iPad 2012 Unit Sales (MM) 20.1

Netbook 2010 Unit Sales (MM) 43

Unit Price, $ $650iPad Cost of Materials, $/unit $264.3iPad Cost of Manufacturing, $/unit $10.5iPad Warranty Service, $/unit $20

iPad Indirect Expenses 2010 (MM$/yr) $2,100iPad Indirect Expenses 2011 (MM$/yr) $2,500iPad Indirect Expenses 2012 (MM$/yr) $2,700

Case Study: iPad™ Income Projection A) Static Analysis

Static analysis uses projections prior to launch

iPad Income Projection: Static Analysis


B) Probabilistic Decision Analysis – iPad Income Projection


Min Target MaxiPad 2010 Unit Sales (MM) 1.3 7.1 12.9iPad 2011 Unit Sales (MM) 8.6 14.4 55iPad 2012 Unit Sales (MM) 14.3 20.1 63.3

Netbook 2010 Unit Sales (MM) 36.6 43 58

Unit Price, $ $500 $650 $830iPad Cost of Materials, $/unit $219.0 $264.3 $335iPad Cost of Manufacturing, $/unit $9.0 $10.5 $12iPad Warranty Service, $/unit $15 $20 $25

iPad Indirect Expenses 2010 (MM$/yr) $1,680 $2,100 $2,520iPad Indirect Expenses 2011 (MM$/yr) $2,000 $2,500 $3,000iPad Indirect Expenses 2012 (MM$/yr) $2,160 $2,700 $3,240

iPad Market Share Projections


13.1% 80.8% 6.1%

9.25% 24.25%

0%

5%

10%

15%

20%

25%

30%

35%

0

1

2

3

4

5

6

7

8

iPad Market Share vs. netbooks 2010


Minimum 2.83%Maximum 32.2%Mean 15.6%Std Dev 5.46%Values 5000

13.7% 67.5% 18.8%

32.0% 80.0%

0%

20%

40%

60%

80%

100

%

120

%

140

%

160

%

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8

2.0



Minimum 17.3%Maximum 143%Mean 57.3%Std Dev 23.7%Values 5000

8.3% 68.1% 23.6%

41.0% 90.0%

20%

40%

60%

80%

100

%

120

%

140

%

160

%

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8



Minimum 27.5%Maximum 159%Mean 71.7%Std Dev 25.3%Values 5000

Market share analysis projects potential level of penetration and substitution

iPad Revenue Projections


8.6% 75.2% 16.2%

$2,400 $6,350

$0

$1, 0

00

$2, 0

00

$3, 0

00

$4, 0

00

$5, 0

00

$6, 0

00

$7, 0

00

$8, 0

00

$9, 0

00

$10, 0

000.0

0.5

1.0

1.5

2.0

2.5

3.0

Valu

es x

10

-̂4

iPad Revenue 2010 (MM$/yr)


Minimum $880.13Maximum $9,811.24Mean $4,685.46Std Dev $1,637.15Values 5000

6.6% 83.9% 9.5%

$8,250 $27,500

$5,0

00

$10, 0

00

$15, 0

00

$20, 0

00

$25, 0

00

$30, 0

00

$35, 0

00

$40, 0

00

$45, 0

00

0

1

2

3

4

5

6

7

Valu

es x

10

-̂5



Minimum $5,144.81Maximum $41,875.43Mean $17,165.42Std Dev $7,070.77Values 5000

5.0% 90.0% 5.0%

$11,605 $35,590

$5,0

00

$10, 0

00

$15, 0

00

$20, 0

00

$25, 0

00

$30, 0

0 0

$35, 0

0 0

$40, 0

0 0

$45, 0

0 0

$50, 0

0 0

0

1

2

3

4

5

6

Valu

es x

10

-̂5



Minimum $8,217.53Maximum $46,917.00Mean $21,472.35Std Dev $7,522.30Values 5000

Upper range of revenue shows dramatic upside potential

Max revenue value is from unit sales analysts estimates 7 months after launch

iPad Gross Profit Projections


7.4% 87.1% 5.5%

$252.50 $475.00

$150

$200

$250

$300

$350

$400

$450

$500

$550

$600

0.000

0.001

0.002

0.003

0.004

0.005

0.006

0.007

iPad Gross Profit, $/unit

iPad Gross Profit, $/unit


10.0% 85.0% 5.0%

45.25% 62.82%30

%

35%

40%

45%

50%

55%

60%

65%

70%

0

1

2

3

4

5

6

7

iPad Gross Profit Margin

iPad Gross Profit Margin

Minimum 31.6%Maximum 68.8%Mean 53.6%Std Dev 6.12%Values 5000

Gross profit analysis can highlight areas of concern

iPad Income Projections


38.7% 54.1% 7.2%

$50 $2,000

-$2,0

00

-$1, 0

00 $0

$1, 0

00

$2, 0

00

$3, 0

00

$4, 0

00

$5, 0

00

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

Valu

es x

10

-̂4

iPad Income 2010 (MM$/yr)


Minimum -$1,869.82Maximum $4,703.02Mean $431.87Std Dev $1,010.77Values 5000

12.0% 77.4% 10.6%

$2,250 $12,500

-$5,0

00 $0

$5, 0

00

$10, 0

00

$15, 0

00

$20, 0

00

$25, 0

00

$30, 0

00

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Valu

es x

10

-̂4



Minimum -$830.65Maximum $26,029.39Mean $6,784.18Std Dev $4,206.06Values 5000

20.5% 64.4% 15.1%

$5,000 $13,750

$0

$5, 0

00

$10, 0

00

$15, 0

0 0

$20, 0

0 0

$25, 0

0 0

$30, 0

0 0

0.0

0.2

0.4

0.6

0.8

1.0

1.2

Valu

es x

10

-̂4




Income analysis shows high potential for losses 1st year, but huge profit upside for 3rd year

iPad 2012 Income Sensitivity Analysis


20.5% 68.3% 11.2%

$5,000 $15,000

$0

$5, 0

00

$10, 0

00

$15, 0

00

$20, 0

00

$25, 0

00

$30, 0

00

0.0

0.2

0.4

0.6

0.8

1.0




0.86

0.48

-0.17

-0.05

-0.01

-0.2

0. 0 0. 2

0. 4

0. 6

0. 8

1. 0

Coefficient Value

iPad Warranty Service, $/unit

iPad Indirect Expenses 2012 (MM$/yr)

iPad Cost of Materials, $/unit

Unit Price, $

iPad 2012 Unit Sales (MM)

iPad Income 2012 (MM$/yr)Regression Coefficients

Sensitivity analysis shows where more market research is needed to reduce uncertainty

Actual iPad Sales Figures 2010 – 14.8 MM Units

Original forecast 7.1 MM Units

2011 Jan-Mar - 4.7 MM UnitsApr-June - 9.3 MM UnitsJul-Sep – 11.1 MM Units

○ Apple sold more iPads than Dell sold in all its PCs together (10.6 million)

2011 Projection - ~ 40 MM UnitsOriginal forecast 14.1 MM Units


Use of DDP

DDP has been successfully used by oil and pharmaceutical companies to tackle high uncertainty projects

More companies are adapting the method, but there is still strong resistance to leave Stage Gate behind.Stage Gate can be used after the results of

DDP have defined the project and reduced the level of uncertainty


Summary

DDP combined with Probabilistic Decision Analysis provides discipline and framework to the discovery process but at the same time provides the flexibility necessary for the project to change direction or iterate as needed


Contact Information

[email protected] www.Brioneja.com Twitter: @Brioneja


mailto:[email protected]

http://www.brioneja.com/

References

1. Discovery Driven Planning, 1995, http://hbr.org/1995/07/discovery-driven-planning/ar/1

2. The Entrepreneurial Mindset, 2000, http://www.amazon.com/Entrepreneurial-Mindset-Continuously-Opportunity-Uncertainty/dp/0875848346/ref=sr_1_1?ie=UTF8&s=books&qid=1274888243&sr=1-1

3. Innovation Killers, 2008, http://hbr.org/product/innovation-killers-how-financial-tools-destroy-you/an/R0801F-PDF-ENG?Ntt=innoation%2520killers

4. Discovery-Driven Growth, 2009, http://www.amazon.com/Discovery-Driven-Growth-Breakthrough-Process-Opportunity/dp/1591396859/ref=sr_1_3?ie=UTF8&s=books&qid=1274887263&sr=8-3

5. Discovery Driven Planning http://faculty.msb.edu/homak/homahelpsite/webhelp/Discovery_Driven_Planning.htm


http://hbr.org/1995/07/discovery-driven-planning/ar/1

http://hbr.org/product/innovation-killers-how-financial-tools-destroy-you/an/R0801F-PDF-ENG?Ntt=innoation%20killers

http://hbr.org/product/innovation-killers-how-financial-tools-destroy-you/an/R0801F-PDF-ENG?Ntt=innoation%20killers

http://www.amazon.com/Discovery-Driven-Growth-Breakthrough-Process-Opportunity/dp/1591396859/ref=sr_1_3?ie=UTF8&s=books&qid=1274887263&sr=8-3

http://www.amazon.com/Discovery-Driven-Growth-Breakthrough-Process-Opportunity/dp/1591396859/ref=sr_1_3?ie=UTF8&s=books&qid=1274887263&sr=8-3

Backup Data


iPad Market Share vs. netbooks 2010 16.5%iPad Market Share vs. netbooks 2011 33.5%iPad Market Share vs. netbooks 2012 46.7%

iPad Revenue 2010 (MM$/yr) $4,615iPad Revenue 2011 (MM$/yr) $9,360iPad Revenue 2012 (MM$/yr) $13,065

iPad Total Cost, $/unit $294.77iPad Gross Profit, $/unit $355.23iPad Gross Profit Margin 54.7%

iPad Direct Expenses 2010 (MM$/yr) $2,093iPad Direct Expenses 2011 (MM$/yr) $4,245iPad Direct Expenses 2012 (MM$/yr) $5,925

iPad Total Gross Profit 2010 (MM$/yr) $2,522iPad Total Gross Profit 2011 (MM$/yr) $5,115iPad Total Gross Profit 2012 (MM$/yr) $7,140

iPad Income 2010 (MM$/yr) $422iPad Income 2011 (MM$/yr) $2,615iPad Income 2012 (MM$/yr) $4,440

Documents

Quantifying and Forecasting Uncertainty in Innovation Project Management - Dr. Jose A. Briones