CreatingCrediblePlans,IntegratedReporting,andControlSystems

Part1

GlenB.AllemanNiwotRidgeL.L.C.glen.alleman@niwotridge.com+13032419633

IPMWorkshop 2015

ThomasCoonceCostAnalysisandResearchDivisionInstituteforDefenseAnalysestcoonce@ida.org+17035756634

RickPriceProjectManagement &PlanningOperationsPrincipalLockheedMartinSpaceSystemsCompanyRick.a.price@lmco.com+13039711826

2

Learning Objectives

• TLO #1 – the student will understand how to apply the 5 core processes needed to produce a credible Performance Measurement Baseline.

• TLO #2 – the student will understand how to objectively measure progress and use the performance information to control the program.

GA

3

Building a credible PMB for Wilbur and Orville’s flying machine.

• Our talk today uses the Wright Brothers Flyer as an example for developing a Credible PMB

• The Urban Legend of Orville and Wilbur's efforts to invent manned flight has been revealed in McCullough’s book.

• Their work has all the attributes of the Five Core Processes.

GA

4

There is a Method to our Madness

• If you’re over 60 you’ll recognize this chart template for HIPO, used to describe Inputs, Processes, Outputs we’re using to create a credible Performance Measurement Baseline, measure progress, and control the program

• These are the minimum HIPO elements, there can be and should be more.

GA

5

The data and processes needed to create a Credible Performance Measurement Baseline

GA

6

A Readable Version

The Five processes needed to build a credible PMB❶ Define Done in Units of Measure Meaningful to the

Decision Makers. ❷ Develop the Plan and Schedule to reach Done as planned.❸ Define the Resources Needed to Reach Done as Planned. ❹ Adjust Plan for Reducible and Irreducible Risks.❺ Finalize the Risk Adjusted Measures of Progress Toward

Done.

GA

7

Inputs and Outputs for Building the Credible Performance Measurement Baseline

• Customer Capabilities• Time phased budget• Desired completion date• Available technologies• Time phase resources• Reference Class Data • Programmatic and technical risks with

probabilities and impacts• Historic uncertainties• Measures of Effectiveness• Measures of Performance

• Work Breakdown Structure• Integrated Master Plan• Technical Performance Management

Plan• Initial Integrated Master Schedule• Time Phased Staffing Plan• Schedule Reserve• Management Reserve• Risk Register and Mitigation Plans• Adjusted Integrated Master Schedule• Risk Management Plan

Inputs Outputs

GA

8

❶ DEFINE DONE

Define Done in units of measure meaningful to the decision makers for each deliverable in the WBS.

Measures of Effectiveness (MOE), Measures of Performance (MOP), Technical Performance Measures (TPM), and …illities are defined before the work starts, for each needed capability that accomplishes the mission or fulfills the Business Case.

TC

9

Systems Engineering Concepts Used By The Wright Brothers [1]

SE Concept Wright Brothers Activities

OperationalConcept

Vision: clear concept of the system they were developingMission Requirements: powered flight for substantial time and distanceScenarios: recognition of a key scenario involving lateral control during flight

Define the System Boundary Understood the need for the pilot to be part of the system

Objectives Definition of distance and time objectives

Derived Requirements

Horsepower and weight of the engineThrust of the propellers

Requirements Management Managed weight requirement of the engine carefully

Functional Analysis

Understood the functions of the airplane, which drove their design processFurther understood the functions of the propulsion subsystem

Physical Architecture & Interfaces

Matched the physical architecture to the functions of the airplaneInterfaces between the pilot and the control surfacesInterface between the engine and propellers TC

10

Systems Engineering Concepts Used By The Wright Brothers [1]

SE Concept Wright Brothers Activities

Prototypes and Testing

Built a successfully more complex set of gliders before building the airplaneFirst airplane was successful, due to the series of gliders and kitesEmployed sophisticated testing with each prototype, atypical of others at the time

Trade-OffDecisions Design favored control of the airplane over stability

Test System

Extensive even during glider testsAllowed problem identification and solving during all test phasesCreated wind tunnel to address discrepancy between their test data and theliteratureIncluded the careful selection of the test siteIncluded in the first airplane

Verification Best illustrated by testing of the engine to determine whether it met itsrequirements

Validation The final test that proved flight was possibleTeam Activities Two brothers formed a design-build team TC

11

Measure of Effectiveness (MoE)

Measures of Effectiveness …• Are stated in units meaningful to the buyer,• Focus on capabilities independent of any technical

implementation,• Are connected to the mission success.

Operational measures of success that are closely related to the achievements of the mission or operational objectives

evaluated in the operational environment, under a specific set of conditions.

MoE’s Belong to the End User“TechnicalMeasurement,”INCOSE–TP–2003–020–01

11 TC

12

Measure of Performance (MoP)

Measures of Performance are …• Attributes that assure the system has the capability and capacity to

perform,• Assessment of the system to assure it meets design requirements to

satisfy the MoE.

Measures that characterize physical or functional attributes relating to the system operation, measured or estimated

under specific conditions.

“TechnicalMeasurement,”INCOSE–TP–2003–020–01

MoP’s belong to the Program – Developed by the Systems Engineer, Measured By CAMs, and Analyzed by PP&C

12 TC

13

Technical Performance Measures (TPM)

Technical Performance Measures …• Assess design progress,• Define compliance to performance requirements,• Identify technical risk,• Are limited to critical thresholds,• Include projected performance.

Attributes that determine how well a system or system element is satisfying or expected to satisfy a technical

requirement or goal

“TechnicalMeasurement,”INCOSE–TP–2003–020–01

13 TC

14

What Done Looks Like for the Wright Brothers’ Military Flyer

• Propose $25,000 to build and deliver a flyer by 1st week in August 1908

• Conduct Acceptance testing for 30 days thereafter

Contract Specification

Value/Goal

Passenger # Two persons

Payload 350 lbs

Range 125 miles

Air Speed 40 mph

Duration Aloft One hour

ManeuverabilityBe steered in all directions w/o difficulty and in perfect control at all times

Ease of UseAn intelligent man become proficient within a reasonable length of time

Logistics

Assembled and disassembled within one hour and packed into an Army wagon

TC

15

❷ DEVELOP A PLAN FOR TECHNICAL SUCCESS

The Technical Plan shows the sequence of work needed to produce products and services that move the program toward Done.

The Technical Plan can take many forms. From a formal Integrated Master Plan / Integrated Master Schedule to a Kanban work plan or Scrum Release/Sprint/Task Plan in Agile Software Development

TC

16

Develop the Plan and Schedule

• Define the technical approach to building the product that will allow it to meet the performance and effectiveness goals– Identify all the variables that will impact the solution to the

technical and operational requirements– Orville and Wilbur used the spiral development approach for

the Wright Flyer

• Sequence the work to assure the increasing maturity of the deliverables supports the plan to reach Done.

TC

17

The Wright Brothers’ Technical Plan

• Modify aircraft to reduce weight from 845 to 800 pounds

• Modify aircraft and controls for two persons

• Increase HP from 21 to 31 horsepower

• Apply a Modify – Test – Modify approach

• Spend lots of time practicing (and not kill yourself)

• Dry run assembly and disassembly procedures

TC

18

❸ DEFINE THE PROGRAMMATIC PLAN

The Programmatic Plan describes all the activities that enable the Technical Plan to function properly. What work do we need to do in what order and what’s the cost of that planned work.

What facilities, resources, equipment, and materials are needed to reach done as planned.

RP

19

Define the Programmatic Plan

• Assure the programmatic plan aligns with the technical plan

• Qualified Staffing Plan to produce the needed outcomes at the needed time

• Facilities and equipment needed to produce the products

• Material availability and readiness Plan

• Phased funding available to sustain the planned progress of the Technical Plan

RP

20

Wright Brothers’ Flyer Resources Needed• Facilities and equipment for modifications

– Fabrication hangar @ Kitty Hawk– Launching track– Catapult

• Materials (propellers, wood, fabric, sewing supplies, chain, etc.)

• Test areas– Kitty Hawk at Kill Devil Hills

• Requisite labor skills– Orville & Wilbur Wright – Charlie Furnas (Dayton mechanic)– Charlie Taylor (engine design/builder)– Carpenters at Kitty Hawk (to rebuild the hangar)

• Money to support the development/build effort

• GFE– Site availability at Ft Myer, VA– Hangar shed at Ft Myer, VA

RP

21

Initial Wright Brothers’ Summary Level IMS

RP

22

❹ ADJUST FOR REDUCIBLE AND IRREDUCIBLE RISK

Any impediment reduces the probability of success. Each impediment must be identified, a plan to address the impediment developed, each handling strategy applied, monitored, and measured to assure there is an increasing probability of success.

GA

2323

Uncertainties are things we can not be certain about.

Uncertainty is created by our incomplete knowledge; not by our ignorance

GA

24

2 Types of Uncertainty – Both Create Programmatic and Technical Risk

Uncertainty

Irreducible(Aleatory)

Reducible(Epistemic)

NaturalVariability

Ambiguity

OntologicalUncertainty

ProbabilisticEvents

ProbabilisticImpacts

PeriodsofExposure

GA

25

Assemble a credible WBS and Integrated Master Plan / Integrated Master Schedule (IMP/IMS)

–WBS Dictionary says what will be built

–IMP/IMS says how, where, when, and what will be built and the processes to build it.

Identify Reducible – Event Based – uncertainties and the resulting risks from WBS Dictionary and IMP Narratives in the Risk Register

Put these risks in the Risk Register, with probability and impacts

Develop risk reduction plans and place them in the IMS using CBB funding.

How to Build a Risk Adjusted IMS in 8 Steps

0

1

2

3

RP

26

Assess the Irreducible – naturally occurring –uncertainties and the resulting schedule and cost risks in the WBS and IMS.

Use Monte Carlo Simulation to determine schedule margin and budget impacts from work activity durations to handle these irreducible uncertainties.

Formulate schedule and cost margin to protect key item deliverables in the IMS.

How to Build a Risk Adjusted IMS in 8 Steps

4

5

6

RP

27

Determine cost and schedule impacts of unmitigatedrisks in the risk register with their exposure and impact and place them in Management Reserve and Schedule MarginAssemble mitigated Irreducible (Aleatory) and Reducible (Epistemic) risks with the unmitigated event–based risks into the Total Allocated Budget– Risk handling plans included in baseline for reducible

and irreducible in the PMB.– Risk Handling in Management Reserve.

Building a Risk Adjusted IMS in 8 Steps (Concluded)

8

7

AdjusttheIMSwithrearrangingparallelorserialwork,changingscope,oraddresources.

RP

28

Wright Brothers’ Risk Register and Risk Strategy

1. Control. Buy $1.6K of key spare parts and send to Kitty Hawk

2. Accept. Add schedule margin in schedule

3. Accept. Add schedule margin in schedule

4. Accept. Add schedule margin in schedule

TC

29

Wright Brothers’ 1908 Flyer Management Reserve Calculation

Element Cost CL Comments

P80 Cost $19,696 80%

This is the cost at the 80% CL assuming only non-reducible risks in the RR

Deterministic BACf

$16,355 < 1%Final BAC based on the resource-loaded schedule

MR (P80 Cost - BACf)

3,341 Calculated MR

MR as a percent of BACf

20.4%Caculated MR % of BACf

TC

30

Wright Brothers’ 1908 Flyer Schedule Margin Calculation

ElementFinish Date CL

MonthsWork Days

IMS Deterministic Pf

Duration and Date:= 7.25 152 9/15/1908 5%

Need Duration and Date:= 7.75 163 9/30/1908 80%

Schedule Margin:= 1 11

Duration

TC

31

Schedule Margin in Final WB’s Flyer IMS

TC

32

❺ DEVELOP OBJECTIVE MEASURES OF PROGRESS

For each deliverable develop an objective measure of progress to plan using physical percent complete supported by Quantifiable Backup Data (Objective Criteria)

Use these measures to assess progress and identify variances requiring corrective actions to Keep the Program Green.

GA

33

Objective Measures of Progress to Plan

• Measures of Effectiveness (MOE)

• Measures of Performance (MOP)

• Technical Performance Measures (TPM)– Weight TPM Plan– Risk Burn Down Plan– Schedule Margin Burn Down Plan

• All the other …ilities– Availability (assemble within one hour)– Usability (simple in construction and operation)

GA

34

Weight TPM for Wright Brothers 1908 Flyer

770780790800810820830840850860870

15-Feb-08 15-Mar-08 15-Apr-08 15-May-08 15-Jun-08 15-Jul-08

Poun

ds

15-Feb-08

28-Feb-08

15-Mar-08

31-Mar-08

15-Apr-08

30-Apr-08

15-May-08

31-May-08

15-Jun-08

30-Jun-08

15-Jul-08

UpperLimit 860 855 845 835 825 815 800 800 800 800 800Planned 845 840 830 820 810 805 800 800 800 800 800Target 800 800 800 800 800 800 800 800 800 800 800

GA

35

What’s Next

• TLO #1 – the student will learn how to produce an integrated report of the Technical progress consistent with the Programmatic progress, detect variances, and suggest corrective actions.

• TLO #2 – the student will understand how to objectively measure progress and use the performance to manage the program.

• Earned Value Management is part of a Technical and Programmatic control system– Integrated risk, performance, effectiveness, technical and other

measures provide leading indicators of program performance.

36

References

1. “The Concepts of Systems Engineering as Practiced by the Wright Brothers,” Dennis Buede, Stevens Institute of Technology School of Engineering, Hoboken, NJ 07030

ExecutingtheCrediblePlan,UsinganIntegratedReporting

andControlSystemPart2

GlenB.AllemanNiwotRidgeL.L.C.glen.alleman@niwotridge.com+13032419633

IPMWorkshop 2015

ThomasCoonceCostAnalysisandResearchDivisionInstituteforDefenseAnalysestcoonce@ida.org+17035756634

RickPriceProjectManagement &PlanningOperationsPrincipalLockheedMartinSpaceSystemsCompanyRick.a.proce@lmco.com+13039711826

38

Learning Objectives

• TLO #1 – the student will learn how to produce an integrated report of the Technical progress consistent with the Programmatic progress, detect variances, and suggest corrective actions.

• TLO #2 – the student will understand how to objectively measure progress and use the performance to manage the program.

• Earned Value Management is part of a Technical and Programmatic control system– Integrated risk, performance, effectiveness, technical and

other measures provide leading indicators of program performance.

GA

39

In Part 1, we talked about …

• Addressed the five core processes needed to produce a credible Performance Measurement Baseline (PMB).

• Showed how to use objective measures of progress and how to inform program performance with these measures

• In Part 2, you’ll see how to execute the program with this information.

GA

40

Executing the Performance Measurement Baseline

GA

41

A Readable Version

The Four processes needed to execute the credible PMB❶ Compare Technical and Programmatic Actuals to Plan❷ Compare BCWP and Cost and Schedule Performance

Data with Management Reserve (MR) and Schedule Margin (SM) balances

❸ Revise Integrated Master Schedule (IMS) and Risk Register

❹ Run or Rerun Monte Carlo Simulation

GA

42

Inputs and Outputs for Building the Credible Performance Measurement Baseline

• Customer Capabilities• Time phased budget• Desired completion date• Available technologies• Time phase resources• Reference Class Data • Programmatic and technical risks with

probabilities and impacts• Historic uncertainties• Measures of Effectiveness• Measures of Performance

• Work Breakdown Structure• Integrated Master Plan• Technical Performance Management

Plan• Initial Integrated Master Schedule• Time Phased Staffing Plan• Schedule Reserve• Management Reserve• Risk Register and Mitigation Plans• Adjusted Integrated Master Schedule• Risk Management Plan

Inputs Outputs

GA

43

❶ COMPARE PLANNED TO ACTUAL PERFORMANCE

Compare technical and programmatic performance to the planned performance. Identify variances needed to make decisions about corrective actions to Keep the Program Green.

TC

44

Compare Planned to Actual Performance

• Physical Percent Complete– Planned– Actual

• Do this at the Work Package level– Quantifiable Backup Data– Avoid averaging out the variances at the Control Account

level

Objectively assess accomplishments at the Work Performance LevelEIA-748-C, page 1

TC

45

Wright Brothers’ Technical, Cost and Schedule Progress

0.80

0.85

0.90

0.95

1.00

1.05

1.10

770780790800810820830840850860870

15-Feb-08 15-Mar-08 15-Apr-08 15-May-08 15-Jun-08 15-Jul-08

CPI/S

PI

Poun

dsWBs'1908FlyerIntegratedProgress

(2/28/1908)

UpperLimit Planned Target Achieved SPI CPI

Compare side-by-side the Technical Performance (Weight) with the Programmatic performance CPI and SPI

TC

46

❷ COMPARE MR AND SM BALANCES

Using the performance data and the variances, the computed BCWP from measures of Physical Percent Complete.

Assess planned Management Reserve and Schedule Margin with the current MR and SM and inform EAC and ETC from past performance.

TC

47

Compare BCWP with MR and SM

• With actual BCWP assess impact on Estimate at Completion and Estimate To Complete.

• Assess planned Management Reserve against actual Management Reserve

• Assess planned Schedule Margin against actual Schedule Margin

• Assess planned risk burn down against actual Risk Burn Down

TC

48

Wright Brothers’ MR Planned Use vs Actuals

$0$500

$1,000$1,500$2,000$2,500$3,000$3,500$4,000

WB 1908 Flyer Planned vs Actual MR Use (15 May 1908)

MR Planned Balance MR Actual Balance

TC

49

Wright Brothers’ Schedule Margin Burn Planned Use vs Actuals

In May, when the schedule margin went below plan, immediate corrective action is needed to get back on Schedule Margin Plan

0

2

4

6

8

10

12

15-Feb-08 15-Mar-08 15-Apr-08 15-May-08 15-Jun-08 15-Jul-08 15-Aug-08 15-Sep-08

Day

s

WB 1908 Flyer Planned vs Actual Schedule Margin Use (15 May 1908)

Schedule Margin Plan Balance Schedule Margin Actual Balance

This dipcharacterized as negative slack

RP

50

❸ REVISE IMS AND RISK REGISTER

With this data, update the Integrated Master Schedule and Risk levels in the Risk Register for future periods of performance.

51

Revise IMS and Risk Register

• Every reporting period update the Risk Register with actual assessment of – Reduced risks– Actual margin needed for unmitigated and irreducible risks

• Adjust Risk Register contents for current assessments– New risks– Retired risks– Changes in risk attributes

• Update ETC, EAC, and ECD with the statused IMS and Risk Register

TC

52

❹ RUN AND RERUN THE SCHEDULE RISK ANALYSIS

Schedule Risk Analysis (per DI-MGMT-81861) at periodic intervals to confirm the Estimate to Complete and the Estimate At Completion are still within acceptable values to ensure the probability of program success.

TC

53

Run and Rerun Schedule Risk Analysis

• With updated Risk Register – rerun the Schedule Risk Analysis

• Reassess EAC, ECD, and ETC

• Determine sensitivity indices (which elements are most likely to cause the forward plan to go awry)

• If needed– Replan IMS, possibly with more schedule margin– Add resources– Adjust technical solution

TC

54

Summary

• Cost and Schedule performance informed by Technical Performance– At IBR Technical Performance plan defines the work activities

in the IMS• Build a risk adjusted PMB

– Reducible and irreducible risks in the risk register, traceable to the IMS.• Budget active mitigation activities• Protect unmitigated risks with Management Reserve and

Schedule Margin• Make sure programmatic data is aligned with Technical

Performance Data• Perform these activities continually during execution to Keep the

Program Green

TC

55

Questions???

55