Changes in Cost Estimating

I n t e g r i t y - S e r v i c e - E x c e l l e n c e

Headquarters U.S. Air Force

1

Changes in Cost Estimating

John Cargill, GS-1515AFCAA/FMAE

ASMC RPDI March 8, 2012Version 1.0.2


Climate

2

Times they are a’changing There is a strong movement towards

data driven estimates require databases increased use of CERs (cost estimating relationships) in

addition to analogies and engineering estimates All three serve as independent crosschecks to each

other Depending upon the center, one may be favored over

the others Defense Cost Analysis and Resource Center (DCARC) has

become the big data repository in the sky MilStd 881 Work Breakdown Structure Quality of data in the 1921’s and 1921-1’s have greatly

improved


Acknowledgements:Collaboration

Thanks to Kristy Golden (AFCAA/FMAE) for her letting me take on the

Missile Sufficiency Review Handbook and Database update Dr. Lew Fichter and Jeff McDowell, of Tecolote Research, for

providing extensive input for potential improvements to the previous editions of the MSSRHbk and accommodated most of my “druthers.”

John Odum (AAC/EBJ) and Bill Bolen (AAC/FMC) for their “wouldn’t it be nice if…” comments during the JASSM/JASSM-ER cost estimate. They will notice some of their wishes have been incorporated

Ron Forch (AAC/FMC) for spending three months digging through Air Force technical manuals for specifications, weights and exploded diagrams

3


Example Databases That Can Be Queried

Unmanned Spacecraft Cost Model – now up to USCM9 NASA-Air Force Cost Model – NAFCOM Civil Engineers have many cost models and databases Realtors have updated models for various neighborhoods all

around the country – independent variables: Neighborhood Bedrooms Sqft Age Number of cars in garage

4

Parametric query

In a nutshell, we want something like this for costing missiles


Fearlessly Going Where No Man Has Gone Before

OutcomesDatabase & Viewer

Multiple sources of conflicting data Structured query

capability

5

Purifying, inflating & normalizing

Hollering, yelling, swearing & going blind

Bucketing, spreading & splitting

Happy, happy, happy

Fewer sleepless nights

Relatively easy access to structured data


Background:Building the Database

History: CR-1147 July 2002 CR-1461 “Missile Systems Sufficiency Review Handbook”

January 2010 CR-1461/1 “Missile Systems Sufficiency Review Handbook”

October 2011

6

MSSRHbk is no longer a local product – it has become the standard with the U.S. Army and the U.S. Navy as well will us at AAC in the Air Force


Acknowledging the Graphics

7

• Where you see pretty pictures, I got them from my Tecolote cohorts in the MSSRHbk update project

• Where you see ugly pictures, those are mine


Most Data Came from DCARC

8


Nothing in Isolation!

9

All the Acronyms you’ll ever need: • DCARC – Defense Cost & Resource Center

(DoD)

• JCARD – Joint Cost Analysis & Research Database( NAVAIR)

• CRUAMM – Air Force Cost Risk and Uncertainty Metrics Manual (AFCAA)

• ACDB Automated Cost Database (AFCAA)


Acquisition Phases Each Development phase data point is named using the current

DoD 5000 series phase terminology of the respective period Efforts typically performed between Milestones B and C (SDD,

EMD, FSED, FSD, Engineering Development, etc.) represent costs of an EMD/SDD phase

Efforts typically performed between Milestones A and B (Dem Val, PDRR, Risk Reduction, Advanced Development, etc.) are deemed to represent the cost of a Technology Development phase

Efforts typically performed beyond Milestone C (P3I and ECP) are deemed to represent the cost of a P3I phase

Schedules referenced to CDR

10


Data Normalization & Mapping:Meticulous Pain and Agony

11

Faux data


Database The work breakdown structure was significantly expanded to

increase the granularity of hardware subsystems New data incorporated

Air vehicle weight Length Diameter (U.S. Army has found high correlation

between diameter and cost) Fly-out Range Speed

“Box-level” “SWAP” and cost Data to support analogies, CERs, and output based simulation Schedule database

12


Databasecontinued

Continued alignment with typical Air Force estimating structures Completely mapable to MilStd 881 Meets the requirements for electronic transfer to the Navy’s Joint

Cost Analysis Research and Database (JCARD) Viewer supports many sort, search, conversion and other

functions Automatically creates an extensive series of statistical

summaries Supports easy benchmarking by programs Creates schedule graphs referenced to CDR for programs

selected Exports charts to MS Word

13


The Magic Tool

14

1. Contains most all the goodness known to man

2. Better than sliced bread


Program Selection

Programs to be included in the data set for analysis can be selected on the basis of Weight, length, diameter, range, speed Platform

Air Surface

Mission Air Surface Missile Defense

15

Allows asking parametric questions like: What would a 1000lb class subsonic air to ground cruise missile that can go 300 miles cost?


Program SelectionBy Type of Guidance

Guidance Multi-mode TERCOM INS Active Radar Semi-active Laser INS/GPS IR EO Passive Radar

16

Laser EO Passive Radar Laser IIR Pre-programmed Semi-active Radar Command


Program SelectionBy Type of Warhead

17

Warhead Blast/Frag Nuclear HE Bomblets Non-nuclear kill vehicle Modular General Purpose Bombs Direct Frag Penetration HE Blast HETSC HE Blast/Frag Penetrator

HE Tandom Shaped Charge Varies Penetrating Blast Frag Penetrator HESC Shaped Charge Explosively Formed

Penetrator Multi-effects HEBF


Program SelectionBy Propulsion, Lead Service, Control

18

Propulsion Engine/Solid Engine Solid None

Control TVC Aero Surfaces None Gas Reaction Control TVC and Aero Surfaces Fluid-injection Jet

Vanes

Lead Service Navy Air Force Army

Selecting: 1. No propulsion 2. Air Force yields3. Aero surfacesGBU-15, JDAM, SDB I and SDB II


Program SelectionFrom Program List

19

Fair warning: You MUST see a program in the “Systems to Include” list BEFORE you hit “Save Picks…” or no program will be selected.


Program SelectionFrom Pre-determined Program Lists

20


Of Course You Can Choose Any Base Year You Want

21

Caveat: Currently, only DoD Inflation is automatically supported for 3600 and 3020


Example:Air-To-Surface Weapons

22


Air-To-Surface SelectionRDTE Burn Rate

23

A good bench-marking chart Compare program of interest against other

programs in the database Compare program of interest against

programs selected If your program is an outlier on burn rate,

maybe you have ramped up too slowly, two quickly or have an extraordinary schedule

Compare by RDT&E Phase

Synthesized Data


Air-To-Surface Selection:RDTE Nonrecurring Factor of Recurring Cost

24

Synthesized Data


Air-To-Surface Selection:RDTE Factor of System PMP Cost

25

Toggled for RDTE Factor of System PMP CostCan toggle for RDTE Factor of Air Vehicle PMP Cost

Synthesized Data


Air-To-Ground Selection:Production Non Recurring Factor of Recurring Cost

26

Synthesized Data


Air-To-Ground Selection:Production Non Recurring Factor of Recurring Cost

27

Toggled for Production Factor of Air Vehicle PMP CostCan toggle for Production Factor of System PMP Cost

Synthesized Data


Air-To-Surface Selection:Production Unit cost (T1-UC1000)

28

Since T1 is often greatly affected by minor changes in CIC, options are provided for Unit Costs at 100, 500, 1000 or User Entered


Air-To-Surface Selection:Production Unit cost (T1)

29

Toggled for Production Unit Cost (T1)Can toggle for Production Unit Cost Several Other Ways

Synthesized Data


Air-To-Surface Selection:Production Unit Cost per Pound (T1-UC1000)

30

Toggled for Production Unit Cost (T1)Can toggle for Production Unit Cost Several Other Ways

Synthesized Data


Air-To-Surface Selection:RDTE Nonrecurring Cost per Pound (T1-UC1000)

31

Toggled for RDTE Nonrecurring Cost per Pound (T1)Can toggle for RDTE Nonrecurring Cost per Pound Several Other

Ways

Synthesized Data


Air-To-Surface Selection:RDTE Nonrecurring Cost per T1-T1000

32

Toggled for RDTE Nonrecurring Cost per T1Can toggle for RDTE Nonrecurring Cost Several Other Ways

Synthesized Data


Air-To-Surface Selection:Crosschecks

33

Toggled for Default CIC Options and Default RDTE Phase Options

Synthesized Data


Air-To-Surface Selection:Crosschecks continued

34


Synthesized Data


Air-To-Surface Selection:Crosschecks continued

35


Synthesized Data


Air-To-Surface Selection:Production Phase Cost Improvement Curves

36

Cost Improvement Curves “Learning Curve” -- database uses unit theory “Rate Surface” -- combination of learning and rate which is

very important with lot buys of varying sizes

Synthesized Data

Note for Regression Experts: Coefficient of Variation is greater for rate slope than learning – implies that economic order quantities are not being followed historically, or possibly there is multicollinearity between the rate and cumulative quantity.


Air-To-Surface Selection:Production Phase Proportions

37

Synthesized Data

Yes, Virginia! There are Non-Recurring costs during production, especially for the first lot


Schedule Analysis

38

This iteration of the MSSRHbk with its Database Viewer is First version to reference program events to how much

before or how much after CDR CDR has the highest correlation with other events Milestones and PDR move around a lot New DoD guidance is that PDR will occur before MS B,

whereas in the past it usually occurred after MS B Program picking (selection) is separate from the programs

picked for costs and the other metrics because it may be necessary to eliminate some programs for cost variances resulting from funding vagaries, etc., yet the schedule might be appropriate to retain

It will not be explained why this button is called “Bonus Schedule Analysis”


Schedule AnalysisAir-To-Surface

39

Same Air-To-Ground program picks as in the cost examples but with a slight twist: You get to select whether you want to work with Median, Mean or Lognormal Mean schedule durations


Schedule

40

Synthesized Data

PDR

MS A

MS B


Schedule:

41

Synthesized Data

CDR

CDR is a real point in time independent of acquisition and milestone strategy – it is a time when most of the engineering drawings are complete and we are ready to start “bending tin”


Schedule:CDR Reference

42

Synthesized Data


Schedule Analysis:MS B Referenced

43

For planning, comparison and cost estimating purposes it is still important to be able to estimate times from MS B, even though the requirements to enter MS B have changed Be careful how you use the MS B reference as it is more

applicable for reviewing historic programs where PDR and CDR occurred after MS B

For modern programs where risk reduction efforts result in System-level PDR and Component-level CDR, you should use the CDR-referenced times

MS B referenced time lines can serve as a rough estimate of the ceiling, as one would expect modern programs to take less time post MS B because more is accomplished before MS B

It will not be explained why this button is called “Bonus Schedule Analysis”


Schedule:MS B Reference

44

Synthesized Data


Exporting Summary Statistics

45

One of the new features of the viewer is the ability to export the entire set of summary statistics charts to MS Word, which may be subsequently pasted to MS Power Point briefing charts

Cell B2 of the Analysis tab has a button that causes exporting the summary charts to MS Word

One feature of this export is that tables are produced for the entire database and for the programs selected


One Last Feature:Pages of Systems Descriptions

46


One More Last Feature:Lots of Exploded Drawings

47


Conclusions

48

User friendly database and viewer More time analyzing and thinking Less time grunting

With viewer, it is convenient to make comparisons and talk to Program Office

Ideally, database and viewer would be Oracle or Access based, however, Excel works reasonably well in the interim


Take Home Message

49

Early adoption of this database technology has brought us closer to our information sources and is

allowing us to work smarter, not harderEarly Adoption of

TechnologyCloser to data

sourcesWorking smarter,

not harder


Questions and Point of Contact

John Cargill GS-1515Air Force Cost Analysis AgencyOperating Location Eglin AFB, FL850-883-3460(w)703-371-0891(c)[email protected]

50

I n t e g r i t y - S e r v i c e - E x c e l l e n c e51

The End

I n t e g r i t y - S e r v i c e - E x c e l l e n c e 52

BACKUP SLIDES

Documents

Changes in Cost Estimating