An employee-owned company Extensions of Auto-Generated Code and NOSTROMO Methodologies (U) Pam McDonaldSandra GilesDan Strickland THAAD Project OfficeTHAAD

an employee-owned company

Extensions of Auto-Generated Code and NOSTROMO Methodologies (U)

Pam McDonald Sandra Giles Dan StricklandTHAAD Project Office THAAD Project Office Dynetics, INC

DISTRIBUTION A: Approved for public release; distribution unlimited.


Rebus (U)

rebus - a mode of expressing words and phrases by pictures of objects whose names resemble those words, or the syllables of which they are composed; enigmatical representation of words by figures; hence, a peculiar form of riddle made up of such representations.

- a puzzle where you decode a message consisting of pictures representing syllables and words


Rebus 1 (U)

Cocoa Moe Two

COCOMO II


Rebus 2 (U)

Donald Reef

Donald J. Reifer

Jay


Rebus 3 (U)

Barry BoehmStrawberry Straw

BoeheimHi


Overview (U)

• Background

• AGC Extensions

• AGC Worksheet Demo

• NOSTROMO Extensions

• NOSTROMO Demo

• Future Enhancements


Background (U)

• In 2002, TPO and Dynetics presented a methodology on the estimated effort of Auto-Generated Code (AGC)

• In 2003, TPO and Dynetics presented a methodology on using Monte Carlo simulation and uncertainty in COCOMO II to estimate risk in estimates

• Both methodologies have been extended and built upon to cover more situations

• Methodologies combined into a single tool suite to offer expanded solutions to estimation needs


AGC Extensions


AGC Terminology (U)

• generating – related to the development, input effort used to auto-generate code

• resultant – related to the code that is output by the auto-generation• synthetic – the normalized estimate of effort that takes into account the

effort in non-testing activities (generating language) and testing activities (resultant language)


DD22.5%

PD15.9%

RQ6.5%

IT26.2%

CUT29.0%

Testing vs. Non-Testing Efforts – COCOMO II Phases (U)

Requirements

Prelim. Design

Detailed Design

Code and Unit Testing

Integration Testing

COCOMO II Effort Distribution – Large Project

Non-Testing

Testing

Waterfall Model

• Requirements, Preliminary Design and Detailed Design are non-test related; Integration Testing is test related

• Code and Unit Testing contains activities in both areas


Testing vs. Non-Testing Efforts - Code and Unit Testing Phase (U)

Code and Unit Testing

Requirements

Design

Programming

Test Plans

V & V

Project Office

CM/QA

Manuals

4%

8%

56.5%

5.5%

8.5%

6%

6.5%

5%

• Code and Unit Test broken up into default activities

• Requirements, Design, and Programming are non-test activities

• Test Plans and V & V are test activities

• Project Office, CM/QA, and Manuals are non-specified - split evenly between both efforts

77.25% Non-Test

22.75% Test

Non-Testing

Testing


Testing vs. Non-Testing Effort - Synthetic SLOC (U)

IT26.2%

DD22.4%

PD15.9%

RQ6.5%

CUT (test)6.6%

CUT (non)22.4%

COCOMO II Effort Distribution

67.2 % Non-Test

32.8% Test

• Normalized effort values for non-test and test efforts

• 67.2% of development size counted in synthetic size

• 32.8% of translated size counted in synthetic size

• Example: 10KSLOC translated to 50KSLOC

– 10,000 (0.672) + 50,000 (0.328) = 23,120 SLOC

• Synthetic SLOC Rule of Thumb: 66.7% non-test, 33.3% test

Non-Testing

Testing


Adapted AGC (U)

NON-AGC CODE

GENERATING

RESULTANT

% Redesign

% Recode

% Retest

AGC CODE

% Redesign

% Recode

% Retest

ADAPTED NON-AGC

ADAPTED AGC

New Non-AGC + Synthetic SLOC + Adapted Code = ESLOC


Sample Calculation (U)

• Adapted Code of 10000 SLOC5000 SLOC Non-AGC Ada955000 SLOC in Ada95 auto-generated by 250 SLOC in a 4GL

• Modification PercentagesRe-design: 5% Re-code: 10% Re-test: 25%

• Non-AGC Adapted: = 5000 * ((0.4)*(0.05) + (0.3)*(0.1) + (0.3)*(0.25)) = 5000 * (0.125) = 625 ASLOC

• AGC Adapted:= 250 * ((0.4)*(0.05) + (0.3)*(0.1)) + 5000 * ((0.3)*(0.25))= 250 * (0.05) + 5000 * (0.075)= 388 ASLOC

• Adapted Synthetic SLOC:= 625 + 388 = 1013 Adapted Synthetic SLOC

• ESLOC Calculation:= (1013 Adapted Synthetic SLOC) + (10000 New Non-AGC SLOC) + (2300 New

Synthetic SLOC)= 13313 ESLOC


AGC Worksheet (U)

Software Item Name Reuse SLOC (Non-AGC) Reuse SLOC (Generating) Reuse SLOC (Resultant) %DM %CM %RT %Mod Adapted Non Adapted Gen Adapted Res Adapted SLOC

Sample 1 0 1500 7500 10.0% 10.0% 20.0% 13.0% 0 105 450 555Sample 2 12000 0 0 5.0% 5.0% 25.0% 11.0% 1320 0 0 1320Sample 3 10000 1000 52000 10.0% 20.0% 35.0% 20.5% 2050 100 5460 7610

• Microsoft Excel Workbook with two main worksheets:• Synthetic SLOC Worksheet – calculates Synthetic SLOC estimate• New Size Worksheet – calculates ESLOC from Synthetic SLOC,

New Non-AGC SLOC, and Adapted SLOC (above)• Cells with yellow background are calculated• Cells with tan background require inputs• Assumes estimates for Generating and Resultant Adapted SLOC are

discrete and known


NOSTROMO Extensions


NOSTROMO Background (U)

• There are 31 factors normally associated with a COCOMO II model– New and Adapted Size (SLOC) – 17 Cost Drivers– 5 Scale Factors– 3 Adaptation Percentages of Modification– 3 Adaptation Cost Drivers– Requirements Evolution and Volatility

• With any parametric model there are degrees of uncertainty and subjectivity; some factors are more concrete than others (Process Maturity is more concrete than Complexity)

• Based on the ranges of uncertainty, a Monte Carlo Simulation can model several hundreds/thousands of COCOMO II estimates with varying degrees

• A Monte Carlo Simulation of COCOMO II can produce a range of estimates and the risk factors associated with each estimate

• GOAL – Develop a Monte Carlo Simulation of COCOMO II based on the ranges of uncertainty associated with each factor


Original NOSTROMO Concept (U)

• NOSTROMO first takes normal COCOMO II inputs• NOSTROMO accounts for uncertainty in the settings• NOSTROMO uses Monte Carlo to simulate hundreds of COCOMO II estimates using the inputs and uncertainties• NEWT – NOSTROMO Entry Writing Tool – a Delphi polling tool that captures Uncertainty

NEWT

DATAPAGE

REPORTS

NOSTROMO

Notional Obscurity STatistical Risk Observation MOdel


Distributions of Uncertainty (U)

Setting n-1 Setting n Setting n+1

Level 1 - Certain

Level 2 - Low Uncertainty - Normal

Level 3 - Medium Uncertainty - Triangle

Level 4 - High Uncertainty - Uniform

NOSTROMO assumes ceiling and floor limits of the highest and lowest default values for each COCOMO II Scale Factor and Cost Driver

NOSTROMO changes the distribution and sets the high and low points based on the level of uncertainty with each factor


NOSTROMO Application (U)

• Microsoft Excel 2002 worksheet with attached macros and Visual Basic code

• Currently on version 0.3.14 (as of 10/4/04)

• Does not use outside applications for Monte Carlo simulation (standard random number generation from Visual basic)

• Limited testing performed on a proof-of-concept application

• Suggest limiting Monte Carlo run size to 500-2000 runs

• Generates two pages of output – Data and Charts

• Data page contains all outputs from Monte Carlo simulation, charting data, and histograms

• Charts page contains Confidence Intervals for output of COCOMO II and Putnam models, charts, and histograms

• NOSTROMO – Data Sheet and CM Control Sheet are Developer pages and should not be removed


New NOSTROMO Features (U)

• COCOMO 18 Cost Driver Difficulties– In 2003, a breakout group focused on rating the difficulty of COCOMO II Cost Drivers– Results were grouped and assigned Uncertainty values based on grouping in NOSTROMO

• COCOMO 18 Conference Comments– Added Default COCOMO II output to NOSTROMO (data and charts)– Added EAF as output to NOSTROMO (data and charts)

• Conditional Formatting– NOSTROMO Entry Page has red-yellow-green-blank notation associated with Uncertainty

values• Delete Project Function

– Removes all Subcomponent Data and Chart pages• Limited Data Validation

– No zero size estimates– All Cost Drivers and Scale Factors must be populated– No blank Subcomponent names

• Summation of Subcomponents– Tracks at the 25th, 50th, 75th, percentiles and default COCOMO II

• AGC Calculator– Works for a single subcomponent at a time– Calculates ESLOC based on new AGC Calculation rules


NOSTROMO Application Version History (U)


Current NOSTROMO Concept (U)

NEWT

DATAPAGE

REPORTSNOSTROMO

DEFAULT UNCERTAINTY

COMBINATION

AGC CALCULATOR

NOSTROMO 0.3.14 offers:• multiple methods for inputting Uncertainty• combination of subcomponents• more reporting capability• AGC methodology


Screenshots – NOSTROMO Input Sheet – Notional Data (U)

Software Item Name

Scale Factors

Code Size

Cost Drivers

Iterations

Putnam Productivity

Function Buttons

Conditional Formatting


Screenshots – NOSTROMO COCOMO II Charts Sheet – Notional Data (U)


Screenshots – NOSTROMO AGC Calculator – Notional Data (U)


Using NOSTROMO (U)

• Initial Estimates – Use NEWT or Default Setting to identify Uncertainty– Run NOSTROMO to get a range of results– Select results based on business criteria

• “What-If” Scenarios– Isolate Cost Drivers and Scale Factors in question by setting Uncertainty > 1– Set non-factors to Uncertainty = 1– Run NOSTROMO to identify risks

• Model Validation– Input current model settings as first entry in NEWT– Poll for additional entries based on survey techniques– Run NOSTROMO with size and phase inputs to get a range of results

• AGC Tradeoffs– Run NOSTROMO without AGC estimated– Run NOSTROMO with an AGC estimate and subsequent size Uncertainty– Compare results


Future Enhancements (U)

• Monte Carlo Enhancement for AGC• More Combination reporting and charts• More data checking• Options to change model settings


Conclusion (U)

New methodologies deliver insight into the real risks associated with software development

• New AGC methodology allows software estimators to address the real-world scenario of AGC carried across incremental software builds

• New NOSTROMO methodology incorporates comments from COCOMO 18, new default Uncertainty values, and combinations of subcomponents without losing the original functionality of previous releases

• The AGC and NOSTROMO tools are combined into a single tool suite that addresses issues in software estimation that are relevant, prevalent, and typically problematic

Documents

An employee-owned company Extensions of Auto-Generated Code and NOSTROMO Methodologies (U) Pam McDonaldSandra GilesDan Strickland THAAD Project OfficeTHAAD