Productivity Data Analysis and Issues

University of Southern California

Center for Systems and Software Engineering

Productivity Data Analysis and Issues

Brad Clark, Thomas Tan

USC CSSE Annual Research Review

March 8, 2010



USC CSSE Annual Research Review - Mar 2010 2

Table of Contents

• Background• Productivity Data Analysis by Application Domain• Reducing the number of domains: Application Difficulty• Topics for further discussion

This work is sponsored by the Air Force Cost Analysis Agency




Background• DoD has been collecting Software resource data for a number of

years– Product and development description

– Product size

– Resources and schedule

– Product quality

• Analyzing ~140 records out of ~300– Additional data is coming in

• Objective: Improved cost estimation of future DoD software-intensive systems, as well as to the DoD cost community. – Characterize different Application Domains within DoD

– Analyze collected data for simple cost estimating relationships within each domain

– Develop rules-of-thumb for missing data

• Make collected data useful to oversight and management entities



Software Resources Data Report





SRDR Data

Notes:SRDR: Software Resources Data Report

Application Domain AvionicsFixed

Ground MissileMobile Ground Shipboard

Unmanned Space Total

Business Systems 4 4Command & Control 1 8 5 14Communications 1 35 2 1 39Controls & Displays 2 1 1 3 7Executive 3 3Information Assurance 1 1Infrastructure or Middleware 2 1 3Mission Management 12 2 3 1 18Mission Planning 1 4 5Process Control 4 4Scientific Systems 3 3Sensor Control and Processing 2 10 12Simulation & Modeling 9 3 12Spacecraft Payload 1 1Test & Evaluation 1 1Tool & Tool Systems 3 3Training 1 1Weapons Delivery and Control 4 7 11Total 21 68 10 16 25 2 142

Operating Environment

Missing Domains: Internet, Maintenance and Diagnostics, Spacecraft bus




Preliminary Results - Do Not Use!




PM = A * (EKSLOC)B

Simple Cost Estimating Relationships

Notes:PM: Person Months (152 labor hours / month)EKSLOC: Equivalent Thousands of Source Lines of Code

Preliminary Results - Do Not Use!



Sizing Issues -1• Multiple SLOC counting methods

– Physical: total number of lines in a file

– Non-commented Source: no blank or comment lines

– Logical

• No Deleted Code Counts

• SLOC Conversion Experiment– Use the results of USC’s Code Count Tool to find conversion ratios

– Physical to Logical

– NCSS to Logical

– Results segregated by programming language




NCSS to Logical Conversion


Ada: 45%C/C++: 61%C#: 61%Java: 72%



Sizing Issues -2• No Modified Code parameters

– Percent Design Modified (DM)

– Percent Code Modified (CM)

– Percent Integration and Test Modified (IM)

– Software Understanding (SU)

– Programmer Unfamiliarity (UNFM)

• Program interviews provided parameters for some records




Effort Issues• Missing effort reporting for different lifecycle phases

– Software requirements analysis (REQ)

– Software architectural design (ARCH)

– Software coding and testing (CODE)

– Software integration (INT)

– Software qualification testing (QT)

– Software management, CM, QA, etc. (Other – very inconsistent)





Collapsing Application Domains• Propose to reduce the number of application domains

– Currently have a “sparse” data table

• Use a model-independent approach– 5-level scale to capture the “difficulty” (and therefore impact) of an

application domain on productivity




Software Application Difficulties

Very Easy

§ Simple handling of events / inputs

§ Relies on O/S or middleware for control

§ Can be restarted with minor inconvenience

Very Challenging § Autonomous

operation § Extremely high

reliability § Complex algorithms § Micro-second

control loop § Micro-code

programming

Firmware (ROM)

Attitude Control

Guidance / Navigation

Radar / Sonar / Telemetry

processing

Process Control

Seismic Processing

Factory Automation

Digital Switches & PBX

Case Tools

/ Compilers

Website Automation

Business Systems

Difficulty would be described in terms of required software reliability, database size, product complexity, integration complexity, information assurance, real-time requirements, different levels of developmental risks, etc.

Application Domains Very Easy Easy Nominal Challenging Very Challenging

Business Systems Large biz system

Trillion $/day transaction

InternetSimple web

pagesWeb application

(shopping)

Mega-web application

Tools and Tool Systems Verification

toolsSafety critical

Scientific Systems Offline data reduction

Large dataset

Simulation and Modeling Low fidelity simulator

Physical

phenomenon

Test and Evaluation Usual Distributed debugging

Training Set of screens Simulation

network

Command and Control Taxi-cab dispatch

SOS (C4ISR)

Mission Management UsualMulti-level security

and safety

Weapon Delivery and Control

Weapon space Safety

Communications Noise,

anomalies handling

Radio Safety/Security

Frequency-hopping

Application Difficulty Issues

14USC CSSE Annual Research Review - Mar 2010




Questions?

For more information, contact:

Thomas Tan

[email protected]

626-617-1128

Or

Brad Clark

[email protected]

703-754-0115

Documents

Productivity Data Analysis and Issues