Technical Tasks
Riding Herd on Total Ownership Costs: Herding Sheep vs. Herding
Cats
Barry Boehm, USCSTC 2015 Keynote AddressOctober 13, 2015
2/21/201111Cowboys Herding Cats2/21/20112
OutlineSoftware O&M Often like Herding CatsSome Root Causes
of Cat-Like BehaviorMany opportunities to reduce total ownership
costa (TOC)By emphasizing software Changeability and
DependabilityBoth rely on Maintainability via SERC System Qualities
OntologyOpportunities organized via Maintainability Opportunity
TreeAnticipate Modifiability NeedsDesign, Develop for Modifiability
Anticipate Repairability NeedsDesign, Develop for
RepairabilityExpedite Diagnosis Improve Modification and Repair
Verifiability; SkillsConclusions
2/21/20113Problem and Opportunity (%O&M costs)
US Government IT: >75%; $62 Billion [GAO 2015]Hardware
[Redman 2008]12% -- Missiles (average)60% -- Ships (average)78% --
Aircraft (F-16)84% -- Ground vehicles (Bradley)Software [Koskinen
2010]75-90% -- Business, Command-Control50-80% -- Complex platforms
as above10-30% -- Simple embedded softwarePrimary current emphasis
minimize acquisition costs2/21/201144Software O&M Often like
Herding CatsSoftware and Users evolve in Incompatible
directionsNon-Developmental Items (COTS, Clouds, Open
Source)Independently evolving co-dependent external
systemsMulti-mission sources of changeBreakage of brittle
point-solution architecturesPriority changes: competition,
technology, organizationsHerders are often ill-preparedMinimal
voice in acquisitionMissing deliverables: diagnostics, test
support, architecture documentation, CM supportDiversity of
deliverables from multiple sourcesUnfamiliar domains,
infrastructureMissing capabilities: Rainy-day use cases
2/21/20115Some Root Causes of Cat-Like BehaviorStovepipe
acquisition of interoperating systemsIncompatible infrastructure,
NDIs, user interfaces, Acquisitions based on lowest-cost,
technically-acceptable implementation of fixed requirements,
resulting inBrittle, point-solution architecturesCAIV-driven loss
of information on post-delivery needsMinimal interpretation of
technically acceptableJust implementing sunny-day
requirementsMinimal maintainer participation, planning,
preparationMissing maintainer deliverables: diagnostics, test
support, architecture documentation, CM supportIncompatibilities
among post-deployment evolution partiesInadequate SysE resources,
leading to severe technical debtFirst to be impacted by optimistic
budgets and schedules
2/21/20116The Conspiracy of Optimism Take the lower branch of
the Cone of Uncertainty
USC-CSSE73/6/2011
706/25/07USC-CSSE8Added Cost of Minimal Software SysEBased on
COCOMO II calibration data
06/25/07USC-CSSE9How Much Architecting is Enough?
Percent of Project Schedule Devoted to Initial Architecture and
Risk Resolution
Added Schedule Devoted to Rework(COCOMO II RESL factor)
Total % Added Schedule10000KSLOC100 KSLOC10 KSLOCSweet SpotSweet
Spot Drivers:Rapid Change: leftwardHigh Assurance:
rightwardOutlineSoftware O&M Often like Herding CatsSome Root
Causes of Cat-Like BehaviorMany opportunities to reduce total
ownership costa (TOC)By emphasizing software Changeability and
DependabilityBoth rely on Maintainability via SERC System Qualities
OntologyOpportunities organized via Maintainability Opportunity
TreeAnticipate Modifiability NeedsDesign, Develop for Modifiability
Anticipate Repairability NeedsDesign, Develop for
RepairabilityExpedite Diagnosis Improve Modification and Repair
Verifiability; SkillsConclusions
2/21/201110Maintainability Challenges and
ResponsesMaintainability supports both Dependability and
ChangeabilityDistinguish between Repairability and
ModifiabilityElaborate each via means-ends relationshipsMultiple
definitions of ChangeabilityDistinguish between Product Quality and
Quality in Use (ISO/IEC 25010)Provide mapping between Product
Quality and Quality in Use viewpointsChangeability can be both
external and internalDistinguish between Maintainability and
AdaptabilityMany definitions of ResilienceDefine Resilience as a
combination of Dependability and ChangeabilityVariability of
Dependability and Changeability valuesOntology addresses sources of
variationReferents (stakeholder values), States, Processes,
Relations with other SQs Need to help stakeholders choose options,
avoid pitfallsQualipedia, Synergies and Conflicts, Opportunity
Trees
Product Quality View of ChangeabilityMIT Quality in Use View
also valuableChangeability (PQ): Ability to become different
productSwiss Army Knife, Brick Useful but not
ChangeableChangeability (Q in Use): Ability to accommodate changes
in useSwiss Army Knife does not change as a product but is
ChangeableBrick is Changeable in Use (Can help build a wall or
break a window?)
ChangeabilityAdaptabilityMaintainabilitySelf-DiagnosabilitySelf-ModifiabilitySelf-Testability
InternalExternalModifiabilityRepairabilityDefectsChangesTestabilityMeans
to EndSubclass of
UnderstandabilityModularityScalabilityPortabilityDiagnosabilityAccessibilityRestorabilityABABC,DMeans
to EndSubclass ofDependability,
AvailabilityReliabilityMaintainabilityDefect
FreedomSurvivabilityFault ToleranceRepairabilityTest Plans,
CoverageTest Scenarios, DataTest Drivers, OraclesTest Software
QualititiesTestabilityCompletePartialRobustnessSelf-RepairabilityGraceful
DegradationChoices of Security,SafetyModifiabilityDependability,
Changeability, and ResilienceTestability, Diagnosability,
etc.06/01/2015USC-CSSEChangeabilityResilienceOutlineSoftware
O&M Often like Herding CatsSome Root Causes of Cat-Like
BehaviorMany opportunities to reduce total ownership costa (TOC)By
emphasizing software Changeability and DependabilityBoth rely on
Maintainability via SERC System Qualities OntologyOpportunities
organized via Maintainability Opportunity TreeAnticipate
Modifiability NeedsDesign, Develop for Modifiability Improve
Modification V&VAnticipate Repairability NeedsDesign, Develop
for RepairabilityImprove Repair DiagnosisImprove Repair, V&V
ProcessesConclusions
2/21/20111408/04/201515Maintainability Opportunity Tree:
ModifiabilityAnticipate Modifiability Needs Design/Develop for
ModifiabilityImprove Modification V&VEvolution
requirementsTrend analysisHotspot (change source) analysisModifier
involvementSpare capacity; product line engineeringDomain-specific
architecture in domain Regression test capabilities Address
Potential ConflictsAddress Potential ConflictsAddress Potential
Conflicts
Service-orientation; loose couplingModularize around
hotspotsEnable user programmability
Modification compatibility analysis
Prioritize, Schedule Modifications, V&V
08/04/201516Maintainability Opportunity Tree:
RepairabilityAnticipate Repairability Needs Design/Develop for
RepairabilityImprove Repair DiagnosisRepair facility
requirementsRepair-type trend analysisRepair skills analysisRepair
personnel involvementReplacement parts logistics development
Replacement accessibility deaignMultimedia diagnosis guidesFault
localizationAddress Potential ConflictsRepair compatibility
analysisRegression test capabilities Address Potential Conflicts
Address Potential ConflictsImprove Repair, V&V ProcessesSmart
system anomaly, trend analysisInformative error messagesReplacement
parts trend analysisRepair facility design, developmentAddress
Potential Conflicts
Switchable spare componentsPrioritize, Schedule Repairs, V&V
Elaborating Modifiability Benefits - IEvolution RequirementsKeep,
prioritize below-the-line IOC requirementsUse to determine
modularization around sources of change, reduce ripple effects of
changesTrend AnalysisIdentify, prioritize responses to sources of
changeMarketplace, competition, usage trends, mobility trendsUse to
refine, evolve architectureAgile Methods, User
ProgrammabilityEnable rapid response to rapid changeHotspot
AnalysisGather data on most common sources of changeUse to
modularize architecture, reduce ripple effects of changes
2/21/201117183/27/2013 Use of Empirical Data in TOC
Models:Pareto 80-20 Cost-to-fix Distribution Contracts: Fixed cost
and nominal-case requirements; 90 days to PDR
18Rework Sources Analysis: Projects A and B- Change processing
over 1 person-month = 152 person-hours3/27/2013 19CategoryProject
AProject BExtra long messages3404+626+443+328+244= 5045Network
failover2050+470+360+160= 3040Hardware-software interface620+200=
8201629+513+289+232+166= 2832Encryption algorithms1247+368=
1615Subcontractor interface1100+760+200= 2060GUI
revision980+730+420+240+180 =2550Data compression
algorithm910External applications interface770+330+200+160=
1460COTS upgrades540+380+190= 1110741+302+221+197= 1461Database
restructure690+480+310+210+170= 1860Routing algorithms494+198=
692Diagnostic aids360477+318+184= 979TOTAL:136201353119C4ISR
Project C: Architecting for ChangeUSAF/ESC-TRW CCPDS-R
Project*3/27/2013 20When investments made in architecture, average
time for change order becomes relatively stable over time* Walker
Royce, Software Project Management: A Unified Framework.
Addison-Wesley, 1998.
20Relative* Total Ownership Cost (TOC)For single system life
cycle (TOC-SS)3/27/2013 21* Cumulative architecting and rework
effort relative to initial development effort~5% architecture
investment~5% architecture investment~25% architecture
investment21Elaborating Modifiability Benefits IIand Repairability
BenefitsService-Oriented Architecture improves
InteroperabilityProduct-Line Engineering improves Total Ownership
Cost (TOC)Identify, modularize around product line
CommonalitiesDevelop domain architecture, interfaces to
VariabilitiesFewer components to modify, repairImproved
Repairability improves Availability, TOCAvailability = MTBF / (MTBF
+ MTTR)Stakeholder Value-Based V&V improves Cost, Mission
EffectivenessPrioritizing inspection, test activitiesBalancing
level of inspection, test activities vs. rapid
fielding2/21/20112223Systems Product Line Flexibility Value
ModelSystemsPL Flexibility Value Model
For Set of Products:Average Product CostAnnual Change
CostOwnership TimePercent Mission-Unique, Adapted, ReusedRelative
Cost of Developing for PL Flexibility via ReuseRelative Costs of
ReuseAs Functions of # Products, # Years in Life Cycle:PL Total
Ownership CostsPL Flexibility InvestmentPL Savings
(ROI)2/21/201123Current Systems Product Line Model
2/21/2011242408/04/201525Reuse at HPs Queensferry
Telecommunication Division
TimetoMarket
(months)Non-reuse ProjectReuse project2/21/201126Tradeoffs Among
Cost, Schedule, and Reliability: COCOMO II
-- Cost/Schedule/RELY: pick any two points (RELY, MTBF (hours))
For 100-KSLOC set of featuresCan pick all three with 77-KSLOC set
of features26Cost of Downtime SurveyIndustry
SectorRevenue/HourEnergy$2.8 millionTelecommunications $2.0
millionManufacturing $1.6 millionFinancial Institutions $1.4
millionInformation Technology $1.3 millionInsurance $1.2
millionRetail $1.1 millionPharmaceuticals $1.0 millionBanking
$996,000Source: IT Performance Engineering & Measurement
Strategies: Quantifying Performance Loss, Meta Group, October
2000.
2/21/20112727Added TOC for Systems of Systems (SoSs)
Multi-mission sources of changeMany independently-evolving
co-dependent systemsOver 155 on Future Combat
SystemsMultiple-contract-change complexitiesAverage 141 vs. 48
workdays to process on 2 large SoSsLimited ability to apply
interoperability improvements
These also affect systems that become parts of
SoSs2/21/201128282/21/201129Examples of Utility Functions: Cost of
DelayValueTimeReal-Time Control; Event SupportValueTimeMission
Planning, Competitive Time-to-MarketValueTimeEvent Prediction-
Weather; Software SizeValueTimeData Archiving
Critical Region292/21/201130Cost of Delay vs. Dependability
Assurance- Early Startup: Risk due to low dependability -
Commercial: Risk due to low dependability - High Finance: Risk due
to low dependability - Risk due to market share erosion
COCOMO II: 012223454Added % test timeCOQUALMO:
1.0.475.24.1250.06P(L)Early Startup: .33.19.11.06.03S(L)Commercial:
1.0.56.32.18.10S(L)High Finance: 3.01.68.96.54.30S(L)Market Risk:
.008.027.09.301.0REmSweet Spot
30Addressing Potential Conflicts With Performance: Loose vs.
tight coupling (supercomputing) With Development Cost and Schedule:
More to design, develop, V&V (rapid fielding)With Usability:
Too many options (Office 2010)With Security: Too many entry points
(Windows)With Scalability, Safety, Security: Agile methodsWith
Dependability: User Programming, Self-AdaptivenessWith
Interoperability: Multi-Domain ArchitecturesWith Cost, Resource
Consumption: Spare Capacity
2/21/201131These are not always conflicts, but candidates to
consider. Need to balance risk of too little Modifiability with
risk of too much.
Example of Use: MS A Preparation GuidanceAlternatives analyzed
shall include at least one architecture organized around:Common
sources of life-cycle changeFor RPVs, these usually include user
interface displays and controls, new payloads, self-defense and
electronic warfare, data fusion, NCSoS protocols, and hardware
maintainabilityRisk analysis and prototyping of critical
off-nominal scenariosFor RPVs, these usually include communications
outages, anti-RPV threats, noisy and intermittent data sources,
redirected missions, and cross-RPV coordination of
responsesAnalyses of alternatives shall include total ownership
cost comparisonsBased on relevantly-calibrated life cycle cost
models2/21/20113232OutlineSoftware O&M Often like Herding
CatsSome Root Causes of Cat-Like BehaviorMany opportunities to
reduce total ownership costa (TOC)By emphasizing software
Changeability and DependabilityBoth rely on Maintainability via
SERC System Qualities OntologyOpportunities organized via
Maintainability Opportunity TreeAnticipate Modifiability
NeedsDesign, Develop for Modifiability Improve Modification
V&VAnticipate Repairability NeedsDesign, Develop for
RepairabilityImprove Repair DiagnosisImprove Repair, V&V
ProcessesConclusions
2/21/201133Conclusions Major opportunities to reduce Total
Ownership CostsUS Government IT: $62 Billion O&M (over 75% of
TOC)
Current investments overfocused on initial acquisition
costsResulting in numerous herding-cats phenomena
Maintainabilty Opportunity Trees offer attractive means to
reduce Total Ownership CostsBut need to balance risks of too little
Maintainability with risks of too much
2/21/201134Chart818389159643231430681078402410214817653827714302555393237143347403600050505050
Percent of Time Added for Architecture and Risk
ResolutionPercent of Time Added to Overall Schedule
Sheet1
Sheet118389159643231430681078402410214817653827714302555393237143347403600050505050
Percent of Time Added for Architecture and Risk
ResolutionPercent of Time Added to Overall Schedule
Sheet2
Sweet Spot10000KSLOC100 KSLOC10 KSLOCPercent of Project Schedule
Devoted to Initial Architecture and Risk Resolution
Added Schedule Devoted to Rework(COCOMO II RESL factor)
Total % Added Schedule
Sheet3
Chart15.466.136.667.338.394.354.895.315.856.693.824.294.665.135.873.824.294.665.135.873.824.294.665.135.87
(VL, 1)(L, 10)(N, 300)(H, 10K)(VH, 300K)Development Time
(Months)Cost ($M)
Sheet1Development Time (Months)(VL,
1)18.25.4620.74.3524.33.8231.63.8238.93.82(L,
10)18.96.1321.44.8925.24.2932.84.2940.44.29(N,
300)19.46.66225.3125.94.6633.74.6641.44.66(H,
10K)207.3322.75.8526.75.1334.75.1342.75.13(VH,
300K)20.98.3923.76.6927.95.8736.35.8744.65.87
Sheet10000000000000000000000000
(VL, 1)(L, 10)(N, 300)(H, 10K)(VH, 300K)Development Time
(Months)Cost ($M)
Sheet2
Sheet3
Chart10.0080.331.0083.0080.338130.0270.090250.2930.8250.117250.2660.7980.090.02640.16680.32040.11640.07680.23040.30.00750.32250.36750.30750.02250.067510.00181.0061.0181.00180.0060.018
Market Share ErosionEarly StartupCommercialHigh FinanceRELYRE =
P(L) * S(L)Combined Risk Exposure
Chart20.0080.33130.3381.0083.0080.0270.090250.2660.7980.117250.2930.8250.090.02640.07680.23040.11640.16680.32040.30.00750.02250.06750.30750.32250.367510.00180.0060.0181.00181.0061.018
Market Share ErosionEarly StartupCommercialHigh FinanceRELYRE =
P(L) * S(L)Combined Risk Exposure
Sheet1RELYMarket Share ErosionEarly StartupCommercialHigh
FinanceVL0.0080.33130.3381.0083.008L0.0270.090250.2660.7980.117250.2930.825N0.090.02640.07680.23040.11640.16680.3204H0.30.00750.02250.06750.30750.32250.3675VH10.00180.0060.0181.00181.0061.018
Sheet2
Sheet3
