DIF 8901 No. 1

COTS-Based Systems Top 10 Lists

Victor R. Basili, Barry Boeham

Addresser: Hao Ding

NTNU/IDI, March 5th, 2003

DIF 8901 No. 2

What is this paper for? Top 10 in COTS-Based Systems (CBS) Conclusion

Outline

DIF 8901 No. 3

What is this paper for? Intention:

starting point for the software community: developing solid empirical methods for coping with COTS-Based data.

Characteristics of CBS: The buyer has no access to the source code; The vendor controls its development; Nontrivial installed base (>one customer, >a few

copies)

DIF 8901 No. 4

Criteria for making the list: Each empirical result has significant current

and future impact on software dependability, timeliness, and cost;

Diagnostic value with respect to cost-effective best practices;

Reasonable generality across applications domains, market sectors, and product sizes.

What is this paper for? (con’d)

DIF 8901 No. 5

What is this paper for? (con’d)

Maintenance

DisplacementImplementation

ContagionInitiation

Visibility

Recommended Adoption Time Frame

Type A Type B Type CTime

BandwagonEffect

Slope ofEnlightenment

Plateau ofProductivity

TechnologyTrigger

Trough of Disillusionment

Peak of Inflated

Expectations

Software Adoption Curve

DIF 8901 No. 6

Top 10 in COTS-Based Systems (CBS) Hypotheses rather than results Software challenges for enhancing our empirical

understanding of CBS 10 hypothesis:

DIF 8901 No. 7

Hyperthesis One

These are the same criteria we used for More than 99 percent of all executing computer instructions come from COTS products. Each instruction passed a market test for value.

DIF 8901 No. 8

Hyperthesis Two

More than half the features in large COTS software products go unused.

DIF 8901 No. 9

Hypothesis Three

The average COTS software product undergoes a new release every eight to nine months, with active vendor support for only its latest three releases.

DIF 8901 No. 10

Hypothesis Four

CBS development and post-deployment efforts can scale as high as the square of the number of independently developed COTS products targeted for integration.

That is, integrating n COTS products involves potentially n(n － 1)/2 interfaces.

DIF 8901 No. 11

Hypothesis Five

CBS post-deployment costs exceed CBS development costs

Example: A cost-constrained developer on a 27-month project might be tempted to present a maintainer from a different organization with about-to-become-unsupported COTS products.

DIF 8901 No. 12

Hypothesis Six

Although glue-code development usually accounts for less than half the total CBS software development effort, the effort per line of glue code averages about three times the effort per line of developed-applications code.

DIF 8901 No. 13

Hypothesis Seven

Non-development costs, such as licensing fees, are significant, and projects must plan for and optimize them

DIF 8901 No. 14

Hypothesis Eight

CBS assessment and tailoring efforts vary significantly by COTS product classes –operating system, database management system, user interface, device driver, and so forth.

DIF 8901 No. 15

Personal capability and experience remain the dominant factors influencing CBS-development productivity.

Hypothesis Nine

DIF 8901 No. 16

CBS is currently a high-risk activity, with effort and schedule overruns exceeding non-CBS software overruns. Yet many systems have used COTS successfully for cost reduction and early delivery.

The Standish Group’s CHAOS report (http://www.scs.carleton.ca/~beau/PM/Standish-Report.html )

Hypothesis Ten

DIF 8901 No. 17

Intention of the list Reprenting preliminary hypotheses for validity-checking

CBS project decisions against small empirical data samples with explained variablility sources

Starting point for the software community

For more general information: http://www.sei.cmu.edu/cbs/ http://www.cebase.org/cots/

<END>

Conclusion

DIF 8901 No. 18

Populating Software RepositoriesIncentives and Domain-Specific Software

Jeffrey S. Poulin

Addresser: Hao Ding

NTNU/IDI, March 5th, 2003

DIF 8901 No. 19

What is this paper for? Intention:

This article presents IBM’s experiences with a corporate RSL reuse incentive programs, and summarizes the results of an enterprise-wide initiative to develop reusable software for the RSL.

DIF 8901 No. 20

RSL RSL - Reusable Software Library The place to deposit software for use Do not guarantee success Depending on factors like availability of quality

and useful software Domain-specific considerations most often

determine the usefulness of software and should therefore influence the population of an RSL

DIF 8901 No. 21

IBM’s RTSC IBM formed the Reuse Technology Support Centre

(RTSC) to promote formal reuse throughout the corporation

Promoting and forwarding “formal” reuse throughout the corporation

RTSC worked with designated representative from development sites

RTSC sponsored enhancement to an existing library system so that it could support and control worldwide distribution of assets

DIF 8901 No. 22

Reuse library considerations 1980’s : reuse programs focused on RSL Quite often ”the reuse program become the

library” Resources are spent on library related activities,

not on the reuse methods and technology. Large RSLs suffer from many drawbacks that

reduce their effectiveness.

DIF 8901 No. 23

The Reuse Library Progression Phase 1

very few parts Phase 2

many parts of low or poor quality Phase 3

many parts of little or no use

DIF 8901 No. 24

RSL overhead

Overhead caused by the formality and rigor needed to manage large quantities of datae.g.. Extensive classification scheme used to describe software components

DIF 8901 No. 25

A Corporate Reuse Library The IBM strategic RSL provides a

central repository for sharing, managing, and reusing software-related products to company cites worldwide

A distributed system of shared libraries The library is owned by the ”local” organization

and access can be granted to RSL users from other sites

Based on a detailed classification scheme to support locating of relevant software

Software in the library to be reused is copied to the local machine

DIF 8901 No. 26

Component management in the RSL Intention:

To assist library managers: Component management functions:

Library access control Component user registration Version control Problem reporting Library shadowing Standard enforcement

DIF 8901 No. 27

Reuse Incentive Programs RTSC in 1991 faced a corporate RSL in the first

stage of the three phases Feedback indicated a need for a wider selection

of quality and useful software RTSC did not sponsor a centralized incentive

program but encouraged and offered guidance to sites that wanted to start or have their own

DIF 8901 No. 28

The supplier-consumer relationship

The supplier those who write and contribute quality

reusable software and related information for others

The reusers those who extract from the RSL and

incorporates the software in their products rather than develop it again (the consumer)

DIF 8901 No. 29

Recognizing suppliers When determining the proper recognition the

program should address: Participation

allocating points to first-time contributors helps motivating developers and generates interest

Popularity Provide more recognition to contributors of very popular

components encourages contributors to look for high-reuse opportunities in their projects

Value A large, robust and generalized component provides more

economic benefit to the organization than small, specific components

DIF 8901 No. 30

SIRBO SIRBO - Source Instructions Reused By Others Recognizes part suppliers The size (line of codes) for each component the

individual contributes by the number of products or organizations actually using it.

Helps ensure that suppliers do not receive recognition for unused software

Rewards suppliers who identify widely needed functions

Rewards contribution of large parts

DIF 8901 No. 31

Recognizing Reusers Reuse results from a business analysis showing possible

financial savings to the organization Forms the basis for individual recognition Recognizing reusers also helps create a demand for

reusable components that leads to financial benefits. RSI – Reused Source Instructions

It provides an excellent metric for rewarding reusers. How to calculate the RSI for a reuser?

Counting the number of code lines in each reused component Credit is received first time component appears in a

product

DIF 8901 No. 32

Recognizing teams A team incentive can reward a number of group

activities: reward developer teams who achieve

a pre-specified goal for the level of reuse receive an unforeseen exceptionally high level of reuse

reward ”parts centre” based on SIRBO

Reward any team with a high number of individuals participating in reuse activities

DIF 8901 No. 33

A corporate incentive program Local reuse programs at IBM started to develop

and mature, but the overall quantity of parts available for general corporate use seemed to stabilize

RTSC sponsored the development of reusable software ”The Parts Stimulation Program”

Moving from phase 1 without the pitfalls of phase 2 and 3

DIF 8901 No. 34

The Parts Stimulation Program Strategy getting development organizations to realize that

the projects would be more valuable if they made portions of the software reusable

the program funded the dev. organizations to study their projects and the business issues related to reuse

the Parts Stimulation Program funded the front-end expenses of conducting the reuse study

performing a domain analysis developing the specification for the reusable parts completing a reuse business case detailing who else

would benefit and the expected return of investments

DIF 8901 No. 35

Incentive Program Results RTSC decided to fund alternative projects that might

benefit the corporation reuse standard documents development tools study of an area of future development

Parts Stimulation Program Summary in RSTC

DIF 8901 No. 36

What makes software usable Programmers cannot reuse a software unless it is

useful Categories for software

DIF 8901 No. 37

Domain independent software single function or set of variables higher-level abstractions may include numerous

interrelated data and functions that act together to perform a specific task

high quality rarely contributes more than 15 –20 % to the total

product making it easy to reuse domain-independent

reusable components is an easy, straight-forward and low cost way to get started in reuse

DIF 8901 No. 38

Domain specific software Findings in various projects show high level of

domain specific reuse The best results will come from a domain that:

has limited scope programmers understand has matured to a level of relative stability requires a lot of custom development based on a set of

core functions Creating a domain library can be bottom-up or

top-down

DIF 8901 No. 39

Conclusion Examination on the implementation of a

corporate RSL Discussion on the local and large-scale reuse

incentive programs that aim to populate the RSL Discussion on the relationship between domain.

<END>

DIF 8901 No. 40

A Comparison of Approaches to Investment Analysis

John Favaro

Addresser: Hao Ding

NTNU/IDI, March 5th, 2003

DIF 8901 No. 41

What is this paper for? Background:

Although significant progress has been made in the areas of reuse metrics and cost estimation, much work to date in reuse investment analysis has not always reflected accepted mainstream financial analysis practices.

Intention:Comparing several approaches that have been described in the reuse literature, points out known problems and indicates remedies.

DIF 8901 No. 42

Software reuse economics Three kinds of

activities: Reuse metrics

the measurement of reuse-related characteristics of software

Cost estimation the estimation of

cost and benefits associated with reusable software development

Reuse investment analysis the evaluation of

investment decisions

DIF 8901 No. 43

The investment analysis Is in the domain of the financial corporate analyst Different perspective from the software engineer

reuse is only one alternative for investment is concerned with the best way to allocate capital and

human resources There is always an alternative to reuse

investment equivalent investment in the capital market that

provides some yearly rate of return reuse-oriented investment should be compared with this

Capital investments are analysed with respect to periods of time, and the same approach should be used for reuse investment analysis

DIF 8901 No. 44

Cost estimation and cash flow Cost estimation = cash flow analysis Estimation of cost/benefit Candidate reuse projects has potential cash flows

positive (e.g.. savings by avoiding work) negative (e.g.. work to generalize a component)

Techniques for quantifying economic benefits are emerging.

Cash flows are forecast over a suitable time horizon

Time period must be neutral estimate, not a desired characteristic

Cash flow analysis is an activity prior to investment analysis

DIF 8901 No. 45

Comparison of approaches

Desirable characteristics depend as much as possible only on forecast

cash flow have a quantifiable acceptance rule to guide

the investment decision suitable for comparing and ranking candidate

projects be able to deal with arbitrarily large or small

projects be able to handle projects of arbitrarily long or

short duration

DIF 8901 No. 46

Net present value PV = Ct / (1 + kt)t

PV = Present Value Ct : future cash flow in period t kt : discount rate in period t

NPV = C0 + PV NPV = Net Present Value add the initial investment as a negative value IF NPV>0 THEN can be accepted

DIF 8901 No. 47

NPV characteristics Acceptance rule is based on the value of NPV Permits realistic comparison with alternative

capital investment possibilities Does not depend on arbitrary factors (e.g..

managers instincts) Values are additive, can be ranked, and are

sensitive to scale Large and small alternative can be compared and

combined

DIF 8901 No. 48

Payback

The time or number of uses required to recover the cost of an investment

The payoff threshold value: N0 = E / (1-b)

N0 = number of times a component must be used before its cost is recovered

E = relative cost of developing a component for reuse

b = relative cost of integrating the component

DIF 8901 No. 49

Payback characteristics Acceptance is based on cost recovery within a

cut-off date Intuitively appealing but problematic

cut-off date is arbitrarily and subjective payback is not sensitive to patterns of cash flow problem of scale: the true value (long term) is not taken

into account choice of cut-off period affects whether short or long

lived projects are accepted, tends to penalize forward looking reuse programs

Ad-hoc approach useful for communicating the result of an investment analysis

DIF 8901 No. 50

Average return on book value Software as a capital asset An amortization schedule for the investment for

reusable work products is agreed upon (deducted as appropriate from future cash flows from those work products)

The book rate of return (of an investment) is calculated by:

dividing the avg. profits from predicted cash flows by the avg. net book value of the investment

DIF 8901 No. 51

Avg. return on book value characteristics Acceptance rule is based on the book rate of

return meets some target set by the analyst e.g.. companies current book rate of return of the industry as a whole

The approach is entirely insensitive to a variable cash flow pattern

The calculation is depending on the choice of amortization schedule

Choice of target book rate is arbitrary and subjective

Not satisfactory approach

DIF 8901 No. 52

Internal rate of return

A way of defining the rate of return of a long lived asset

Internal Rate Return (IRR) is related to NPV IRR is the discount rate which makes NPV

equal to zero

C0 = Ct / (1 + IRR)t

Ct : future cash flow in period t

DIF 8901 No. 53

IRR characteristics Acceptance rule:

accept a project if its IRR is greater than the opportunity costs of the project

Can be used to produce results equivalent to NPV Proper use of IRR is more difficult Can be undermined by certain patterns of cash

flow in a project Exhibits problems with respect to the scale of

projects

DIF 8901 No. 54

Profitability Index

Examines the ratio of benefits to costs Conceptually closest to NPV Numerous variations

ROI (return of investment) Q (Quality of investment) CDCF (Cumulative discounted cash flow)

Values are note additive

DIF 8901 No. 55

Conclusion Net Present Value (NPV) Payback Average Return on Book Value Internal Rate of Return (IRR) Profitability Index (PI)

<END>