The Capability Maturity Model for Software

Software Engineering Institute• US DoD funded institute associated with Carnegie Mellon• Mission is to promote software technology transfer

particularly to defense contractors• Work has been very influential in process improvement

An important role of the SEI is to use the CMM to assess contractors bidding for US govt defense contracts Maturity model proposed in mid-1980s, refined in early 1990s. The model is intended to represent organisational capability not the

practices used in particular projects Within the same organisation, there are often wide variations in

processes used Capability assessment is questionnaire-based with on-site visits

Capability Maturity Model (CMM)

Developed by DoD-sponsored Software Engineering Institute (SEI); includes real-world best-practices from hundreds of companies

Focus - practices under control of the software group

Minimum set of recommended practices that enhance software development and maintenance capability Defines the expectation (the “what”) Implements without constraint (the “how”)

Used by governments and industries to measure ‘maturity’ of software development organizations

…which bridges us into the 5 CMM Maturity Levels…

Level 1Initial

Level 1Initial

Level 2Repeatable

Level 2Repeatable

Level 3 Defined

Level 3 Defined

Level 4Managed

Level 4Managed

Level 5Optimizing

Level 5Optimizing

CMM Five Maturity Levels

Continuous process capability improvement

Product quality planning, tracking of measured software process

Software process defined and institutionalized to provide product quality control

Management oversight and tracking of project; stable planning and product baselines

Ad hoc; success depends on heroes

ResultKey Process Areas

Level Characteristic

Optimizing Continuous process Process change management (5) capability improvement Technology change mgmt Defect prevention

Managed (4)

Defined Software process defined (3) and institutionalized to provide product quality control

Repeatable (2)

Initial (1)

Product quality planning; Software quality managementtracking of measured Quantitative process managementsoftware process

Management oversightand tracking of project;stable planning andproduct baselines

Key Process Areas

Ad hoc(success depends on heroes)

"People"

Productivity& Quality

RiskRisk

Productivity& Quality

Software configuration management Software quality assurance Software subcontract management Software project tracking & oversightSoftware project planningRequirements management

Peer reviews Intergroup coordinationSoftware product engineering Integrated software managementTraining programOrganization process definitionOrganization process focus

SEI Capability Maturity Model

1

Time/$/...

Targ

et N

Prob

abilit

y

2

Time/$/...

Targ

et N

+a

Time/$/...

3

Targ

et N

-x

Time/$/...

4Ta

rget

N-y

Time/$/...

5

Schedule and cost targets are typically overrun at Level 1

Plans based on past performance are more realistic at Level 2 but estimates are perceived to be "worse”

With well-defined processperformance improves at Level 3

Based on quantitative understandingof process and product, performance continues to improve at Level 4

Performance continuously improves at Level 5

Targ

et N

-z

Act

ual

Why Increase Your Process Maturity?

Level 2 Key Process Areas & Purposes

Project-centered focus

KPA PURPOSERequirements Management

Establish a common understanding of requirements among the customer and all development groups

Software Project Planning Establish complete & reasonable project plans

Software Subcontractor Management

Support selection of qualified subcontractors and effective management of their activities

Software Quality Assurance

Provide mgmt with visibility into process & products

Software Configuration Establish & maintain the integrity management of the software products throughout the life cycle

Level 3 Key Process Areas & Purposes

KPA PURPOSEOrganization Process Focus To establish organizational responsibility for

software process improvement activities

Organization Process Definition To develop and maintain software process assets for the organization

Training Program To develop the skills and knowledge of individuals so they can perform their roles effectively and efficiently

Integrated Software Management

To integrate the software engineering and management activities into a coherent, defined software process for better project management

Organization-centered focus

KPA PURPOSESoftware Product Engineering To integrate all the appropriate software

engineering activities to produce correct, consistent software products

Intergroup Coordination To promote teamwork among all engineering groups on a project, and to better satisfy the customer’s needs

Peer Reviews To remove defects from the software work product early and efficiently, and reinforce engineering practices and standards

Organization-centered focus

Level 3 Key Process Areas & Purposes(…continued)

Level 4 Key Process Areas & PurposesKPA PURPOSE

Software Quality Management

Develop a quantitative understanding of the quality of software products and achieve specific quality goals

Quantitative Process Management

Control process performance of the software project quantitatively. Focus on identifying special causes of variation within a stable process and correcting the problems that caused the variation to appear.

Process-centered focus

Level 5 Key Process Areas & PurposesKPA PURPOSE

Process Change Management

Continually improve software processes

Technology Change Management

Identify new and useful technologies and incorporate them into the software process in an orderly manner

Defect Prevention Identify causes of recurring defects, particularly those related to process; prevents such defects from recurring

Continuous Software Process Improvement

CMM Pros and Cons

Advantages: The CMM has become a U.S. DoD standard The CMM widely used internationally The CMM is well publicized, understood, and

supported • Lots of training, workshops, and symposia

Process Improvement is Measurable Many anecdotal reports of benefits

CMM Pros and Cons

Disadvantages: Pressure to “study for the test” Relatively easy to achieve a given level in name only Not “modernized” for non-DoD, small software houses Focuses on project management rather than product

development Ignores the use of technologies like rapid prototyping,

risk analysis Steep learning curve due to immense ambiguity Overly bureaucratic for a supplier selection model Encourages/ promotes low maturity (SEI Level 2/ 3)

Organization Payoff Summary References

Boeing Info. Systems

Project estimates within 20% using historical data, CPK 38% better, defect containment effectiveness at 80%, cycle time improved 36%, staff support needs down 62%, staff size reduced 31%, customer satisfaction score up10%, $5.5M saved in 1996 alone (1992 – 1996 results)

Vu, J. (1997)

Boeing STS Customer satisfaction rated excellent, pre-release defect containment effectiveness at 99%, 31% reduction in rework-inspections benefit, employee satisfaction level from mean of 5.7 to 8.3, operational systems performance close to bull's-eye, level 5 process injected into new programs

Yamamura, G. & Wigle, G. (1997)

Bellcore Defects 10X lower than industry average, customer satisfaction rates improved from 60 to 91% over 4 years, achieved 9 hr. cutover to add 888 to 800 system with no reported defects.

Bellcore Press Release, Feb. 5, 1997

HP SESD 3X3 SPI program, 1 year benefits include: cycle time reduced by 33%, major open defects reduced from 4.6 to 1.6, fewer missed deadlines, ROI 9:1

Lowe & Cox (1996)

Harris ISD DPL

2.5X productivity gain over norm, 90% defect rate reduction, cycle time down to 6-9 months Robeson, D., Davidson, S. & Bearden, L. (1997)

Motorola 3X productivity improvement, 3X cycle time reduction, 7X quality improvement, results from ‘92-’96 representing 85% of all products & released software, 75% of product development orgs. Are >= level 3

Major, J. (1996)

Motorola GED On 34 current programs compared to baseline – each CMM level increases quality by 2X, significant decreases in cycle time as higher levels reached (2-7X), productivity increases of 2-3X at highest levels of maturity, 6.77X SPI ROI

Diaz, M. & Sligo, J. (1997)

SAIC Health Tech.

50% improvement in customer satisfaction, 71% reduction in error rate, 12% annual improvement in developer productivity, production rate up 30%

Lane, J. & Zubrow, D. (1997)

Resource: Noumena Consulting Group http://www.noumena.com/sw-roi-published.htm#1996-1997

Summary of US Org SPI Payoffs

CMM CostsExample Costs:

Up to 5 to 7 years for solid implementation US $250,000 to $10,000,000 for initial implementation US $200,000 per formal external audit Results in up to 4,000 pages of software processes Results in over 20,000 pages of formal audit results

SEI Response: “Branding” of the CMM:• Capability Maturity Model Integration (CMMISM)

• SW-CMM® Capability Maturity Model for Software

• P-CMM People Capability Maturity Model

• SA-CMM Software Acquisition Capability Maturity Model

• SE-CMM Systems Engineering Capability Maturity Model

• IPD-CMM Integrated Product Development CMM

Tailored CMM to small organizations Defined “Team” & “Personal” software processes

CMM: What’s Next?

CMM is “sunsetted” (retired) Training not offered after 2003 Appraisals continuing to 2006

Organizations asked to migrate to CMMI The “Capability Maturity Model for Integration”

• Adds KPAs for Measurement (L2), Acquisition (L3)• Adds additional capabilities outside the scope of CMM

– e.g. outsourcing, acquisition, and risk management

Will this fly? Organizations invested a lot to get CMM levels Lightweight processes are becoming more popular CMM will always remain as a historical marker

• The 1st systemic approach to software process maturity• Other standards (ISO-9000) not software specific