CMMI Implementation for Software at JPL
NDIA/SEI CMMI Technology Conference and User Group
November 16-20,2003, Denver, CO
P. A. “Trisha” Jansma SQI Deployment Element
Software Quality Improvement (Sal) Project Jet Propulsion Laboratory
California Institute of Technology Pasadena, CA 91 109-8099
Topics to be Covered
k i Background - About JPL and JPL’s Software - Software Quality Improvement
ki JPL’s CMMI Implementation
JPL
Community (SQI) Project Overview
Approach - JPL CMMI Profile - Process Areas by FY - JPL CMMI Achievement Roadmap
ki JPL Lessons Learned - CMMI Implementation - CMMI Class B Appraisals
- SQI Accomplishments to Date - SQI Services and Products
H Backup Slides
1 111 612003 PAJ- 2
About Jet Propulsion Laboratory -1 - ) - -
*...e**.. BS@/ I....*$ ''\ JPL
H Non-profit federally funded research and development center (FFRDC), located in Pasadena, California.
61 Operated under contract by the California Institute of Technology (Caltech) for the National Aeronautics and Space Administration (NASA).
center for robotic exploration of the solar system. - Also conducts tasks for a variety of other federal agencies, such
H Part of the US. aerospace industry, and NASA's lead
as Dept. of Defense, Dept. of Transportation, Dept. of Energy, etc.
H Has approximately 5500 employees: - 4500 in the technical and programmatic divisions - I000 in the administrative divisions.
H Annual budget of approximately $1.4 billion.
1 1 /I 612003 PAJ- 3
n
cn- W
L
cn
0)
m S
5
r" a, 0
v
a, 0
a,' -1
a, Pr
I
a a
L
m 3 L
.- ab
m- C
E' E za,
0
c)
I I
II
I
I
m c, S
E" a, m
m
0
cn w Ir n 7
5 C
a, S
* .I
.-
.. m
I
&
c,
3
LS
m 0
0
N
2
I
cr\ J - Key Motivators for Software .... iiZlS@/'' ... .... ..* Quality Improvement at JPL JPL I
H Some recent, highly visible failures occurred in which software was implicated in mission loss (e.g., Mars '98)
H Experience as well as formal studies revealed frequent budget overruns and schedule slips for mission-critical software.
f H Software is an increasingly significant risk element for a Project.
- Missions require increasing software capability and complexity. - Software must often be developed late in the mission life-cycle,
minimizing opportunities for schedule recovery. I I
I Real changes in the external environment
k i Many missions are in concurrent software development - Institutional processes reduce project start-up times
change was needed. €4 Addressing complex software with aggressive budgets
requires reuse of software implementing common functions. €4 The NASA CIO, Chief Engineering Office, and Office of Safety and
Mission Assurance are requiring all NASA Centers to implement software qua I ity improvement prog rams. 1 1 /I 612003 PAJ- 5
Establish an operational program that results in the continuous measurable
improvement of software quality at JPL.
H Improve software cost and schedule predictability, and the
Id Reduce project start-up times Increase software development productivity
H Reduce software defect rates during testing and operations k i Establish an infrastructure that promotes reuse of software products
quality of mission-critical software
1 1/16/2003 PAJ- 6
h
a, m ..c
I
rc
cn a, >
m S a, cn a, Q
a,
.I
c,
-c,
L
I
5 U 0
z
a, Y
m cn a, U
-
c,
.- 3 A m -
U
S
m n
S 0
m m
a, S
U
S
m S 0
m S
U 0
0
0
0
a,
Q
.- + L
c,
.-
.- - .I
L
.c,
'0' L
E 0
L
+
b I
a a U
S
m a, t
I
6g v)m
cn
s
2 a, m
m S r" C 0
0 0
m h
a, x
3
0
LL
I .- c,
I
m
C
a, .- L
LI
El a,- .c
a,- a, C
m
t
w
.- L 0
LL
0
0
a,
2 C
I
I
.- n I
I -1
.2 a, a, '5
W
O
CU
ak
I
I I
m m
S 0
.- c,
> .-
m
Y
s
1
I I
m m
m
c r y J - - SQI Approach ........*.*.. D#Q/ * "' A Continuous Improvement Focus
SQI transitioned to using CMMl as a framework for achieving JPL's business
JPL
1 1 /I 6/2003
Asset Development and Deployment
Leverage from i nd us try best practices
Involve practitioners
Build from past JPL & industry experiences
Focus on role-based education & training
Conduct outreach via we bsite, brochure and direct contact
1
Project Support
Support all mission software deveiopment, but initially focus on mission-critical software
PAJ- 8
b. .- Considerations Used for Selecting ........ ~S@/‘’ o.....) JPL’s CMMI Profile (PAS by FY) JPL
H Although the Continuous Representation will be used, JPL’s Senior Management and other oversight groups are more familiar and comfortable with the staged level concept, as per CMM.
H SQI should take into account the interdependencies among the CMMI Process Areas, Generic Goals, and Generic Practices in order to formulate capability profiles that make sense.
H SQI should create some opportunities to show progress early - e.g., improvement in Capability Level in one or more
process areas relatively quickly (say within a year) H SQI should strike a balance between choosing improvements
that will be perceived by the overall JPL software community as meaningful, vs. improvements that are potentially “less popular,” but required by the model.
€4 SQI should strike a balance between breadth (number of PAS) and depth (capability level) in proposing profiles.
1 1 /I 612003 PAJ- 9
K&-- i h ’-- JPL’s Initial Target CMMl Profile
I Level 2 PAS I Level 3 PAS I
d '1 Q
)
E m t L.
mm
E" I- I
C a E a > Q
)
m
E
E
c)
1
M
m,
JPL CMMI Lessons Learned -\ ,+
kii Commitment of management is very important - Director for. . . Section Manager, Technical Group Supervisor
Id Bottom-up implementation is tough (vs. “top-down” decree) - Need to win the “hearts and minds” at the local level
Id Focus CMMI Implementation on a small target audience - Focus on four representative “target” sections and at most
Focus section efforts on receptive groups and projects. Monitor projects throughout the year to provide guidance on progress.
two projects per section.
- Name a section Process Engineer and assign a corresponding
- Aim for “Adoption” on the OCM curve for the target sections. - Aim for “Awareness” and “Understanding” on the OCM curve
SQI Representative or “Shepherd” to support him.
for the broader Software Community. Id Shift focus from rote statement of CMMI PAS to a deeper
understanding CMMI processes, goals and practices. 1 1 /I 6/2003 PAJ- 12
kii Emphasize supporting software development (project work) with the
kii Evaluate findings, plus recommendations provided by findings, to
ki Plan activities to raise the level of selected specific and generic
CMMI practices as a guide
prioritize practices to be addressed
practices. - Avoid slavish creation of artifacts to satisfy the model.
ki Conduct annual Class B Appraisals, focusing on CMMI M2 Process
ki Conduct training regularly to support selected CMMI objectives. ki Consider benchmarking other organizations
- Raytheon, Northrup Grumman, other NASA Centers (MSFC & JSC) ki Address PPQA Process Area and GP 2.9 earlier than originally planned
Areas for at most six projects and follow up on findings
since it affects all other PAS,
1 1/16/2003 PAJ- 13
K p \ J - *
....*.........) ESQ/'\ Training Is a Key Component JPL
Cumulative SQI Software Training
Date (Quarter)
1 1 /I 6/2003 PAJ- 14
,-i _ _ JPL Class B Appraisal Approach Q-
..*.*.........> @SQ/’’ JPL
FY2002 Class B Appraisal H 11 CMMI Process Areas included: H 8 CMMI Process Areas included:
FY2003 Class B Appraisal
- Project Planning - Project Monitor and Control
- Project Planning - Project Monitor and Control
- Supplier Agreement Management - Process and Product Quality
- Supplier Agreement Management - Process and Product Quality
Assurance Assu ran ce - Con fig u ra t i o n Man age men t - Configuration Management - Measurement and Analysis - Measurement and Analysis - Requirements Management - Requirements Management - Requirements Development - Verification - Risk Management TTi 6 projects included - Organizational Process Focus - 2 each from 3 target sections - Organizational Process Definition H 6 appraisers (3 experienced, 3 new)
- 1 experienced & 1 new person per team; 1 person per project
k i 4 projects included, plus the SQI Project (for Process Mgmt PAS)
Id 6 appraisers all new to CMMI TTi 2 external CSM Auditors
in 3 “mini-teams” --
H 2 external CSM Auditors 1 1 /I 6/2003 PAJ- 15
Comparison of Appraisal Effort by FY -- - ~
- ---
JPL CMMI Class B Appraisal Effort Comparison
2500
2000
c, E 1500 Q)
1000 Q cn 500 i!
I 0
3 0
-500
-1 000
I FY2002
I Difference
Activities I
FY2003 effort was -70% that of FY2002 ~ _ _
PAJ- 16
Comparison of FY2002 to FY2003
H Effort for FY2002 Class B Appraisal was -2100 hours. H Effort for FY2003 Class B Appraisal was reduced to -1500 hours due
to the following factors: - An experienced team, which led to better a understanding of the
- Assessed 8 PAS for 6 projects vs. 9 PAS for 4 Projects + 2 SQI PAS - A change in the way artifacts were collected:
model and a sharper focus on evidence
Used relaxed criteria for artifacts (“one artifact” rule) Exhortation not to over-engineer things (Le., “Cut time in half.”)
- Involvement of task lead in characterization (on one project) - Interleaving evidence collection, pre-characterization, and leveling - Consultant support throughout the evidence collection and pre-
characterization - But project “protection” of personnel may have excluded relevant
data and did hamper project learning about CMMI best practices PAJ- 17 1 1/16/2003
a, -c
0
0)
S
U S
m cn
U S 3
CI
LC
.- c,
&
co 7
I
a a S
0
>s m a, U
a, cn cn a, U
U
m
I
L
I
7J a, a, S
3
0
>
c-
a, 0 S
a, U
>
a, U S
c
U
S
m r, >
0
0
cn U 0
U 0
cn I= S
.- &
.I
- CI
E d-
U
a, a, 7
c,j
m
m
3
0
S
a, 0 t
C.
m
S
S
m 23)
.I
.- '13 Q
) t
a, -0
0
cn U
a, a, S
m
ni Q
(3
U S
m m a, cn cn a, 0 0
- ? a' 6
U
Q
Q
m
S
ea t m
o
rc
Q) I a cn cn
cn S
0
cn m
uj 4
Q
c)
0
0
t
.c1
+
E trl c 0
c,
- m cn
0
m
c,
d! c)
Xs
a,=
c,
.- m
a, 0
0
.- r 0
cn s
I
m m
s a, 0
. cn .I 0
4-J 0
cn- -c
I 0
a, -
& > 0
.- E 4-J
S 0
m m Q
a, Q
U
m Q
v, I .- 4-J L
L
P
E 0
a, 0 r, 0
!= - 0
-
4-J - 3
0
0 0
CI
4-J .-
a, -
cn
r
a,- Y
a,
m>
U-
TE
0 t
1. m a m t
a
a
C t
m >
a, cn
& Q
Q
a
I I
I m a
m m L
-cI 0
m S
U
.- L
I cn cn m m
t)
L
0
U
a, m
cn
cn C
0
cn cn a I
- a,
U
E" a, -c m
S
U
S
m a, U
S 3
*
.I
-cI
i2
0
0
0
cnk
L
.- -c1
U S 0
0
a, cn h
cn m a, a,
-
.- L
E"
a, cn m
S
-cI
.-
t)
+
0
m k r m a,
s
CI
0
0
0 0
a, - - 2
I
ua,
.I
Sa,
>Q
a, .r
-0
I I I 'cn
m m
m m
- >;
- 3 a,
m 0 0
m S
U 0
0
0
m cn m Q
m a, cn S
0
3
0
>r 0
a, a, a rc
L
L
c,
.- L
-
.- z - I
.- L
c,
-
m
-c-r .- a- -
.- m a, U
-c-r
m a x
0
U
S
m U
a, N
c
m 0
L
3
.- P ti a a, 0 0
Q
L
I
a, L
U
c,
F
c,
ma
,
I"5
m
uj - m cn
m Q
Q
m
.- L
I
F L 0 S rc
.- 8 +
S
0
m N
.- c, .I
5 - 0 m m z
0
a, -c S
cn 0
a,
Q
a, >
0
>
S
L
-c,
.- c,
'01 L
-
-
m
u
a, rc 3
cn a, 0
0
m Q
0
k
0
a, Q
cn m S
a, L
a, S
-
- .I
rc
.- .I
c,
L
.- 3
E til .- - a, U 0 m c.r
.- ti) t - .- -c cn a
m cn W
.- - CI
m
0
i;=
.- 0
a, Q
cn
ir, a,
.-
a, a0
m
U S
m U
S
a, c[I - c, S
m 0 -c
I S
E a, E E 0
0
a, m cn S
-cI
L
c,
E a, U
uj a, 3
cn cn .- 06 cn cn a, m
0
Q
cn cn 3 0
cn U 0
>s z a, L
L
.- -c.r
P E E -c.r a,
E
m a, S
- -cI
cn cn a, cn cn m a, L
a, >
m I
m c,
>; - .- >
0
m 0
a, Q
>r >r 12 >r Y
a,
.I
c1
+
c,
- P S 0 cn Q
-c 0
m a, 0
r
a, U
0
5 L
+
8 8
c? m
0
hl
vi- <
m 0
<
5
:: 5 E L
i
3
-c O
a, cn S
a, cn U 0
0
m
m S
Y-
.-
d 1 i
a a
SQI Project Elements JPL
Process & Product Definition Measurement & Benchmarking Capture, define, and refine repeatable processes and a set of engineering
practices for project use
Pro vide measurement infrastructure for projects, conduct empirical analysis,
Software Technology Infusion Identify, evaluate, and support software
tools and techniques to facilitate process and product improvement
Promote communication and infuse practices into project use; provide education, training and
consulting for projects; provide SQI Project infrastructure.
1 1 /I 6/2003 PAJ- 22
Q-1 . J ,
... FsQ/'' I*.......**$ SQI Accomplishments to Date JPL
'ii SQI Project Management - SQI Project Plan - $olid budget and funding $ources - PEMs and element teams staffed - Quarterly Management Reviews (QMRs),
PEMC and SEMOG meetings - Draft CMMl OPF and OPD PA practices
51 Processes, Products & Artifacts - Software Development Require men ts - Software process hierarchy - Numerous handbooks, guides, templates,
checklists & sample docs
- Software Profile - Software Metrics program
51 Software Technology & Tools
51 Measurement
Rationale and Approach Software Project Measures Guide
- Software Tool Service - Software Tool Catalog - Software Tool Survey
- Two internal CMMl Class B Appraisals - CMMl Profile and Timeframe Appmva!
'ii CMMl Assessments
1 1 /I 6/2003
'ii Infrastructure & Operations - SQI Electronic Library (Docushare) - Action Item Tracking System (AITS) - Remedy ARS Service Group for SQI - SQI Customer Lists - SQI Intellectual Property (IP) Approach
Document Review k i Communications, Inreach & Outreach
- Draft OCM/Communications Plan - Software Website releases - SQI Brochure, Bookmark, Cubicle clip - SQI Overview presentation - SDR Awareness Briefings - Participation in NASA SWG, numerous
shared artifacts 51 Education & Training
- Software Training Plan - CMMl Org. Training PA practices - Offered numerous courses in:
Software Management Software Engineering Process Improvement (CMMI)
51 Project Support - Identified 4 Target Sectlms and software
Process Support Reps - Supported nearly 100 JPL projects with
artifacts, tools and consultingpw 23
SQI Consulting Service Areas Software Project Management
Software Project Planning Software Cost Estimation Software Acq u i si t io n Ma nag em en t Risk Management Software Project Monitor and Control Management Reviews
Software Eng i neeri ng Software Documents Software Requirements Management Software Verification Peer Reviews & Inspections
Software Support Software Quality Assurance Project Measures/Metrics Software Configuration Management SDR Conformance and Tailoring Implementing CMMl Practices
Software Technology and Tools Software Tech no logy Studies Software Tools Support
11/16/2003
SQI Services and Products Shopping List” by Process Category JPL
L L
Available SQI Products
Software Management Plan (SMP) Template Software Cost Estimation Handbook Software Supplier Agreement Management Plan Template Draft Risk Management Handbook Software “EVM Lite” -- Point Counting Methodology Software Reviews Handbook
Handbooks, Guides, Document Templates, Examples SRD Template, SW Requirements Engr: Practices & Techniques Software Stress Testing Guideline, STP Template Software Reviews Hand book, Peer Review Checklists
SQA Processes and Templates, SQA Activity Checklist Software Project Measures Guide, Defect TrackinglAnalysis Tool Software Configuration Management procedures and tools SDR Compliance Matrix, Software Process Tailoring Guide Internal evaluations and assessments
Soft;;;are Technology Repsi*cs, Tool Surv‘ey, Eicperieim Captiire Software Tool Service, Software Tool Catalog
PAJ- 24
Recommended
