Root CauseAnalysis

Copyright 2003 -- Raytheon Company,

Experiences inRoot Cause Analysis and

Defect Prevention MethodsKelly L. Lanier

RaytheonNetwork Centric Systems

[email protected]


Case for Action

Output of a R6SProject

Characterization

Deliver Quality and Reduce Costly Rework!

Six Sigma Specialist Project thatfocused in reducing escaping defects.Design defects that escape toIntegration are 10 times moreexpensive than if they were caughtin the Design stage.Another thing to note is that one of thebiggest problems we have found with cost andschedule is in integration.


Set the Foundation

Define and agree to the the following: Defect

A flaw or imperfection that results in incorrect software. Adefect may or may not be detected during software use.

A deficiency which has the potential of producing incorrectresponse orundesired effect.

Comment TypeAssignment, checking, performance, etc.

Comment Priority1 through 5


When to Count a Defect

Count a defect if: The requirements documentation could lead to

incorrect source codeBy being incorrectOr by leading to incorrect designEtc.

The design documentation could lead to incorrectsource codeBy being incorrect itselfOr by being easy to misunderstand

The source code is incorrect


Defect Analysis and Prevention

Defect Analysis is the process of analyzing adefect to determine its root cause.

Defect Prevention is the process of addressingroot causes of defects to prevent their futureoccurrence.


Process Focus

Defect Containmentfocus Finding defects in the

stage they wereintroduced and asearly in the lifecycle aspossible

Eliminating escapingdefects

Defect Preventionfocus Preventing the

occurrence of anindividual defect orgroup of defects

Requirements Design Code & TestSW

Integration Qual TestSYS

IntegrationPost-Release Totals

Requirements 3 3

Design 1 15 16

Code & Test 2 10 45 57SW

Integration 1 2 12 20 35

Qual Test 0 0 5 0 5 10SYS

Integration 2 3 2 2 0 2 11

Post-Release 0 1 2 0 0 0 5 8

Totals 9 31 66 22 5 2 5 140

S

t

a

g

e

D

e

t

e

c

t

e

d

Stage Originated

Defect Prevention

Defect Containment

Focus on

AND


Get DetailedDefect Datafrom DefectLogger and

Synergy

PerformCausal

Analysis

Identify RootCauses andSolutions

Next Priority

Defect Analysis andPrevention

Select Defects forFurther Analysis

ContinuousImprovement

DevelopAction Plan

andMeasurementfor Success

ImplementAction Plan

SubmitImprovement

toOrganization

Identify andPrioritize

ImprovementOpportunities

Select Defects

CCBs, Peer Reviews,Defect Containment

MetricPareto Charts

MonitorAgainstPlanned

Measurement

R

e

-

e

v

a

l

u

a

t

e

Defect Analysis and PreventionProcess



The program metrics analysis team regularlyreviews defect data to determine if defectanalysis is necessary Defect metric has exceeded threshold Defect data shows a trend Individual defect is flagged for analysis


Select Defects


MetricPareto Charts


Select Defects


MetricPareto Charts


Analysis Tools

FishboneFishboneDefect Containment MatrixDefect Containment Matrix

Five WhysFive Whys

ParetoPareto

Requirements Design Code & TestSW

Integration Qual TestSYS

IntegrationPost-Release Totals

Requirements 3 3

Design 1 15 16

Code & Test 2 10 45 57SW

Integration 1 2 12 20 35

Qual Test 0 0 5 0 5 10SYS

Integration 2 3 2 2 0 2 11

Post-Release 0 1 2 0 0 0 5 8

Totals 9 31 66 22 5 2 5 140

S

t

a

g

e

D

e

t

e

c

t

e

d

Stage Originated

? ? ? ? ?

http://homext.ray.com/sixsigmahttp://homext.ray.com/sixsigma

G e t D e t a i l e dD e f e c t D a t a

f r o m D e f e c tL o g g e r a n d

S y n e r g y

P e r f o r mC a u s a l

A n a l y s i s

I d e n t i f y R o o tC a u s e s a n d

S o l u t i o n s

D e f e c t A n a l y s i s a n dP r e v e n t i o n


ContinuousImprovement (1 of 3)

Identify and prioritize improvementopportunities based on the potentialsolutions to defect root cause

Submit improvement opportunities to theorganization via the OrganizationalImprovement Website

Next PriorityContinuousImprovement

DevelopAction Plan



SubmitImprovement

toOrganization




Measurement


Organizational Improvement Website A proposal is a request to have the SWEC

organization evaluate, select, and adopt arecommended improvement and to provide a fundingand evaluation path for the proposal.

An advisory is an informational message to the SWECorganization advising that a program is providing afunding and evaluation path for an improved process,method, tool, technology, etc to support its ownbusiness/project goals. However, this improvementmay have relevance to the organization at large andhave strategic importance to the enterprise



DevelopAction Plan



SubmitImprovement

toOrganization




Measurement


Work as a Six Sigma Project Create an action plan for the improvement

Include how to measure success of theimprovement

Implement the action plan Monitor the progress of the action plan

Track progress for measurementof success

Communicate the results to theprogram & organization



DevelopAction Plan



SubmitImprovement

toOrganization




Measurement


Defect Prevention

Examples New checklists Modified processes Updated plans Scheduled training Tool support


DevelopAction Plan



SubmitImprovement

toOrganization




Measurement


Defect PreventionNext PriorityContinuousImprovementDevelop

Action Planand

Measurementfor Success


SubmitImprovement

toOrganization




Measurement

Check to mark a defect

Check to analyze


Organization Defect Analysis andPrevention Process

Level 5 Behaviors

Get DetailedProgram

Defect Data

PerformCausal

Analysis

Identify RootCauses andSolutions

Next Priority

Defect Analysis andPrevention


ContinuousImprovement

ExperimentInstitutionalize



SubmitImprovement

Proposal

Select Defects

Out of Threshold OrgDefect Metric, Quarterly

Process CapabilityAnalysis

Monitor

R

e

-

e

v

a

l

u

a

t

e

Program Defect Analysis and Prevention


Results

SWEC Defect Containment Trend

81%84%

50%

60%

70%

80%

90%

100%

Feb

01M

ar01

Apr 0

1M

ay01

Jul 0

1Se

p01

Dec

01

1Q02

2Q02

3Q02

4Q02

1Q03

2Q03

3Q03

4Q03

Upper DROV Nominal value Lower DROV Mean Performance


Defect Density

Improved Defect Density by 44 percentage points, and reduced variation by 31%

SWEC Defect Density TrendFe

b01

Mar

01Ap

r 01

May

01Ju

l 01

Sep

01D

ec01

1Q02

2Q02

3Q02

4Q02

1Q03

2Q03

3Q03

4Q03

Upper DROV Nominal value Lower DROV Mean Performance

CMM Level 4June 2001

CMMI Level 5September 2003


Summary

Out of phase defects cause expensive rework Use common definitions and counting approach Focus on defect detection and prevention Analyze metrics at least monthly using R6 tools Prioritize and implement defect prevention

activities Promote lessons learned to the organization

Experiences in Root Cause Analy...Case for ActionSet the FoundationDefine and agree to the the foll...DefectA flaw or imperfection that resu...A deficiency which has the poten...Comment TypeAssignment, checking, performanc...Comment Priority1 through 5

When to Count a DefectCount a defect if:The requirements documentation c...By being incorrectOr by leading to incorrect desig...Etc.The design documentation could l...By being incorrect itselfOr by being easy to misunderstan...The source code is incorrect

Defect Analysis and PreventionDefect Analysis is the process o...Defect Prevention is the process...

Process FocusDefect Containment focusFinding defects in the stage the...Eliminating escaping defectsDefect Prevention focusPreventing the occurrence of an ...

Defect Analysis and Preventio...Select Defects for Further Anal...The program metrics analysis tea...Defect metric has exceeded thres...Defect data shows a trendIndividual defect is flagged for...

Analysis ToolsContinuous Improvement (1 of 3)Identify and prioritize improvem...Submit improvement opportunities...

Continuous Improvement (2 of 3)Organizational Improvement Websi...A proposal is a request to have ...An advisory is an informational ...

Continuous Improvement (3 of 3)Work as a Six Sigma ProjectCreate an action plan for the im...Include how to measure success o...Implement the action planMonitor the progress of the acti...Track progress for measurementCommunicate the results to the

Defect PreventionExamplesNew checklistsModified processesUpdated plansScheduled trainingTool support

Defect PreventionOrganization Defect Analysis and...ResultsSlide 17SummaryOut of phase defects cause expen...Use common definitions and count...Focus on defect detection and pr...Analyze metrics at least monthly...Prioritize and implement defect ...Promote lessons learned to the o...

Documents

Root CauseAnalysis