Process Improvement in the Aerospace Industry CMMI and Lean Six Sigma

Rick Hefner, Ph.D. rhefner@caltech.edu

USC CS510

Agenda

• Current Challenges Facing the Aerospace Industry

• Current Industry Approaches – Capability Maturity Model Integrated – Lean Six Sigma

• Northrop Grumman Approach

2

NDIA Top 5 Systems Engineering Issues (2006)

• Key systems engineering practices known to be effective are not consistently applied across all phases of the program life cycle.

• Insufficient systems engineering is applied early in the program life cycle, compromising the foundation for initial requirements and architecture development.

• Requirements are not always well-managed, including the effective translation from capabilities statements into executable requirements to achieve successful acquisition programs.

• The quantity and quality of systems engineering expertise is insufficient to meet the demands of the government and the defense industry.

• Collaborative environments, including SE tools, are inadequate to effectively execute SE at the joint capability, system of systems (SoS), and system levels.

3 Systems Engineering Update, NDIA Top 5 Issues Workshop. July 26, 2006. Briefing by Mr. Robert Skalamera

Agenda

• Current Challenges Facing the Aerospace Industry

• Current Industry Approaches – Capability Maturity Model Integrated – Lean Six Sigma – Agile

• Northrop Grumman Approach

4

Heritage of Standards for Systems Engineering

5

EIA / IS 632

ISO/IEC 15288

Mil-Std-499B Mil-Std-

499A

1994

1994

1994

1998 2002

1974

(Not Released) Mil-Std-499

1969

(Trial Use) IEEE 1220

1998

(Full Std)

Legend Supersedes Source for

EIA 632

1998

EIA/IS 731

SE CM

IEEE 1220

(Full Std) (Interim Standard)

2002 CMMI-

SE/SW/IPPD

(Interim Standard)

(FDIS)

ISO/IEC 19760

2002

(PDTR)

2002 ISO/IEC 15504 (FDIS)

Standards for Systems Engineering, Jerry Lake, 2002

The Frameworks Quagmire Sarah A. Sheard, Software Productivity Consortium

6 http://stsc.hill.af.mil/crosstalk/1997/sep/frameworks.asp

Two Complimentary Approaches to Process Improvement

Data-Driven (e.g., Lean Six Sigma)

• Clarify what your customer wants (Voice of Customer)

– Critical to Quality (CTQs)

• Determine what your processes can do (Voice of Process)

– Statistical Process Control

• Identify and prioritize improvement opportunities

– Causal analysis of data

• Anticipate your customers/ competitors (Voice of Business)

– Design for Six Sigma

Model-Driven (e.g., CMMI)

• Determine the industry best practice – Benchmarking, models

• Compare your current practices to the model

– Appraisal, education

• Identify and prioritize improvement opportunities

– Implementation – Institutionalization

• Look for ways to optimize the processes

7

Agenda

• Current Challenges Facing the Aerospace Industry

• Current Industry Approaches – Capability Maturity Model Integrated – Lean Six Sigma

• Northrop Grumman Approach

8

What is the Capability Maturity Model Integrated?

• The CMMI is a collection of industry best-practices for engineering, services, acquisition, project management, support, and process management

– Developed under the sponsorship of DoD – Consistent with DoD and commercial standards

Three Constellations sharing common components and structure

• CMMI for Development - used by engineering organizations

• CMMI for Acquisition - used by buyers (e.g., govt. agencies)

• CMMI for Services - used by service providers (e.g., help desk)

9

Basic Building Blocks – 22 Process Areas

Project Management • Project Planning • Project

Monitoring and Control

• Supplier Agreement Management

• Integrated Project Management)

• Risk Management

• Quantitative Project Management

10

Engineering • Requirements

Development • Requirements

Management • Technical

Solution • Product

Integration • Verification • Validation

Support • Configuration

Management • Process and

Product Quality Assurance

• Measurement and Analysis

• Decision Analysis and Resolution

• Causal Analysis and Resolution

Process Management • Organizational

Process Focus • Organizational

Process Definition

• Organizational Training

• Organizational Process Performance

• Organizational Performance Management

Implemented by each project

Implemented by the organization

Expected Practices Provide Guidance for Implementation & Institutionalization

11

SG 1 Establish Estimates SP 1.1 Estimate the Scope of the Project SP 1.2 Establish Estimates of Work Product

and Task Attributes SP 1.3 Define Project Lifecycle Phases SP 1.4 Estimate Effort and Cost

SG 2 Develop a Project Plan SP 2.1 Establish the Budget and Schedule SP 2.2 Identify Project Risks SP 2.3 Plan Data Management SP 2.4 Plan the Project’s Resources SP 2.5 Plan Needed Knowledge and Skills SP 2.6 Plan Stakeholder Involvement SP 2.7 Establish the Project Plan

SG 3 Obtain Commitment to the Plan SP 3.1 Review Plans That Affect the Project SP 3.2 Reconcile Work and Resource Levels SP 3.3 Obtain Plan Commitment

GG 2 Institutionalize a Managed Process GP 2.1 Establish an Organizational Policy GP 2.2 Plan the Process GP 2.3 Provide Resources GP 2.4 Assign Responsibility GP 2.5 Train People GP 2.6 Manage Configurations GP 2.7 Identify and Involve Relevant

Stakeholders GP 2.8 Monitor and Control the Process GP 2.9 Objectively Evaluate Adherence GP 2.10 Review Status with Higher Level

Management GG 3 Institutionalize a Defined Process

GP 3.1 Establish a Defined Process GP 3.2 Collect Improvement Information

Project Planning – Implementation Project Planning - Institutionalization

Practice Ratings for the Organization/Projects

12

How is the CMMI Used for Process Improvement?

13 www.sei.cmu.edu/ideal/

IDEAL Model

Typical CMMI Benefits Cited in Literature

• Reduced costs – 33% decrease in the average

cost to fix a defect (Boeing) – 20% reduction in unit

software costs (Lockheed Martin)

• Faster Schedules – 50% reduction in release

turnaround time (Boeing) – 60% reduction in re-work

following test (Boeing)

• Greater Productivity – 25-30% increase in

productivity within 3 years (Lockheed Martin, Harris, Siemens)

• Higher Quality – 50% reduction of software

defects (Lockheed Martin)

• Customer Satisfaction – 55% increase in award fees

(Lockheed Martin) 14

Agenda

• Current Challenges Facing the Aerospace Industry

• Current Industry Approaches – Capability Maturity Model Integrated – Lean Six Sigma

• Northrop Grumman Approach

15

What is Lean Six Sigma (LSS)?

• Lean Six Sigma is a powerful approach to improving the work we do

• LSS improvement projects are performed by teams

• Teams use a set of tools and techniques to understand problems and find solutions

• Lean Six Sigma integrates tools and techniques from two proven process improvement methods

16

+

Six Sigma

• A management philosophy based on meeting business objectives by reducing variation

– A disciplined, data-driven methodology for decision making and process improvement

• To increase process performance, you have to decrease variation

17

Defects Defects

Too early Too late

Delivery Time

Reduce variation

Delivery Time

Too early Too late

Spread of variation too wide compared to

specifications

Spread of variation narrow compared to

specifications

• Greater predictability in the process

• Less waste and rework, which lowers costs

• Products and services that perform better and last longer

• Happier customers

DMAIC Roadmap

18

Define Control Analyze Improve Measure

Define project scope

Establish formal project

Identify needed data

Obtain data set

Evaluate data quality

Summarize& baseline data

Explore data

Characterize process & problem

Identify possible solutions

Implement (pilot as needed)

Define control method

Implement

Update improvement project scope & scale

Document

Select solution

Evaluate

[Hallowell-Siviy 05]

DMAIC Toolkit

19

Benchmark

Contract/Charter

Kano Model

Voice of the Customer

Voice of the Business

Quality Function Deployment

GQIM and Indicator Templates

Data Collection Methods

Measurement System Evaluation

Statistical Controls:

Control Charts

Time Series methods

Non-Statistical Controls:

Procedural adherence

Performance Mgmt

Preventive measures

Control Improve Measure Define Analyze Design of Experiments

Modeling

ANOVA

Tolerancing

Robust Design

Systems Thinking

Decision & Risk Analysis

PSM Perform Analysis Model

Cause & Effect Diagrams/ Matrix

Failure Modes & Effects Analysis

Statistical Inference

Reliability Analysis

Root Cause Analysis, including 5 Whys

Hypothesis Test

Design for Six Sigma (e.g., DMADV)

20

Define Verify Analyze Design Measure

Define project scope

Establish formal project

Identify customers

Research VOC

Benchmark

Quantify CTQs

Explore data

Design solution

Develop detailed design

Develop pilot

Evaluate pilot

Scale-up design

Predict performance

Document

Refine predicted performance

Lean

• Series of tools and techniques refined by Toyota and called the “Toyota Production System”

– Called “Lean” by Womack, Jones and Roos in The Machine That Changed the World

• Focused on increasing efficiency by eliminating non-value added process steps and wasteful practices

• Being adopted world-wide by both manufacturing and transactional based organizations

• Utilizes tools like “Value Stream Mapping,” “Just in Time” and “Kaizen”

21

LEAN FOCUS: ELIMINATE WASTE AND REDUCE CYCLE TIME

Wastes in Production

•

22

Types of

Waste

CORRECTION

WAITING

PROCESSING

MOTION

INVENTORY CONVEYANCE

OVERPRODUCTION

Repair or Rework Any wasted motion

to pick up parts or stack parts. Also wasted walking

Wasted effort to transport materials, parts, or finished goods into or out of storage, or between processes.

Producing more than is needed before it is needed

Maintaining excess inventory of raw mat’ls, parts in process, or finished goods.

Doing more work than is necessary

Any non-work time waiting for tools, supplies, parts, etc..

Organizational Adoption: Roles & Responsibilities

• Champions – Facilitate the leadership, implementation, and deployment

• Sponsors – Provide resources

• Process Owners – Responsible for the processes being improved

• Master Black Belts – Serve as mentors for Black Belts

• Black Belts – Lead major Six Sigma projects – Typically requires 4 weeks of training

• Green Belts – Lead minor Six Sigma teams, or serve on improvement teams under a Black Belt

– Typically requires 2 weeks of training

23

A Typical Lean Six Sigma Project in Aerospace

The organization notes that systems integration has been problematic on past projects (budget/schedule overruns)

A Six Sigma team is formed to scope the problem, collect data from past projects, and determine the root cause(s)

The team’s analysis of the historical data indicates that ineffective peer reviews are leaving significant errors to be found in test

Procedures and criteria for better peer reviews are written, using best practices from past projects

A pilot project uses the new peer review procedures and criteria, and collects data to verify they solve the problem

The organization’s standard process and training is modified to incorporate the procedures and criteria, to prevent similar problems on future projects

24

Agenda

• Current Challenges Facing the Aerospace Industry

• Current Industry Approaches – Capability Maturity Model Integrated – Lean Six Sigma

• Northrop Grumman Approach

25

Northrop Grumman Approach: Mission Success Requires Multiple Approaches

26

Process Effectiveness

Program Effectiveness

Mission Assurance & Enterprise Excellence

Operations Effectiveness

Dashboards for Enterprise-Wide

Measurement

Communications & Best-Practice Sharing

Robust Governance Model (Policies, Processes,

Procedures)

Risk Management

Systems Engineering

Independent Reviews & Cost Estimates

Training, Tools, & Templates

CMMI Level 5 for Software, Systems, and

Services

ISO 9001 and AS-9100 Certification

Six Sigma

Organizational Infrastructure Required for CMMI Level 3

27

Policies, Processes, Templates & Tools

Best-Practice Libraries

Process Group

Audits & Appraisals Measurement Repositories Predictive Modeling

Developing and maintaining mature processes requires significant time and investment in infrastructure

Process Improvement Training Program

Communications

0

5

10

15

20

25

1 11 21 31 41 51 61 71

UCL

_X

Def

ects

per

com

pone

nt

Component #

0

5

10

15

20

25

1 11 21 31 41 51 61 71

UCL

_X

Def

ects

per

com

pone

nt

Component #

Northrop Grumman Approach: Institutionalizing Our Improvements

28

Internal Best Practices

ISO/AS9100 Findings

CMMI Appraisal Findings

Policy

Procedures

Process

eToolkit PAL

Workbench StartIt! My MS Portal

Tools

Information

Checklists and Guides

Templates and Examples

Disposition

Independent Audits

• Systems/ Software Engineering Process Group

• QMS Working Group

• Program Management Advisory Board Lessons Learned

& Metrics

Analysis

Customer Comments

Configuration Control Board

External Best Practices

Industry Standards

Six Sigma Projects

We systematically analyze quality and process data and trends to determine

how to improve our processes

We improve our process assets based on internal and external best

practices

Deployed to

programs

Increasing program efficiency

msCAS

PCDB

Northrop Grumman Approach: Lessons Learned

• Multiple improvement initiatives helps encourage a change in behavior as opposed to “achieving a level”

– Reinforces that change (improvement) is a way of life

• Benefits results from institutionalizing local improvements across the wider organization

– CMMI establishes the needed mechanisms

• CMMI and Lean Six Sigma compliment each other – CMMI can yield behaviors without benefits – Lean Six Sigma improvements based solely on data may miss innovative

improvements (assumes a local optimum)

• Training over half the staff as Lean Six Sigma Green Belts has resulted in a change of language and culture

– Voice of Customer, data-driven decisions, causal analysis, etc. – Better to understand/use tools in everyday work than to adopt the “religion”

29