Upload
tabitha-robinson
View
226
Download
7
Tags:
Embed Size (px)
Citation preview
MDA Journey to Mission Success
Randolph StoneDirector-Quality, Safety & Mission
AssuranceMissile Defense Agency
March 3, 2008
Approved for Public Release08-MDA-3305 (29 FEB 08)
2
Approved for Public Release08-MDA-3305 (29 FEB 08)
Agenda
• Reasons for Quality
• Ground-Based Midcourse Defense System (GMD)- History- Major Problems- Impact- Systems Engineering Improvements- Cradle-to-Grave Approach- Lessons Learned- GM Successes
• Kinetic Energy Interceptor (KI)- Award Fee- Knowledge Points- MDA Assurance Provisions
• Reasons for Quality
• Ground-Based Midcourse Defense System (GMD)- History- Major Problems- Impact- Systems Engineering Improvements- Cradle-to-Grave Approach- Lessons Learned- GM Successes
• Kinetic Energy Interceptor (KI)- Award Fee- Knowledge Points- MDA Assurance Provisions
3
Approved for Public Release08-MDA-3305 (29 FEB 08)
Reasons for Quality
4
Approved for Public Release08-MDA-3305 (29 FEB 08)
Great Prophet 2
5
Approved for Public Release08-MDA-3305 (29 FEB 08)
Taepo-Dong-1
6
Approved for Public Release08-MDA-3305 (29 FEB 08)
Ground-Based Midcourse Defense
Ground-Based Interceptor Emplacement
Sea-Based X Band Radar
7
Approved for Public Release08-MDA-3305 (29 FEB 08)
GM History
• Program Description- National Missile Defense (NMD) Program began in 1996 under BMDO program
- Realigned as the Ground-Based Midcourse Defense System (GMD) under MDA in 2002– A key component of the nation’s multi-layered Ballistic Missile Defense System
– The GMD includes sensors, command systems, fire control systems, and Ground Based Interceptors (GBI)
- In spiral development – 3 phases simultaneously– Design and Development, Operational, Support
8
Approved for Public Release08-MDA-3305 (29 FEB 08)
GM Major Problems
• Directed to produce and field while still in the development & test phase - Rapid deployment – enormous pressure on PD to meet program schedule
- Led to high-risk decisions regarding schedule, product quality, and program cost
• Acquisition reform – removed Mil Stds, relied on industry “best practices”- Traceability and sub-contractor flow-down- High percentage of issues developed with sub-contractors– Unauthorized changes, poor communication, lack of prime oversight
9
Approved for Public Release08-MDA-3305 (29 FEB 08)
“Why Not Fly?”
GM Major Problems
• EKV team focused on technical aspects of design instead of producability of end item- No production control, constant engineering changes
- Schedule slips, nonconformances and quality escapes
• Commercial off the Shelf (COTS) - No formal qualification in design and development for COTS flight computer
- Computer reset – Root Cause unknown for over a year
- COTS became logistically expensive; would have been easier to build
“Why Not Fly?”
10
Approved for Public Release08-MDA-3305 (29 FEB 08)
Impacts
• GM experienced a string of failures between 2000 and 2005- “Interceptor production slowed as the year progressed primarily because technical problems were discovered…these problems were traced back to poor oversight of subcontractors, too few qualification tests, and other quality assurance issues.” GAO Report, March 2006
GMD Test Failures
11
Approved for Public Release08-MDA-3305 (29 FEB 08)
IFT-13c
Example
Countdown automatically aborted due to a built-in-test anomaly that detected excessive drops of Electronic Control Unit (ECU) messages.
Cause: Roundoff error propagation caused a timing mismatch. When the SIGI and FC clocks crossover, timing jitter occurs and occasionally data is sent to the ECU when it is busy, resulting in no responses. To avoid a recurrence, need to raise the tolerance level for missed messages -- changing one line of computer software code.
IFT-13c – December 2004
Launch Abort
ed
12
Approved for Public Release08-MDA-3305 (29 FEB 08)These Tests Were Wake-Up Calls
IFT-14
Example
One of the last steps in the launch sequence is to open the silo doors and retract the horizontal stabilizers that hold the missile in place. Sensors indicated that one of the three lateral support group mechanisms had not retracted, causing the missile to abort launch
Cause: Corrosion and dissimilar metals (galvanic corrosion) caused the failure. One mechanism failed to latch due to wrong crush blocks
IFT-14 – February 2005
Launch Abort
ed
These Tests Were Wake-Up Calls
13
Approved for Public Release08-MDA-3305 (29 FEB 08)
MDA Systems Engineering Improvements
• MDA began to take a hard look at its quality practices- Uncovered test and development problems through audits of EKV and OBV– Example: “Functional performance test not consistently conducted during environmental testing”
- Commissioned 2005 Independent Review Team (IRT) study– Dr. William Graham, Dr. Bill Ballhaus, MG (R) Bill Nance
– Confirmed not enough Ground Testing, Deficiency in performance verification and systems engineering
– Recommended: — Strengthen systems engineering — Hold contractors accountableComplete Halt to Flight TestsThese Tests Were Wake-Up CallsComplete Halt to Flight Tests
14
Approved for Public Release08-MDA-3305 (29 FEB 08)
• MDA began to use a back-to-basics systems engineering approach- Mission Readiness Task Force (MRTF)
– Improved readiness through systematic proven processes— System Verification, Critical Design Reviews, System Function Reviews
– Improved test rigor & gained confidence in testing procedures
- Restructured Contracts & Award Fee Processes– “Paid for Failures”
- Program Management began supporting Quality– Strategic Improvements: Infusing quality in contract, empowering Mission Assurance Organization
– Tactical Improvements: Test Like You Fly, Dedicated Ground Test Missiles, End-to-End Testing
MDA Systems Engineering Improvements
These Tests Were Wake-Up Calls“Why Should It Fly?”
15
Approved for Public Release08-MDA-3305 (29 FEB 08)
CO
NT
RA
CT
DE
SIG
N
AN
DD
EV
EL
OP
ME
NT
PR
OD
UC
TIO
N
INT
EG
RA
TE
D
TE
ST
AN
D
EV
AL
UA
TIO
N
LAUNCHLAUNCH
LA
UN
CH
AC
TIV
ITIE
SAcquisition Strategy Panel
-MAP, PMAP on Contract
-Mission Assurance Criteria
-Award Fee Requirements
Peer Reviews
-Surface correctable problems early
-Identify risk
-Ensure Best Practices
System Functional Review
Software Specification Review
-Evaluate human interfaces, controls, displays
-S/W safety issues & Audits
Preliminary Design Review
Critical Design Review
-Subsystem and functional issues
-Design Performance
-Producibility and risk areas
System Verification Review
Joint Services IM Munition WG
Design Certificate Review
Safety Hazards Assessment
Development Test for Risk Reduction
-FMECA, Sneak Analysis
Continuous Activities- Mission Assurance Reps (MARs) on-site
(Gov. & Industry sites)
- Mission Assurance Audits
- Award Fee Boards
--Independent Readiness Review Teams
--Mission Readiness Reviews
Mission Assurance Audits
-Average 6 MA audits annually
Pre-Shipment Reviews
BMDS Parts & Materials Control Board (PMCB)
-Element PMP Control Boards (PMPCB)
-Voting Member
Tests and Inspections
-FOD Prevention, NDT, Acceptance Inspection
-Mandatory Gov. Inspections (MGIs)
Failure Review Board
Material Review Board
Supplier Road Maps
Stakeholder Initiatives
Integrated Product Teams (IPT)
Test Design Reviews (TDR)
-Voting Member
Mission Operations Review
Interface Control WG
-Interface integration activities
-Ensures integration and compatibility
-Resolve technical issues
Interoperability and/or Integration Test
BMDS System Safety Working Group
System Requirements Review
Pedigree Review Flight Readiness Review
Flight Operations Review -Voting Member
Pre-Flight Readiness Review
Launch Readiness Review
Mission Readiness Review
-4 to 6 weeks Prior to launch
-Voting Member
BMDS Safety Officers (BSOs)
-6 BSOs at Colo. Spgs.
MDA Executive Level Flight Test Reviews
-Series of 4 Flight test reviews leading to launch
-Voting MemberMDA Programs
DVKV KI AL IP BC AB TH GM TCSN
AF AF AF AF
AF Award Fee Board
Contract Award Fee Review - Annually
Voting Member
SN
MDA Cradle-to-Grave Approach
TEST LIKE YOU FLY
16
Approved for Public Release08-MDA-3305 (29 FEB 08)
FTG-02
17
Approved for Public Release08-MDA-3305 (29 FEB 08)
GM Lessons Learned
• Good Quality practices from the beginning will benefit in the long run
• It is hard to change once the program has started- Hard to change culture- Costly to change design
Intercept
17/17 Successful Intercepts
These Tests Were Wake-Up CallsNO INTERCEPTOR FAILURES SINCE 2005
18
Approved for Public Release08-MDA-3305 (29 FEB 08)
KEI – Kinetic Energy Interceptor
KEI Command Center
KEI Launcher
Kinetic Energy Interceptor
These Tests Were Wake-Up CallsMDA’s New Program Built with Mission
Assurance
• Mission Assurance Philosophy Built-In- Contracts & Program Culture
• In Design and Development Phase- 1st and 2nd stage static fire tests
- KEI Fire Control and Communication• First Flight Test – FY09
19
Approved for Public Release08-MDA-3305 (29 FEB 08)
Kinetic Energy Interceptor
• Contract integrated with high-level of Mission Assurance- Program Integration with Mission Assurance Government Personnel – Matrixed People from MA Organization– KIQ: QS Direct Line
- Contractors held fully accountable for mission success
- Real-time incorporation of lessons learned– Not sitting in a database– Experience from across BMDS utilized
- Early Integration Product Team (IPT) Approach with all suppliers
• Contract Structured with Award Fees
These Tests Were Wake-Up CallsGovernment Involvement from the
Beginning
20
Approved for Public Release08-MDA-3305 (29 FEB 08)
KEI Award Fee Boards
• Unique Award Fee Criteria – Based on Knowledge Points and Mission Assurance- Schedule comes 2nd
• Find problems and identify issues early and often
• Increased scrutiny on performance- No longer “paid for failures”
• Set of steps to determine accurate award fee- Performance Monitor – Including Mission Assurance Personnel
- Award Fee Review Board – Including Mission Assurance Personnel
- Award Fee Advisory Board – Including Mission Assurance Management
- Fee Determining Official – MDA Director
Mission Assurance Actions and Metrics
These Tests Were Wake-Up CallsIncentives to “Get It Right the First
Time”
21
Approved for Public Release08-MDA-3305 (29 FEB 08)
Award Fee Criteria – Knowledge Points
• MDA Director’s Knowledge points- Preplanned event that provides objective evidence for key decision making
- Each added commitment hinges on knowledge gained about critical risks
• Completed KEI Knowledge Points- Demonstrate ONIR Accuracy for Boost Phase Fire control (April 2006)
- Successful 1st and 2nd stage static firings
• Future Knowledge Points- Booster Flight Tests (2008)
Kinetic Energy Booster
22
Approved for Public Release08-MDA-3305 (29 FEB 08)
MAP Rev A
Award Fee Criteria – MDA Assurance Provisions
MAP Requirements Sections3.1 Management
3.2 Design and Development
3.3 Software and Firmware
3.4 Technical and Mission Assurance Reviews
3.5 Reliability, Maintainability, Availability
3.6 Parts and Materials Control Program
3.7 Integrated Test and Evaluation Program
3.8 Test, Measuring, and Diagnostic Equipment and Standards
3.9 Interface Management
3.10 Configuration Management
3.11 Control of Nonconforming Items and Materials
3.12 Fabrication and Quality
3.13 Supplier Management
3.14 Safety
These Tests Were Wake-Up Calls160 Pages – Over 1800 “Shall”
Statements
23
Approved for Public Release08-MDA-3305 (29 FEB 08)
Mission Assurance Toolbox
MDA Assurance Provisions
Mission Assurance Representatives
MDA Advisories
Supplier RoadmapsSupplier Roadmaps
Mission Assurance And Safety Audits
Government / Industry
Partnerships
Lessons Learned
QSMAUnfettered
Access
QSMAUnfettered
Access
CEODays
DCMA QALetter of
Delegation
DCMA QALetter of
Delegation
IndustryStakeholderInitiatives
PedigreeReviews
MDA PedigreeReview Certification
MemorandumOf
Agreement(MOA)
MemorandumOf
Agreement(MOA)
MDA
MDA Parts, Materials
AndProcesses
Plan
MDA Parts, Materials
AndProcesses
Plan
24
Approved for Public Release08-MDA-3305 (29 FEB 08)
FTG-03a
25
Approved for Public Release08-MDA-3305 (29 FEB 08)
Questions?