NASA Independent Reviews-Lessons Learned-

NNEW Data Format and Decoder Meeting

November 19, 2009

Boulder, CO

Michael Little

michael.m.little@nasa.gov

Independent Review Board

• Provides expert assessment of the  technical and programmatic approach, risk posture, and progress  against the program/project baseline. – When appropriate, it may offer recommendations to improve

performance and/or reduce risk.

– Its outputs  are briefed to the program/project under review prior to being reported to the next higher level of management.

– Required [comments] will be reconciled internally within the [review board] and with the program/ project prior to the [management] review.

• High level overview of:– the program/project's progress against the Program/Project Plan

– its readiness to proceed to the next phase

– compliance with Agency PM requirements

– the adequacy and credibility of the Integrated Baseline (at PDR and  later)

November 19, 2009 NASA Independent Review Experience Page: 2

Motivation from Experience

• Ground breaking development is hard to review objectively– Most experts are already involved in the project

– Definition of objectives sometimes hard to quantify

• Give confidence in Program Status– Higher level management

– Interfacing projects

• Identify issues created by organizational myopia– Some problems need a fresh look

– Program owners tend to be optimistic

– Some risks are understated

November 19, 2009 Page: 3NASA Independent Review Experience

Typical Topics

• Planning– Schedule

▫ Does project consistently meet milestones?▫ Is time sufficient to perform work?

– Resources▫ Planned vs. Actual?▫ Availability of adequate technical capability?

• Design– Objectives

– Requirements

– Technical Approach and Solution

– Risks

• Implementation– Test and Evaluation

– Transition or Implementation Plan

– Plan for Operations and Maintenance

November 19, 2009 NASA Independent Review Experience Page: 4

2007 GOES-R Review Plan

November 19, 2009 NASA Independent Review Experience Page: 5

Key Issues

• Who is Convening Authority?– Has management responsibility for Project funding and direction– Establish mission need and key assumptions– What decisions need to be made?– What are the right questions to be asked?

• Who has the expertise and credentials to chair/staff IRB?– Balance experience, expertise and objectivity– Coverage of key science, engineering and IT issues

• What is Project Baseline against which review should be conducted?– Objectives, mission need– Schedule and Budget

• How to keep it at a high enough level while still detecting fundamental problems?– Avoid providing project re-direction– Not a detailed design review or a code review

November 19, 2009 NASA Independent Review Experience Page: 6