Upload
nasapmc
View
13.415
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
HeadquartersIndependent Program & Cost Evaluation
State-of-the-Art in
Independent Review Execution
Tahani Amer and Kaiser Adeni Review Managers
Independent Program Assessment Office
8th Annual 2011 NASA Program Management Challenge
February 10, 2011
HeadquartersIndependent Program and Cost Evaluation
Outline
• Look Back at FY10 Reviews• Review Process Improvements• SRB Balance & Structure • SRB Members Roles & Responsibilities • SRB Coordination (between Mission Directorates,
Centers, and Programs/projects (P/p))• SRB Products• SRB – HB updates• IPAO State-of-the-Art Activities• Look Forward
2
HeadquartersIndependent Program and Cost Evaluation
Look Back at FY10 Reviews
• 8 Programs and 20 project review activities executed in FY10: Preliminary Design Review: MAVEN Critical Design Reviews: GRAIL, RBSP, GPM, TDRS, LDCM, MMS,
OCO-II System Integration Reviews: MSL, GRAIL and Juno ORR/FRR/PLAR: WISE Program Approval Review: RPS and LQP Program Implementation Review: SCaN, ESMP, D/NF Special Reviews: MSL, CxP, Aquarius, SOFIA 8 Non-IPAO reviews: 2- GOES-R, ASP, AvSP, ATP, FAP, ENAS, CAS
3
HeadquartersIndependent Program and Cost Evaluation
FY10 Completed Reviews
4
HeadquartersIndependent Program and Cost Evaluation
FY10 Completed Reviews
5
HeadquartersIndependent Program and Cost Evaluation
FY10 IPAO Completed ReviewsIPAO supported 8 non-IPAO review activities in FY10
6
HeadquartersIndependent Program and Cost Evaluation
Review Process Improvements
• SRB process improvements: • Quick Look Reports (one-pagers);• 30-day reporting requirement;• Increased coordination for programmatic assessments;• Readiness-to-proceed assessments;• Alternate opinions for non-consensus boards;• Key Decision Point (KDP) Decision Memo improvements;• Deferral of program reviews approved at APMC;• Institutionalized electronic signature of SRB approval letters;• Streamlining of ToR content as defined in draft NPR 7120.5E;
7
HeadquartersIndependent Program and Cost Evaluation
Review Process Improvements
• SRB process improvements (continued): • Better coordination of SRB team nomination process; early
stakeholders involvement in the process to ensure proper team balance of competency/currency/independence;
• Final reports are posted on APMC electronic repository (https://nx.arc.nasa.gov/nx/dsweb/View/Collection-93608); notification at APMC when reports are posted;
• Strengthening and improving the rigor and integration of the technical and programmatic assessments;
• Working with SMD and the Centers to formulate principles on conducting joint Program/project reviews when there is a significant external partner involved. Efforts are evolving w/ExoMars and JPSS.
8
HeadquartersIndependent Program and Cost Evaluation
SRB Balance & Structure
• SRB members should be competent, current, and independent from the management or advocacy chain of the P/p, with membership balanced between the host Center and other organizations to ensure the needs of the convening authorities are met.
• Although balance of each SRB member is important, ultimately the goal is to have the SRB, as a whole, balanced.
• More inclusive set of discussion with Center and TA • It is not a numerical formula, but it goals to meet
Agency’s goals
9
HeadquartersIndependent Program and Cost Evaluation
Competency Relevant experience and expertise with the technical, specific
technologies and programmatic domain of the P/p under review. Currency
Addresses recent/current experience and expertise in programmatics or the technical domain(s) of the project or program under review.
Understanding of current Agency governance, project management and systems engineering policies, procedures and methods, specifically NPR 7120.5, NPR 7123.1, NPD 1000.5, specific Center practices and procedures and the SRB Handbook.
Independence Not in the programmatic chain of command of the program or
project and have no conflicts of interest either personally, institutionally or organizationally.
10
SRB Balance & Structure
HeadquartersIndependent Program and Cost Evaluation
SRB Members Roles & Responsibilities
• A member of an SRB is an “agent” of the convening authorities: More emphasis on programmatic risk assessment and analysis;
More emphasis on the individual member independent report (IMIR);
Advise the SRB Chair and the Review Manager (RM) on areas that require particular attention by the SRB per their area of expertise;
Support the SRB “Quick-Look Report” summaries and briefings to the Convening Authorities (CA), and provide expert opinion to SRB preliminary and final reports.
11
HeadquartersIndependent Program and Cost Evaluation
SRB Coordination
• Coordination with the Mission Directorates: Early and frequent coordination is conducted with the Mission Directorate Program
Executive (PE) to: Establish SRB membership
Chair with support from RM establishes the discipline areas to be covered PE/MDs provide a list of potential candidates. CAs approve or provide alternate
nominations RM leads the vetting effort for Organizational and Personal Conflict of Interest
(OCI/PCI) in coordination with the LaRC Legal Office and Contracting Officer(s) Finalize Terms of Reference (ToR) content
Content is reviewed and agreed-to prior to final approval by the CA Identify any additional Mission Directorate review criteria
Establish timing of the Site Visit Establish post-review out brief schedule
CMC/DPMC briefing APMC briefing (Cat 1 and some Cat 2 Programs/projects) APMC special topics for Cat 2 project life cycle reviews
Pause and Learn (PAL)
12
HeadquartersIndependent Program and Cost Evaluation
SRB Coordination
• Coordination with the Center Coordination with the Center is conducted to:
Coordinate and develop a list of SRB candidates from the Center based on the discipline areas established by the Chair
Facilitate the review of the ToR content and ensure that the Center specific requirements are being met
Golden Rules JPL Rules
Facilitate Center CA approvals of ToRs, team nominations, etc. Establish dates for post-review CMC briefing
Establish a Community of Practice (CoP) at each Center (in progress) Knowledge sharing of best practices and expectations for independent assessment Review manifest coordination and approval Identify Center personnel to participate in SRBs Assist with nomination of high potential candidates for RM assignment with IPAO Request potential Center Review Managers as detailees to IPAO Develop a CoP for the Agency after completion on the discussion with Centers
13
HeadquartersIndependent Program and Cost Evaluation
SRB Coordination
• Coordination with Program/project: Program/project pre-planning coordination begins ~120 days prior to Site Visit to:
Establish Site Visit review requirements Discipline areas to be covered
Data drop timeline for cost and schedule products Establish agreement on Terms of Reference (ToR) review criteria
Single ToR for LCR (0.5E) Readiness to Proceed Review Establish timing of the Site Visit Establish post-review out brief schedule
Quick- Look Report (One Pagers) P/p briefing CMC briefing DPMC briefing APMC briefing (Cat 1 and some Cat 2 P/p)
14
HeadquartersIndependent Program and Cost Evaluation
SRB Products
SRB products produced for each Site Visit: Briefings
One-pager (Quick Look Report) P/p Briefing CMC Briefing DPMC APMC Briefing (if a Cat 1 Project)
ToRs, team nomination letters, alternate opinionsVetting Package
OCI/PCI mitigation plans/annual vetting SRB member resumes/bios SRB approval letters
Reports Final SRB summary report to include each SRB individual reports
to DA. https://nx.arc.nasa.gov/nx/dsweb/View/Collection-93608 IPAO Review Record RRD for the Agency
15
HeadquartersIndependent Program and Cost Evaluation
SRB –HB Update
Guidance to the P/p and SRB members
The SRB-HB is posted on the http://nodis.hq.nasa.gov/policy_letters/NM_7120-81_.pdf
Updates to comply with NPR 7120.5E
16
HeadquartersIndependent Program and Cost Evaluation
IPAO State-of-the-Art Activities
17
• JCL assessments and improved programmatic analysis: Using Defense Contract Management Agency (DCMA) and Government
Accounting Office (GAO) criterion as standard evaluation criteria for schedules
Implementation of standard analysis timeline allowing for time to work with the projects to improve cost and schedule concerns
Analysis processes have been defined and documented in Standard Operating Procedure Instructions (SOPI)
• Better risk assessments: Using state-of-the-art tools for integrated risk analysis Establishment of Schedule Analysis Working Group (SAWG) to develop a
CoP for programmatic risk analysis
• Forensics Study: Analysis of the SRB findings Identify trends and systematic issues Provide recommendation to improve PM at NASA
HeadquartersIndependent Program and Cost Evaluation
IPAO State-of-the-Art Activities
• Lessons Learned (LL) after each review IPAO members share the LL with RMs and PAG Analysts Database of LL
• ILCR - Customer/CA/SRB member surveys Developed ILCR surveys & Approved by CA Implementing the survey: RBSP
• Training: RMs & Chairs• Outreach Effort: Articles, PM Track, Visits to Centers, PAL
• Developmental Program: Detail opportunity to support Agency Project Management
18
HeadquartersIndependent Program and Cost Evaluation
Look Forward
• Implementation of the NPR 7120.5E• Update the SRB Hand Book• Continue coordination with MDs, Centers, and P/p• Communities of Practice• SRB Balance• Independent Reviews involving external partners• Independent Reviews and Technology P/p
19
HeadquartersIndependent Program and Cost Evaluation
Back-up Slides
20
HeadquartersIndependent Program and Cost Evaluation
One Step PDRLife Cycle Review Overview
Page 21
KDP-C KDP-C
PDR Readiness Assessment PDR-LCR
Technical Baseline with C/S/R and Integrated Assessment of Technical and Programmatic
Baseline
KDP-B KDP-B
Periodic SRB Involvement as Appropriate
P/pBrief
CenterBrief
MDBrief
Not To Scale
(30 Days)
Quick Look Report
(30- 90 days)
-Required prior to LCR-Report to DA for life cycle reviews preceding KDP B&C and during any major replan or rebaseline (3)
Programmatic Data Drops to SRB (includes JCL Model)
Deliveries start at 100 days before site review
(2)
FOOTNOTES:1. A One Step Review may be used for any LCR.2. Appendix I provides information on the readiness assessment, quick-look reports and checkpoints associated with life cycle reviews.3. For all other life cycle reviews, report to Chief Engineer if significant unresolvable disagreements.
(2)
CheckPoint if needed.(2)
21
HeadquartersIndependent Program and Cost Evaluation
Two Step PDRLife Cycle Review Overview
Page 22
Resolve Tech Issues/Risks, Update
Cost/Schedule Baseline
Independent Integrated PDR Assessment
PDR Readiness Assessment PDR
-Required prior to LCR-Report to DA for life cycle reviews preceding KDP B&C and during any major replan or rebaseline (3)
Technical Baseline with Cost,
Schedule, and Risk Information
Integrated Assessment of Technical and Programmatic
Baseline
KDP-B KDP-B
Periodic SRB Involvement as Appropriate
P/pBrief
CenterBrief
MDBrief
Not To Scale
(1-6 months) (30 Days)
Quick Look Report
(30-90 days)
PDR LCRKDP-C KDP-C
Quick Look Report
Programmatic Data Drops to SRB (includes JCL Model)
(2)
(2)(2)
FOOTNOTES:1. A Two Step Review may be used for any LCR2. Appendix I provides information on the readiness assessment, quick-look reports and checkpoints associated with life cycle reviews3. For all other life cycle reviews report to Chief Engineer if significant unresolvable disagreements
CheckPoint if needed.(2)
22