PDR Risk Checklist - · XLS file · Web viewc. Has the risk assessment addressed future risks to detailed design, developmental test, operational test, training, ... LORA and testability

PDR Page 1 of 87

“Systems Engineering for Mission Success”

Preliminary Design Review Name of the program being reviewed / date

Name / Code / Technical Specialty of reviewer

Risk Character

Legend: R Y G U NA Item Comments / Mitigation

0 0 0 0 0

1

programmatic

1.a

Program Risk Assessment Checklist (28 June 2007 version)

OVERVIEW: Although the checklist can be printed and completed as a "hard copy", it is designed to be completed electronically as an Excel spreadsheet. When viewed electronically, the small number buttons in the upper left corner of the screen are used to select the level of indenture for the questions in the checklist. A left mouse click on a number button will expand or collapse the entire checklist to the desired level. A left click on the "+" symbol in the left margin of the spreadsheet will expand the level of indenture for that section. A left click on the "-" symbol in the left margin of the spreadsheet will collapse the level of indenture for that section. The buttons in Row 11 run specific macros. The buttons in Column A allow a user to designate and sort specific questions as "Special Interest" (i.e., High Priority, Flagged, Question). The colored buttons in Row 11, Column C allow the user to sort questions by Technical Discipline, to provide a Level 1 roll-up of the risk characters assigned, or to hide specific information. For example selecting the "Logistics" button results in the display of all Level 1 Logistics-related questions and assigned information. All other questions will be hidden.COMPLETING THE CHECKLIST:1. In the upper right corner of the checklist, enter the name of the program being reviewed, the date(s) of the review, along with the name, code and technical specialty of the person(s) completing the checklist.2. A "Risk Character" (i.e., R / Y / G / U / NA) should be assigned for each question by direct entry or left clicking in each box to activate the "drop down" menu. To delete a "Risk Character" from a box, click in the box and press the "Delete" button on the keyboard, or right click on the cell and select"clear contents". The assigned Risk Characters will automatically total and display in the Level 1 (and Level 2, as applicable) row(s). Selection of a summary tab (Excel "Sheet") at the bottom of the checklist will provide a summary of all questions assigned a particular risk character (e.g., selecting the RED tab will display all questions assigned a RED risk character). 3. Any question requiring further attention (Special Interest) should be similarly marked in Column A as "High Priority", "Flagged", or "Question" to facilitate follow-up.4. Narrative, amplifying, and / or mitigation information should be entered in the "Comments Mitigation" box (Column J) at the right of each question.

SAVING THE CHECKLIST: Save the completed checklist in a new file with a unique name such as "UAV PDR 21June07ajo".

R = Red, Y = Yellow, G = Green, U = Unknown / Unavailable, NA = Not ApplicableSpecialInterest

Technical Discipline

Level 1, software, T&E, logistics, HSI, training, risk, programmatic

1. Timing / Entry Criteria

a. Has a System Functional Review (SFR) been successfully completed?

CAUTION: Entries, changes, deletions or comments should only be made on the checklist. Any entries entered directly on the summary pages will not be recorded within the checklist and will disable linkage between the checklist and the summary pages.

Question

Flagged

High Priority

HSI

Interoperability

Logistics Training

PQMProgrammatic

RiskSoftware T&E

Technology

Sh o w

All

Hide TD Unhide TD Unhide NAHide NA

Le vel 1

EVM

Use this text box to filter the "Comments/ Mitigation" column

Search

PDR Page 2 of 87

Risk Character




1.b

1.c

1.d

software, T&E, logistics, HSI, training, risk, programmatic

b. Is the program ready to conduct a Preliminary Design Review (PDR) based upon satisfying PDR entry criteria vice a pre-determined schedule date?


c. Have all prior technical review Requests for Action (RFAs) been properly dispositioned, and closed?

software, logistics, programmatic

d. Is the program using an effective Integrated Data Environment (IDE) to store data?

PDR Page 3 of 87

Risk Character




0 0 0 0 0

2

programmatic

2.a

2.b

2.c

2.d

2.e

programmatic

2.f

2.g

2.h

2.i

Level 1, software, T&E, PQM, training, risk, logistics, HSI, programmatic, interoperability

2. Planning

a. Was a systems engineering knowledgeable chairperson assigned?

software, T&E, training, HSI, logistics, programmatic

b. Did the review agenda address all applicable PDR review entry criteria?

software, T&E, training, HSI, logistics, risk, programmatic

c. Is the technical review Board properly staffed, and are the appropriate competencies participating in the review?


d. Has a Systems Engineering Plan (SEP) been developed and implemented?


e. Was a Manpower Estimate Report (MER) completed and approved?

f. Was the Acquisition Strategy (AS) developed and documented?

PQM, HSI, logistics, programmatic

g. Does the AS address a plan to satisfy Human Systems Integration (HSI) requirements for each domain addressed in the Capability Development Document (CDD) / Capability Production Document (CPD), including minimum standards for those domains not specifically addressed in the CDD / CPD?

T&E, programmatic, interoperability

h. Are key Government / contractor interfaces identified for the Test and Evaluation (T&E) program, and does planning reflect Integrated Test Team (ITT) organization and testing (contractor / Developmental Test (DT) / Operational Test (OT))?

HSI, software, logistics, T&E training, risk, programmatic

i. Is adequate staffing (required expertise and quantity of expertise for both the contractor and the Government) available to execute the test schedule?

PDR Page 4 of 87

Risk Character




0 0 0 0 0

2.j

2.j(1)

2.j(2)

2.j(3)

2.j(4)

2.j(5)

2.j(6)

2.k

2.l

2.m

2.n

2.o

2.p

software, T&E, HSI, logistics, training, risk, programmatic, interoperability

j. Test Planning

T&E, programmatic

(1) Have developmental test plans been formulated in accordance with the Test and Evaluation Master Plan (TEMP)?

T&E, programmatic

(2) Does the T&E strategy meet the TEMP requirements?

T&E, programmatic

(3) Has detailed test planning been initiated?

T&E, programmatic

(4) Are test requirements tied to verification requirements, and is there a method to ensure traceability of test requirements to the verification requirements?


(5) Does the TEMP reflect Net-Centric Operations and Warfare requirements?

software, T&E, HSI, logistics, training, risk, programmatic

(6) Does the TEMP address metrics and test procedures to ensure that human integration requirements for each domain are delivered and satisfy the CDD / CPD requirements?


k. Is there a clear understanding of the deficiency documentation process, and is there plan for a deficiency documentation and tracking system?

T&E, HSI, logistics, programmatic

l. Has a draft flight clearance process been established?

T&E, software, logistics, HSI, programmatic

m. Have metrics been established to track the test program?

T&E, programmatic

n. Have facilities / test resources (contractor and Government) been defined and included in the planning?

T&E, programmatic

o. Is there user acceptance to the test planning, and are there provisions for user participation?

T&E, programmatic

p. Has OT been involved with all aspects of test planning, and are OT requirements considered as a part of DT planning?

PDR Page 5 of 87

Risk Character




2.q

2.r

2.s

2.t

2.u

2.v

0 0 0 0 0

2.w

2.w(1)

2.w(2)

2.x

T&E, training, programmatic

q. Are training requirements documented for Developmental Test and Evaluation (DT&E) and Operational Test and Evaluation (OT&E)?

T&E, programmatic

r. Are Systems Engineering (SE) requirements for the T&E program understood, and is testing for unique SE included in the test plans?

T&E, programmatic

s. Are Government and contractor T&E facility requirements solidified, and will facilities be available to meet the schedule?

software, programmatic

t. Were the software metrics and those provided to the program office to manage the software program provided to the software Subject Matter Expert (SME)?

T&E, risk, programmatic, interoperability

u. Has the necessary System of Systems / Family of Systems (SoS / FoS) testing been addressed and / or planned?

programmatic, interoperability

v. Has system data been entered / updated in the Net-Centric Operations and Warfare implementation baseline database?

software, T&E, logistics, HSI, training, risk, programmatic, interoperability

w. Net-Ready Key Performance Parameter (NR-KPP)


(1) Has the updated system and Information Support Plan (ISP) been certified or approved?


(2) Have the updated architecture products (Core Architecture Data Model (CADM) compliant) been delivered?


x. Have the changes required to Doctrine, Organization, Training, Materiel, Leadership and Education, Personnel and Facilities (DOTMLPF) as a result of the fielding of this system been appropriately addressed in order to advance joint war fighting capabilities?

PDR Page 6 of 87

Risk Character




0 0 0 0 0

3

0 0 0 0 0

3.a

3.a(1)

3.a(2)

3.a(3)

3.a(4)

programmatic

3.a(5)

3.a(6)

programmatic3.b

programmatic3.c

0 0 0 0 0

3.d

3.d(1)

3.d(2)

Level 1, software, T&E, training, logistics, risk, technology, HSI, programmatic, interoperability

3. Program Schedule

software, risk, logistics, HSI, training, T&E, programmatic, interoperability

a. Schedule Planning

software, logistics, HSI, training, T&E, programmatic

(1) Does the program have an updated schedule with sufficient detail to support development, and are the tasks linked?


(2) Has the software schedule been updated based upon the preliminary software design?


(3) Were any problems that caused schedule slips identified as risks prior to their occurrence, and was their associated mitigation plan implemented?


(4) Are the SEP project schedule and software schedule consistent with one another?

(5) Does the project schedule accommodate establishment and / or updates to required process documentation?

software, logistics, HSI, training, T&E, programmatic

(6) Does the project schedule accommodate the establishment of an effective configuration / data management system for the preliminary design artifacts?

b. Is the schedule built upon “bottom up” task planning?

c. Is the schedule reflective of available resources?

software, risk, T&E, technology, programmatic

d. Program Critical Path

risk, technology, programmatic

(1) Does the program schedule have an identified critical path, and is that critical path consistent with overall technical risk?


(2) Are any components of the software on the program's critical path, and what are those components?

PDR Page 7 of 87

Risk Character




3.d(3)

3.d(4)

3.d(5)

programmatic3.e

3.f

3.g

3.h

3.i

software, T&E, programmatic

(3) Have Government Off-the-Shelf (GOTS) / Commercial Off-the-Shelf (COTS) software procurement and test been considered in the critical path calculation?


(4) Has the development of support software (test benches, simulations, automated test software, etc.) been considered in the critical path calculation?


(5) Are there any software deliverables from outside sources (COTS, GOTS, reusable components) on the software development critical path?

e. What is the status versus critical path?

T&E, programmatic

f. Does the test and evaluation program have an updated schedule with sufficient detail to support development, and are the tasks linked?

T&E, programmatic

g. Is the test and evaluation schedule built upon “bottom up” task planning?

T&E, programmatic

h. Is the T&E schedule executable with respect to timeframe and required resources (manpower, ranges, facilities)?

risk, programmatic, interoperability

i. Is this system on the critical path for delivery of a SoS / FoS capability, and have the risks to that critical path been identified and mitigated?

PDR Page 8 of 87

Risk Character




0 0 0 0 0

4

0 0 0 0 0

4.a

4.a(1)

4.a(2)

0 0 0 0 0

4.b

4.b(1)

4.b(2)

4.b(3)

4.b(4)

4.b(5)

4.b(6)

Level 1, EVM, logistics, PQM, software, HSI, training, T&E, technology, risk, programmatic, interoperability

4. Management Metrics

risk, logistics, software, HSI, training, T&E, programmatic, interoperability

a. Cost / Schedule / Performance / Key Performance Parameters (KPP)


(1) Is the latest revised estimate of each KPP in accordance with the Acquisition Program Baseline (APB)?


(2) Are the KPPs reflective of program risks and technical results?

risk, logistics, software, T&E, technology, programmatic

b. Latest Cost Estimate


(1) Is the cost estimate consistent with the technical risk of the program, the critical path plan, and the available resources?


(2) Has the software estimate been updated based upon the preliminary software design?


(3) Has the software estimate been updated based upon actual measured project software development performance and productivity to date?

logistics, software, HSI, programmatic

(4) Has the cost of acquiring, licensing and configuring COTS and / or GOTS computer hardware and software been considered?


(5) Has COTS and / or GOTS computer hardware and software obsolescence and upgrade impacts been considered as part of the estimate?

logistics, T&E, software, HSI, programmatic

(6) Has the development of support software (test benches, simulations, automated test software, test scripts etc.) been considered as part of the estimate?

PDR Page 9 of 87

Risk Character




0 0 0 0 04.c

4.c(1)

4.c(2)

0 0 0 0 04.d

4.d(1)

4.d(2)

4.e

4.f

4.g

0 0 0 0 04.h

4.h(1)

4.h(2)

4.h(3)

4.h(4)

4.h(5)

PQM, programmatic

c. Production Costs Estimates

PQM, programmatic

(1) Is the estimate for production costs consistent with the preliminary design as disclosed?

PQM, programmatic

(2) Are all elements of production cost addressed?


d. Operating and Support (O&S) Costs Estimate


(1) Is the estimate for O&S costs consistent with the preliminary design as disclosed?


(2) Are all elements of O&S cost addressed?

logistics, HSI, programmatic

e. Have supportability analysis products from the system integration work effort been made available to the cognizant PDR participants prior to the review?


f. Are current logistics documents available for review (Product Support Plan (PSP), Logistics Requirement Funding Summary (LRFS), preliminary maintenance plan)?


g. Have all prior logistics review RFAs been properly dispositioned, and closed?

risk, EVM, technology, programmatic

h. Earned Value Management (EVM)

EVM, programmatic

(1) Are the EVM data up to date?

EVM, programmatic

(2) Is the EVM baseline being used as a program execution tool (i.e. by management and at the working level)?

EVM, programmatic

(3) Are the work packages based on earned value vice level of effort?

risk, EVM, technology, programmatic

(4) Are the EVM data consistent with known technical risks and challenges in the program?

EVM, technology, programmatic

(5) Are the EVM data being used to adjust program resources to address risk issues?

PDR Page 10 of 87

Risk Character




4.h(6)

0 0 0 0 0

4.i

4.i(1)

4.i(2)

4.i(3)

4.i(4)

0 0 0 0 04.j

4.j(1)

4.j(2)

4.j(3)

4.j(4)

4.j(5)

4.j(6)

EVM, programmatic

(6) Have the metrics to track EVM been clearly articulated and have sufficient fidelity to understand the status of the product development?

logistics, HSI, software, risk, technology, programmatic

i. Work Breakdown Structure (WBS) review

technology, risk, programmatic

(1) Is the WBS consistent with the technical risks of the program?

technology, risk, programmatic

(2) Is the WBS decomposed to an appropriately detailed level to address all technical tasks?

logistics, HSI, software, risk, technology, programmatic

(3) Are all Configuration Items (CIs) (as identified in the preliminary design) addressed in the WBS (including software)?


(4) Are the requirements tracked, traced, and modeled using an automated tool?

logistics, HSI, software, risk, programmatic

j. Software Metrics


(1) Has a software metrics (status versus plan) program been implemented?


(2) Are adequate software metrics in place and being used to manage the software effort?


(3) Do the metrics indicate status versus plan, and what level of risk do the metrics indicate?

software, risk, programmatic

(4) Is the software staffing adequate for the magnitude / complexity of the software and the level of software risk?


(5) Are the software sizing metrics adequate and consistent with the preliminary design, and do they indicate readiness for detailed design?


(6) Are computer resource utilization metrics known and allocated to CI?

PDR Page 11 of 87

Risk Character




4.j(7)

4.j(8)

4.j(9)

4.j(10)

4.j(11)

4.j(12)

4.j(13)


(7) Are other software complexity metrics being used and do these metrics indicate adequate understanding of complexity versus resources (schedule, funding, staffing) available to ensure detailed design success?


(8) Does the Statement of Work (SOW) require the contractor to define, establish, and operate a metrics data collection, analysis, and reporting system that provides quantitative information on key software program management issues?


(9) Does the software plan include metrics to track and manage the software requirements changes, deletions and additions (software requirements volatility) for each CI?


(10) Is there a metrics process in place to track implementation of software requirements in accordance with the program cost and schedule plan?


(11) Is there a metrics process in place to insure that quality is designed and built into the software, and what metrics have been used to track quality during the software requirements phase?


(12) Are appropriate metrics in place to allow the tracking, management, and mitigation of significant software risks?


(13) Have Software Resource Data Reports (SRDR) been submitted in accordance with DoD Instruction 5000.2 and DoD 5000.4-M-2 for ACAT IA, IC, ID with a software development effort exceeding $25M (FY02 dollars)?

PDR Page 12 of 87

Risk Character




0 0 0 0 0

5

5.a

5.b

5.c

5.d

Level 1, HSI, software, T&E, logistics, training, technology, risk, programmatic, interoperability

5. Program Staffing

HSI, logistics, software, risk, training, T&E, technology, programmatic

a. Is there a complete organization structure shown, and is the organization consistent with the technical challenges / risks of the program?

HSI, software, T&E, logistics, training, risk, technology, programmatic, interoperability

b. Are key Government / contractor interfaces identified and / or represented at the PDR, and are these consistent with program risks including key external interoperability interfaces for other Joint programs?

HSI, software, training, T&E, logistics, risk, programmatic

c. Is adequate staffing (required expertise and quantity of expertise for both the contractor and the Government) available to execute the schedule?

T&E, programmatic

d. Is there confidence that all required flight clearance performance monitors are involved and do they concur with the preliminary design?

PDR Page 13 of 87

Risk Character




0 0 0 0 0

6

0 0 0 0 06.a

6.a(1)

6.a(2)

6.a(3)

programmatic

6.a(4)

0 0 0 0 0

6.b

6.b(1)

6.b(2)

6.b(3)

6.b(4)

6.b(5)

Level 1, software, logistics, PQM, training, HSI, risk, technology, T&E, programmatic, interoperability

6. Process Review


a. Program Management


(1) Are the program management processes, in the Program Management Plan (PMP), in place adequate to address the technical challenges of the program and adequate to address program risks?


(2) Is there an updated PMP that is reflective of the emergent technical issues and risks?


(3) Are there program management processes in place to properly manage the detailed design and attendant technical emphasis areas?

(4) Is the program being managed to adjust resources to address issues in the preliminary design?

logistics, software, HSI, programmatic, interoperability

b. Configuration Management (CM)


(1) Is the CM plan in place and up to date?


(2) Are the CM decisions based on factors that best support implementation of performance-based strategies throughout the product life cycle?


(3) Is the preliminary design for each CI documented and being managed in accordance with the CM Plan?


(4) Are changes to the managed CI configurations controlled and tracked to higher level (system specification and CDD / CPD), and lower level (preliminary design) documents?


(5) Does the CM plan define or link to the process for management of externally controlled requirements and SoS / FoS configurations?

PDR Page 14 of 87

Risk Character




0 0 0 0 0

6.c

programmatic6.c(1)

programmatic

6.c(2)

programmatic

6.c(3)

6.c(4)

6.c(5)

6.c(6)

6.c(7)

6.c(8)

6.c(9)

0 0 0 0 0

6.d

0 0 0 0 0

6.d(1)

software, HSI, logistics, T&E, technology, programmatic, interoperability

c. Systems Engineering Processes

(1) Is there a defined SE process?

(2) Are the SE processes shared by the Government and contractor team?

(3) Are the SE processes for design development and system trades in place and being used?

technology, programmatic

(4) Are the technical reviews planned and event driven vice schedule driven?

technology, programmatic

(5) Are the SE processes adequate to support the technical requirements of the technical reviews, and are the technical teams working against a defined technical baseline?


(6) Is the program using a SE automated tool (i.e. DOORS, CORE, SLATE etc.) to manage traceability of each CI?


(7) Does the program demonstrate that it is executing a comprehensive HSI process integrated with the SE process?


(8) Are processes being established to ensure proper emphasis on identification of critical safety items?


(9) Do the test processes as detailed in the TEMP and the contractor's overarching T&E Strategy appropriately address the end-to-end testing of SoS / FoS distributed services, and are all certification test requirements identified?

HSI, risk, software, logistics, T&E, programmatic, interoperability

d. Acquisition Logistics Support Management and Staffing

HSI, risk, software, logistics, T&E, programmatic, interoperability

(1) Has the PSP been updated to reflect the maintenance and support concepts at both the system and major hardware configuration item levels?

PDR Page 15 of 87

Risk Character




6.d(1)(a)

6.d(1)(b)

6.d(1)(c)

6.d(1)(d)

6.d(1)(e)

6.d(1)(f)

6.d(1)(g)

6.d(1)(h)

6.d(1)(i)

6.d(1)(j)

6.d(1)(k)


(a) Have alternative logistics concepts been adequately considered, and preliminary cost benefit trades conducted to justify the product support strategy in the PSP?


(b) Does the PSP reflect force provider performance agreements pertaining to logistics reviews and comments concerning maintenance planning and support concepts been considered?

logistics, HSI, programmatic, interoperability

(c) Was a market analysis conducted to scope available systems and product support capabilities (public and private), and to define opportunities for achieving support objectives through design and product support strategies?


(d) Are logistics metrics identified in the APB?

logistics, T&E, software, HSI, programmatic

(e) Will support related performance and acceptance criteria be demonstrated during planned testing and / or modeling and simulation?

HSI, T&E, logistics, programmatic

(f) Are logistics parameters and tests included in the TEMP?


(g) Are Initial Operational Capability (IOC) / Full Operational Capability (FOC) dates established and defined?


(h) Are the Performance Based Logistics (PBL) strategy and its implementation reviewed throughout the life cycle to evaluate best value and performance options against cost and performance parameters?


(i) Are trade studies conducted on a continuous basis to ensure that performance and supportability goals are met?


(j) Have trade studies considered alternate operating and support concepts, with specific consideration of performance requirements?


(k) Is logistics support included as a part of the life cycle SE approach to supportability, including information interoperability requirements?

PDR Page 16 of 87

Risk Character




6.d(1)(l)

6.d(1)(m)

6.d(1)(n)

6.d(1)(o)

6.d(1)(p)

6.d(1)(q)

6.d(1)(r)

6.d(2)

0 0 0 0 0

6.e

0 0 0 0 0

6.e(1)

6.e(1)(a)

6.e(1)(b)


(l) Do logistics provider agreements and contracts contain sufficient flexibility to meet surge requirements and to re-establish organic or commercial support capability as necessary?

risk, HSI, logistics, programmatic, interoperability

(m) Has a risk management program been established that includes both Government and contractor participation and sharing of risks?

risk, HSI, logistics, programmatic

(n) Have logistics support program risks and mitigation plans been identified and assessed?

HSI, logistics, programmatic

(o) Have post IOC plans been developed for continued evolution of sustainment strategies?


(p) Are logistics and overall sustainment performance requirements stated in the CDD and CPD?


(q) Have the user logistics support summary been reviewed and coordinated with the user?

HSI, logistics, software, T&E, programmatic

(r) Has methodology been established and data collected to provide for assessment of performance?


(2) Does the Supportability Integrated Product Team (IPT) have user representation or other System Commands (SYSCOM)?

training, HSI, logistics, technology, programmatic

e. Automated Information Technology (AIT)

training, HSI, logistics, technology, programmatic

(1) Radio Frequency Identification (RFID)

logistics, HSI, technology, programmatic

(a) Does the program manager have an implementation plan and strategy for RFID on equipment containers for storage and shipment for new and mature acquisition programs?


(b) Has there been an analysis and site survey(s) conducted to ascertain the level of effort, period of implementation, and cost of RFID implementation?

PDR Page 17 of 87

Risk Character




6.e(1)(c)

6.e(1)(d)

6.e(1)(e)

0 0 0 0 06.e(2)

(a) Are UID requirements being considered? 6.e(2)(a)

0 0 0 0 0

6.e(2)(b)


(c) Has the required amount of funding to implement RFID implementation been identified, budgeted, allocated, and added to the LRFS?


(d) Do the applicable industrial partners have a plan for RFID implementation?

training, HSI, logistics, programmatic

(e) Have publications, drawings, maintenance plans, training regimens, etc. addressing RFID been updated?


(2) Unique Identification (UID)



(b) Does the program manager have an implementation plan and strategy developed for defining the specified format for UID parts marking and labeling as prescribed by the applicable Defense Federal Acquisition Regulation Supplement (DFARS) clause (see http://www.acq.osd.mil/uid)?


(I) Has a UID implementation plan been drafted as per Office of the Secretary of Defense (OSD) policy, and has it been submitted, approved, and updated?

6.e(2)(b)(I)


(II) Does the plan comply with applicable OSD guidance with for whether or not UID is being applied to items equal to or greater than $5,000, serially managed, mission essential, controlled inventory, or requiring permanent identification?

6.e(2)(b)(II)


(III) Has an analysis been conducted to ascertain the level of effort required, period of implementation, and cost of UID implementation?

6.e(2)(b)(III)


(IV) Has the required amount of funding to implement UID (on applicable components and items) been identified, budgeted, allocated, and added to the LRFS?

6.e(2)(b)(IV)


(V) Do the applicable industrial partners have a plan for UID implementation?

6.e(2)(b)(V)

PDR Page 18 of 87

Risk Character




0 0 0 0 0

6.f

6.f(1)

risk, programmatic

6.f(2)

6.f(3)

6.f(4)

6.f(5)

6.f(6)

6.f(7)

0 0 0 0 0

6.g

0 0 0 0 0

6.g(1)


(VI) Have publications, drawings, maintenance plans, training regimens, etc., addressing UID been updated?

6.e(2)(b)(VI)

risk, software, logistics, T&E, HSI, training, technology, programmatic, interoperability

f. Risk Management (RM) Processes

risk, software, logistics, HSI, training, T&E, programmatic

(1) Is there a defined risk management process, and is the Risk Management Plan (RMP) up to date and being used?

(2) Is the RM process shared by the Government and contractor team?

risk, programmatic, interoperability

(3) Are risks associated with SoS / FoS requirements integrated into the RM process, and are risks managed external to the program?


(4) Does the RM process properly track all risks on a continuous basis and provide for update of the mitigation approaches?


(5) Are mitigation approaches in place for all “yellow” and “red” risks, and are risk mitigations resourced?

risk, software, logistics, HSI, training, T&E, technology, programmatic

(6) Does the risk management process provide for risk updates to support the technical reviews and program management (acquisition) reviews?


(7) Is the system’s safety risk mitigation plan being managed by the program's Risk Management Board (RMB)?


g. Logistics Budgeting and Funding


(1) Has the program office prepared a LRFS or equivalent document, and has it been kept updated?

PDR Page 19 of 87

Risk Character




6.g(1)(a)

6.g(1)(b)

6.g(1)(c)

6.g(1)(d)

6.g(1)(e)

6.g(1)(f)

6.g(1)(g)

6.g(1)(h)

6.g(1)(i)

6.g(1)(j)

6.f(1)(k)


(a) Is there adequate documentation to support the requirements identified in the LRFS?


(b) Are logistics funding requirements developed using Cost As an Independent Variable (CAIV), accepted cost estimating methods, and risk management principles?


(c) Have life cycle cost estimates, including cost reduction efforts, been developed and validated to optimize total ownership of costs and schedules, including end of life?


(d) Does the LRFS support the budgetary requirements of the logistics support plan?


(e) Do the funding requirements in the LRFS coincide with the support requirements in the PSP and other planning documents?


(f) Are the correct appropriations identified for each logistics requirement, and have appropriate decisions been made regarding the type of funds used for procurement of PBL resources?


(g) Are funding shortfalls and impacts identified, prioritized, fully documented, and addressed to the program manager and resource sponsor?

HSI, risk, logistics, programmatic

(h) Are the impacts of funding shortfalls understood, and are plans in place to mitigate risk?


(i) Are all traditional logistics elements estimated for both initial logistics (procurement dollars) and recurring logistics (operations and maintenance funding dollars) estimated in the analysis?


(j) Has logistics planning identified impact of interoperability and data services supported by other programs?


(k) Are logistics use profiles and associated timelines, based on the system detailed design and maintenance plan, prepared and updated over the life cycle?

PDR Page 20 of 87

Risk Character




6.f(1)(l)

0 0 0 0 06.g(2)

6.g(2)(a)

6.g(2)(b)

6.g(2)(c)

0 0 0 0 0

6.h

6.h(1)

6.h(2)

(3) Has detailed test planning been initiated? 6.h(3)

6.h(4)

6.h(5)

6.h(6)

6.h(7)

6.h(8)


(l) Are there any missing data or cost elements that can improve the confidence in the completeness of the Ao analysis?


(2) Has the LRFS been staffed and approved?


(a) Are funding requirements appropriately time phased?


(b) Are funding requirements identified in the APB?


(c) Are program logistics management personnel conversant and aware of the methodologies used to develop cost estimates?

T&E, logistics, software, HSI, programmatic, interoperability

h. Test Processes (TEMP, T&E Strategy, etc.)

T&E, programmatic

(1) Have developmental test plans been formulated in accordance with the TEMP?

T&E, programmatic

(2) Does the contractor's T&E Strategy meet the TEMP requirements?

T&E, programmatic

T&E, programmatic

(4) Is there a clear understanding of the deficiency documentation process, and is there plan for a deficiency documentation and tracking system?

T&E, programmatic

(5) Are test requirements tied to verification requirements, and is there a method to ensure traceability of test requirements to the verification requirements?

T&E, software, HSI, logistics, programmatic

(6) Have metrics been established to track the test program?

T&E, software, HSI, logistics, programmatic, interoperability

(7) Does the TEMP address metrics and test procedures to ensure that human integration requirements for each domain are delivered and satisfy the CDD / CPD requirements?

T&E, logistics, programmatic, interoperability

(8) Have facilities / test resources (contractor and Government) been defined and included in the planning?

PDR Page 21 of 87

Risk Character




6.h(9)

6.h(10)

6.h(11)

0 0 0 0 06.i

6.i(1)

6.i(2)

6.i(3)

6.j

0 0 0 0 0

6.k

6.k(1)

0 0 0 0 06.k(2)

6.k(2)(a)

6.k(2)(b)

6.k(2)(c)

T&E, programmatic

(9) Is there user acceptance of the test planning, and are there provisions for user participation?

T&E, programmatic

(10) Has OT been involved with all aspects of test planning, and are OT requirements considered as a part of DT planning?

T&E, programmatic

(11) Has the draft flight clearance process been established, if appropriate?

PQM, risk, programmatic

i. Production Processes (ISO 9000, etc.)

PQM, programmatic

(1) Have production processes been considered in the preliminary design?


(2) Have production requirements been properly captured and addressed in the risk assessment?

PQM, programmatic

(3) Have long lead items been identified, and are the production processes sufficiently mature for this phase of the program?


j. Have the lessons learned by other programs been utilized to reduce risk?

risk, logistics, training, HSI, software, T&E, programmatic

k. Software


(1) Is the software development life cycle appropriate to the development, and does the software life cycle being used contribute to reducing overall software development risk?

software, training, programmatic

(2) Are software requirements allocated to COTS, GOTS and reused software appropriate?


(a) Does the COTS, GOTS and / or reused software’s implementation meet the software requirements allocated to it?


(b) Is the development team familiar with and / or trained in the use of the COTS, GOTS or reused software?


(c) Is documentation readily available?

PDR Page 22 of 87

Risk Character




6.k(2)(d)

6.k(2)(e)

0 0 0 0 0

6.k(3)

6.k(3)(a)

6.k(3)(b)

6.k(3)(c)

6.k(3)(d)

6.k(3)(e)

0 0 0 0 0

6.k(4)

6.k(4)(a)


(d) Is training readily available and has it been scheduled and budgeted for?


(e) Is the COTS, GOTS or reused software fully tested and reliable, and has adequate schedule and resources been included to test and rework it?

software, HSI, logistics, risk, programmatic

(3) Has COTS and / or GOTS obsolescence issues been considered if COTS or GOTS computer hardware and / or software is being used?


(a) Has the long term viability of the COTS and / or GOTS product provider been considered for the program life cycle?


(b) Are COTS and / or GOTS software and computer hardware upgrades caused by COTS and / or GOTS obsolescence considered for both the software development and the remainder of the software life cycle?


(c) Has the likely impact of updating a component of COTS and / or GOTS computer hardware or software been considered, and in respect to how it may force other COTS and / or GOTS upgrades?


(d) Has the impact on the project's custom software of COTS and / or GOTS computer hardware and / or software upgrades been considered?


(e) Have the impacts of COTS and / or GOTS software and computer hardware obsolescence and upgrades on the software development and integration environment been considered?

HSI, T&E, software, logistics, risk, programmatic

(4) Are facilities and resources available to support: software integration testing, formal qualification testing, systems testing, DT, and OT?


(a) Have adequate hardware, software, personnel, and spares been allocated to laboratory, ground, and flight testing to achieve the program schedule?

PDR Page 23 of 87

Risk Character




6.k(4)(b)

6.k(4)(c)

6.k(4)(d)

6.k(4)(e)

0 0 0 0 0

6.k(5)

6.k(5)(a)

6.k(5)(b)

6.k(6)

6.k(7)


(b) Does the program place an excessive and or unreasonable emphasis on ground, flight, or laboratory testing, and is the appropriate and most cost effective means of testing utilized for different testing phases?


(c) Has the impact on the testing schedule of the laboratory(s) being unavailable been considered if the laboratory(s) are also planned to be used for flight or ground testing spares?


(d) Are there any test environment resource limitations that may result in a bottleneck or chokepoint in testing, and what actions have been taken to mitigate these bottlenecks or chokepoints?


(e) Are adequate resources and schedule provided for the development and modification of any special purpose test, simulation and data analysis software for use during the software development provided?

programmatic, software, risk

(5) Is the software developer performing at a Software Capability Maturity Model (SW-CMM) or Capability Maturity Model Integration (CMMI) level III?


(a) What mitigating action is being taken to reduce the increased risk of cost, schedule, and quality deficiencies if the software developer is performing below a SW-CMM or CMMI level III?

programmatic, software, risk

(b) What corrective action is being taken, if the software developer is performing below the SW-CMM or CMMI level III that they proposed during source selection?

software, logistics, HSI, programmatic

(6) What software data rights have been procured by the Government, and are they consistent with the Government's plans for maintenance and upgrade of the software over its life cycle?


(7) Is the physical security and software security implementation consistent with the security level of the software and any data or crypto stored and managed by the software during both development and operational use?

PDR Page 24 of 87

Risk Character




6.k(8)

6.k(9)

6.k(10)

6.k(11)


(8) Are peer reviews of the software requirements and software preliminary design part of exit criteria for determining if the requirements are complete and ready to be placed under configuration control?

T&E, risk, software, logistics, HSI, programmatic

(9) Have software quality criteria for entrance into OT been identified?


(10) Does the preliminary design and project plan provide direction for the implementation of any DoD software architecture requirements and or standards?


(11) Is the software development life cycle appropriate to the development, and does the software life cycle being used contribute to reducing overall software development risk?

PDR Page 25 of 87

Risk Character




0 0 0 0 0

7

7.a

0 0 0 0 0

7.b

7.b(1)

7.b(2)

7.c

7.d

7.e

7.f

0 0 0 0 0

7.g

Level 1, HSI, T&E, logistics, training, risk, software, programmatic, interoperability

7. Requirements Management

software, HSI, logistics, T&E, training, risk, programmatic, interoperability

a. Is there a process in place for requirements management, and is it being applied to properly address this stage of the program, to include Joint, SoS and FoS requirements?

software, HSI, logistics, T&E, training, risk, programmatic

b. Are requirements being managed and traced from higher level (parent) requirements to lower level (offspring) requirements, and are there any “orphan” or “childless” requirements?


(1) Is full traceability of systems requirements allocated to software provided through software requirements, software design, interface requirements, interface design, source code, and test procedures?


(2) Are any COTS, GOTS or reused software traced to systems requirements, software requirements, interface requirements, interface design, software design, and test procedures?

risk, T&E, software, logistics, HSI, programmatic

c. Have airworthiness requirements been addressed and documented in the preliminary design?

HSI, software, logistics, T&E, programmatic

d. Is adequate requirements traceability in place to ensure compliance with the CDD / CPD at Operational Testing (OT)?


e. Are both effectiveness and suitability requirements being addressed and allocated in the preliminary design?


f. Are there plans to ensure test requirements are addressed and documented to the same level of detail as functional requirements (operation and suitability)?

HSI, software, logistics, T&E, programmatic, interoperability

g. Has an allocated baseline, or equivalent, been established, and is it complete and under configuration control?

PDR Page 26 of 87

Risk Character




7.g(1)

7.g(2)

7.g(3)

7.g(4)

7.h

7.i

7.j

7.k

7.l


(1) Are the software preliminary design documents and interface definitions complete and under configuration control?


(2) Are the driving requirements for the preliminary design complete and under configuration control?


(3) Are the interface design documents complete and under configuration control?


(4) Has the traceability of the interface design to the allocated requirements been verified by audit?

HSI, software, logistics, programmatic

h. Was a manpower estimate completed and approved?

HSI, software, logistics, training, programmatic

i. Does the PSP reflect the results of the Training Planning Process Methodology (TRPPM) analysis?

T&E, software, logistics, HSI, programmatic

j. Are there plans in place to ensure test requirements are addressed and documented to the same level of detail as functional requirements (operation and suitability)?

T&E, HSI, logistics, programmatic

k. Have test methodologies and metrics for Reliability and Maintainability (R&M) requirements been defined, and is there concurrence on the methodology / metrics from OT?

software, HSI, logistics, T&E, programmatic, interoperability

l. Are interconnectivity (data links, etc.), network centric warfare, and Sea power 21 requirements addressed and included in the preliminary design?

PDR Page 27 of 87

Risk Character




0 0 0 0 0

8

8.a

0 0 0 0 0

8.b

8.b(1)

8.b(2)

8.b(3)

8.b(4)

0 0 0 0 0

8.c

8.c(1)

8.c(2)

8.c(3)

8.c(4)

8.c(5)

Level 1, software, risk, logistics, T&E, training, HSI, programmatic, interoperability

8. Net-Centric Operations and Warfare Consolidated Compliance Checklist


a. Is the system aligned with Net-Centric Operations and Warfare functional concept?

software, risk, logistics, T&E, training, HSI, programmatic, interoperability

b. Does the preliminary design align with Net-Centric Operations and Warfare Technical Standards (Technical View (TV-1 and TV-2))?


(1) Have the system functions been mapped to the common system function list?

software, logistics, T&E, training, HSI, programmatic, interoperability

(2) Has the program defined an Open Architecture (OA) approach?

software, risk, logistics, T&E, training, HSI, programmatic, interoperability

(3) How does the system design support its Internet Protocol (IP) based and Non IP based networking requirement?

software, logistics, T&E, training, HSI, programmatic, interoperability

(4) Are the established data exchange requirements for the preliminary design consistent with DoD directed integrated architectures?

software, logistics, HSI, programmatic, interoperability

c. Net-Centric Operations and Warfare Policy Requirements

HSI, logistics, programmatic, interoperability

(1) Is the design compliant with HSI Compliance Action List (CAL)?


(2) Is the design compliant with E3 / Spectrum Supportability CAL?


(3) Is the design compliant with Information Assurance (IA) CAL?


(4) Is the design compliant with the Geospatial / Time Standards (GTS) CAL?


(5) Is the design aligned with naming and design rules?

PDR Page 28 of 87

Risk Character




0 0 0 0 0

9

programmatic

9.a

programmatic

9.b

0 0 0 0 0

9.c

9.c(1)

9.c(2)

9.c(3)

9.c(4)

0 0 0 0 0

9.c(5)

9.c(5)(a)

9.c(5)(b)

9.c(5)(c)

Level 1, T&E, HSI, PQM, software, technology, risk, logistics, training, programmatic, interoperability

9. System Preliminary Design

a. Are subsystem requirements traced to system requirements and the CDD / CPD?

b. Is the subsystem preliminary design traced to subsystem requirements?

logistics, risk, software, T&E, technology, HSI, PQM, training, programmatic, interoperability

c. System Requirement Assessment for individual CIs


(1) Have the system requirements been assessed for overall system, and each CI?


(2) Have the KPPs and other performance requirements, both explicit and derived been defined, quantified and documented?


(3) Have all functional requirements in the functional baseline been allocated to a CI, and are these documented in the preliminary design and allocated baseline?


(4) Is the “trade space” among HSI domains clearly defined, allowing economic and risk based design decisions that contribute to the blended solution?

software, HSI, logistics, T&E, programmatic, interoperability

(5) Reliability and Maintainability


(a) Have R&M, and Built-In-Test (BIT) requirements been addressed in the preliminary designs?

software, HSI, logistics, programmatic, interoperability

(b) Is the final mission profile definition complete and available?


(c) Are the R&M block diagram and math models to the Weapons Replaceable Assembly (WRA) level available?

PDR Page 29 of 87

Risk Character




9.c(5)(d)

9.c(5)(e)

9.c(5)(f)

9.c(5)(g)

9.c(5)(h)

9.c(5)(i)

9.c(5)(j)

9.c(5)(k)

9.c(5)(l)

9.c(5)(m)

9.c(5)(n)

9.c(5)(o)

0 0 0 0 0

9.c(6)

9.c(6)(a)


(d) Is a Failure Mode Effects and Criticality Analysis (FMECA) to the subsystem level available?


(e) Does the FMECA indicate that critical safety items are being identified, and that creation of a composite critical safety items list is being considered?


(f) Are preliminary R&M allocations to the WRA available?


(g) Is a preliminary reliability prediction using parts count technique to the WRA level available?


(h) Is a preliminary maintainability prediction to the WRA level available?


(i) Is a preliminary BIT assessment to the WRA level available?


(j) Are preliminary thermal, vibration, and shock analyses to the WRA level available?


(k) Is a preliminary derating analysis available?


(l) Have lessons learned been addressed?

software, HSI, logistics, programmatic

(m) Have trade studies been addressed?


(n) Have R&M risk assessment questions been addressed?

software, HSI, logistics, T&E, programmatic

(o) Have test methodologies and metrics for R&M requirements been defined, and is there concurrence on the methodology / metrics from OT?

risk, T&E, HSI, software, logistics, programmatic, interoperability

(6) Electromagnetic Environmental Effects (E3) and Spectrum Supportability


(a) Does the CPD address spectrum certification compliance, spectrum supportability, host nation approval, the control of E3, and safety issues regarding the Hazards of Electromagnetic Radiation to Ordnance (HERO)?

PDR Page 30 of 87

Risk Character




9.c(6)(b)

9.c(6)(c)

9.c(6)(d)

9.c(6)(e)

9.c(6)(f)

9.c(6)(g)

9.c(6)(h)

9.c(6)(i)

9.c(6)(j)


(b) Have the requirements been completed and submitted to for spectrum supportability approval to support a Milestone (MS) C decision?


(c) Has the system design taken into account any limitations or restrictions on RF spectrum use contained in the Military Communications-Electronics Board (MCEB) approved design guidance recommendations?

software, logistics, HSI, programmatic, interoperability

(d) Have the results of the integrated topside design analysis / study been received and incorporated into the overall acquisition strategy?


(e) Does the E3 performance specification verification matrix show all specifications and the method of verification, and is it included in the TEMP?


(f) Are all of the E3 interface specifications of Military Standard (MIL-STD-461E) and MIL-STD-464A, which includes electrical bonding, Precipitation Static (P-static), Electrostatic Discharge (ESD), subsystem Electromagnetic Interference (EMI) (including COTS and NDI), intra-system Electromagnetic Compatibility (EMC), inter-system EMC and High Intensity Radiated Fields (HIRF), lightening effects (direct and indirect), HERO, Hazards of Electromagnetic Radiation to Personnel (HERP) and Hazards of Electromagnetic Radiation to Fuel (HERF), TEMPEST, Electromagnetic Pulse (EMP) effects, and life cycle E3 hardening fully understood and being addressed?


(g) Have the conclusions and recommendations of the E3 IPT or Electromagnetic Compatibility Assessment Board (EMCAB) been incorporated into the preliminary system design and / or E3 risks appropriately addressed?


(h) Are the E3 risks shown as part of the program risk assessment plan?

HSI, risk, T&E, logistics, programmatic

(i) Does the master program schedule show the E3 development (flight worthiness) testing and EMI qualification demonstration?


(j) Does the program schedule allow adequate time to correct EMI deficiencies prior to production start?

PDR Page 31 of 87

Risk Character




9.c(6)(k)

(7) Producibility 0 0 0 0 09.c(7)

(a) Does the CPD address producibility? 9.c(7)(a)

9.c(7)(b)

9.c(7)(c)

0 0 0 0 09.c(8)

9.c(8)(a)

9.c(8)(b)

9.c(8)(c)

9.c(8)(d)

0 0 0 0 09.c(9)

9.c(9)(a)

9.c(9)(b)

9.c(9)(c)

0 0 0 0 09.c(10)

9.c(10)(a)


(k) Are E3 and spectrum management included in the SOW, Contract Data Requirements Lists (CDRLs) and Contract Line Item Numbers (CLINs) as appropriate?

risk, PQM, programmatic

PQM, programmatic

PQM, programmatic

(b) Has the system design taken into account any producibility limitations?


(c) Are the producibility risks shown as part of the program risk assessment plan?

logistics, risk, software, HSI, programmatic

(8) Human System Safety


(a) Have system safety goals been achieved or means to comply identified?


(b) Does the CPD address human system safety?


(c) Has the system design taken into account any human system safety limitations?

logistics, risk, software, HSI, programmatic

(d) Are the human system safety risks shown as part of the program risk assessment plan?


(9) Aeromechanics


(a) Does the CPD address aeromechanics?


(b) Has the system design taken into account any aeromechanics limitations?


(c) Are the aeromechanics risks shown as part of the program risk assessment plan?


(10) Structures


(a) Does the CPD address structures?

PDR Page 32 of 87

Risk Character




9.c(10)(b)

9.c(10)(c)

0 0 0 0 09.c(11)

9.c(11)(a)

9.c(11)(b)

9.c(11)(c)

0 0 0 0 09.c(12)

9.c(12)(a)

9.c(12)(b)

9.c(12)(c)

0 0 0 0 0

9.c(13)

9.c(13)(a)

9.c(13)(b)

9.c(13)(c)

9.c(13)(d)


(b) Has the system design taken into account any materials limitations?


(c) Are the structures risks shown as part of the program risk assessment plan?


(11) Materials


(a) Does the CPD address materials?


(b) Has the system design taken into account any materials limitations?


(c) Are the materials risks shown as part of the program risk assessment plan?


(12) Mass Properties


(a) Does the CPD address mass properties?


(b) Has the system design taken into account any mass properties limitations?


(c) Are the mass properties risks shown as part of the program risk assessment plan?

training, software, logistics, HSI, technology, programmatic

(13) Human Systems Integration Engineering


(a) Have the human integration design issues been addressed and implemented or alternative concepts identified?

HSI, logistics, technology, programmatic

(b) Is there a system for identifying, prioritizing and tracking HSI lessons learned from Science and Technology (S&T) applications, legacy systems, and current system IOC?


(c) Has the program adequately identified, analyzed and integrated human interface requirements?


(d) Has the program employed an HSI process in the development of preliminary design?

PDR Page 33 of 87

Risk Character




9.c(13)(e)

9.c(13)(f)

9.c(13)(g)

9.c(13)(h)

0 0 0 0 0

9.c(13)(i)

9.c(13)(j)

9.c(14)

9.c(15)

0 0 0 0 0

9.c(16)

9.c(16)(a)


(e) Does the system preliminary design meet or exceed the human systems engineering requirements appropriate to the system?


(f) Do the human interfaces minimize or eliminate systems characteristics that require excessive cognitive, physical or sensory skills?


(g) Do the program Human Machine Interface (HMI) concepts conform to Human Factors Engineering (HFE) standards in MIL-STD-1472 and ASTM 1166?


(h) Does the preliminary design adequately address aviation life support, escape and survivability requirements?

HSI, logistics, training, software, technology, programmatic

(i) Has the program developed a preliminary training systems design to maximize use of new learning techniques, modeling and simulation technology, embedded training, and instrumentation systems that provide anytime, anyplace training, and reduce the demand on the training establishment?

HSI, training, logistics, programmatic

(I) Does the training system preliminary design reflect the results of the Training System Requirements Analysis (TSRA)?

9.c(13)(i)(I)


(II) Does the preliminary design include embedded training capabilities?

9.c(13)(i)(II)


(j) Has a top down functional analyses been performed to address HSI functions?


(14) Does the system preliminary design address habitability engineering requirements appropriate to the overall system?


(15) Does the system preliminary design adequately address human survivability and safety engineering requirements appropriate to the system?


(16) Have Environmental, Safety, and Occupational Health (ESOH) requirements been implemented?


(a) Have hazard identification and reduction efforts been completed?

PDR Page 34 of 87

Risk Character




9.c(16)(b)

9.c(16)(c)

9.c(16)(d)

0 0 0 0 0

9.c(16)(e)


(b) Are there potential risks of frequent errors, acute or chronic illness, disability or injury, repetitive stress or death with this design?


(c) Has a program to eliminate ESOH hazards or manage the risk where the hazard cannot be avoided been established?


(d) Has integration of the system planning and development begun?


(e) Program Environmental, Safety and Health Evaluation (PESHE)


(I) Has a Program Environmental, Safety and Health Evaluation (PESHE) been developed that describes the strategy for integrating ESOH considerations into the systems engineering process using the methodologies in the Standard Practice for System Safety, MIL-STD 882D?

9.c(16)(e)(I)


(II) Has a PESHE been developed that describes identification of responsibilities for implementing the ESOH strategy?

9.c(16)(e)(II)


(III) Has a PESHE been developed that describes the approach to identify ESOH hazards eliminate or reduce the hazards and to implement controls for managing those ESOH risks where they cannot be avoided?

9.c(16)(e)(III)


(IV) Has a PESHE been developed that describes identification and status of ESOH risks including approval by proper authority for residual ESOH risks (based on DoD policy and MIL-STD 882D)?

9.c(16)(e)(IV)


(V) Has a PESHE been developed that describes the method for tracking progress in the management and mitigation of ESOH hazards and associated risks and for measuring the effectiveness of ESOH risk controls?

9.c(16)(e)(V)

PDR Page 35 of 87

Risk Character




0 0 0 0 09.c(17)

0 0 0 0 0

9.c(17)(a)

9.c(17)(b)

9.c(17)(c)


(VI) Has a PESHE been developed that describes a schedule for completing National Environmental Policy Act (NEPA) / Executive Order (E.O.) 12114 documentation including the approval authority of the documents as detailed in DoD?

9.c(16)(e)(VI)


(VII) Has a PESHE been developed that describes identification of all Hazardous Materials (HAZMAT) and hazardous waste associated with the system and the plan for their demilitarization / disposal?

9.c(16)(e)(VII)

HSI, risk, logistics, T&E, PQM, training, programmatic

(17) Environmental Regulations


(a) Has a Plan of Action and Milestones (POA&M) (NEPA / EO 12114 Compliance Schedule) been developed to identify significant program events to ensure NEPA or EO 12114 compliance?

HSI, logistics, T&E, PQM, programmatic

(I) Does the POA&M cover conducting test and evaluation of the system and / or subsystem?

9.c(17)(a)(I)

HSI, logistics, T&E, PQM, programmatic

(II) Does the POA&M cover contracting for production? 9.c(17)(a)(II)


(III) Does the POA&M cover planning basing, training, and home porting locations?

9.c(17)(a)(III)

HSI, PQM, logistics, programmatic

(IV) Does the POA&M cover planning new or major upgrades to facilities or supporting infrastructure to support the system?

9.c(17)(a)(IV)


(V) Does the POA&M cover demilitarization and disposal of the system?

9.c(17)(a)(V)


(b) Has NEPA decisions result in categorical exclusion finding of no significant impact based upon an environmental assessment or record of decision based upon an environmental impact statement?


(c) Do the impact assessments include the Clean Water Act and National Pollutant Discharge Elimination System Permits?

PDR Page 36 of 87

Risk Character




9.c(17)(d)

9.c(17)(e)

9.c(17)(f)

9.c(17)(g)

9.c(17)(h)

9.c(17)(i)

9.c(17)(j)

9.c(17)(k)

9.c(17)(l)

0 0 0 0 09.c(18)

9.c(18)(a)

9.c(18)(b)

9.c(18)(c)

9.c(18)(d)


(d) Do the impact assessments include the National Pollutant Discharge Elimination System Permits and Marine Mammal Protection Act?


(e) Do the impact assessments include the Clean Air Act?


(f) Do the impact assessments include the air permits?


(g) Do the impact assessments include the National Emissions Standards for Hazardous Air Pollutants?


(h) Do the impact assessments include the National Ambient Air Quality Standards?


(i) Do the impact assessments include the Resource Conservation and Recovery Act?


(j) Do the impact assessments include the Endangered Species Act?


(k) Do the impact assessments include the Marine Mammal Protection Act?


(l) Is the support system performing as expected?


(18) Safety and Health


(a) Is there a process to assure the system's noise abatement is compliant with all Federal and state standards?


(b) Is there a process to assure the system's material toxicity is compliant with all Federal and state standards?


(c) Is there a process to assure the system's personnel protective equipment is compliant with all Federal and state standards?


(d) Did the Program Manager provide a safety release(s) to developmental and operational testers prior to any test using personnel?

PDR Page 37 of 87

Risk Character




0 0 0 0 09.c(19)

9.c(19)(a)

9.c(19)(b)

9.c(19)(c)

9.c(19)(d)

9.c(19)(e)

9.c(19)(f)

9.c(19)(g)

9.c(19)(h)

0 0 0 0 09.c(20)

9.c(20)(a)

9.c(20)(b)

software, risk, logistics, HSI, programmatic

(19) System Safety


(a) Has an interaction between the system safety program and the systems engineering been established?


(b) Are system safety design requirements specified, and have legacy systems, subsystems, and components been analyzed and incorporated into the design requirements?

software, risk, logistics, HSI, programmatic

(c) Are hazard risk and assessment criteria specified for operations and support personnel, facilities, and the system?


(d) Is the hazard analysis performed during the design process to identify and categorize hazards, including hazardous materials and associated processes?


(e) Is corrective action taken to eliminate or control the hazards, or to reduce the hazard to an acceptable level?


(f) Is a closed-loop hazard tracking system implemented?


(g) Is Weapon System Explosive Safety Review Board (WSERB) approval obtained, if appropriate?


(h) Have all systems containing energetics complied with insensitive munitions criteria?

software, training, logistics, HSI, programmatic

(20) Hazardous Material Management


(a) Have hazardous materials prohibited in the system design due to operation, maintenance, and disposal costs associated with the use of such materials been identified and communicated via contracts to include sub contractors?


(b) Have hazardous materials and associated processes whose use cannot be avoided been documented and communicated to the user and support installations?

PDR Page 38 of 87

Risk Character




9.c(20)(c)

9.c(20)(d)

9.c(20)(e)

9.c(20)(f)

9.c(20)(g)

9.c(20)(h)

9.c(20)(i)

9.c(20)(j)

0 0 0 0 09.c(21)

9.c(21)(a)

9.c(21)(b)

9.c(21)(c)


(c) Has the inventory of materials incorporated into the system (including COTS and Non-Developmental Items (NDI)) during production and those materials required for maintenance also been documented and communicated to the user and support installations?


(d) Has the program planned for tracking, storing, handling and disposing of hazardous materials and hazardous waste consistent?


(e) Are hazardous material findings and determinations incorporated into the training program for all system related personnel?


(f) Does the user logistics support summary identify hazardous materials required to support the system?


(g) What efforts will be made to reduce or eliminate the use of hazardous material for the support of the system?


(h) Are material safety data sheets available for all hazardous items?


(i) Are applicable hazardous material safety procedures called out in associated Maintenance Requirement Cards (MRCs)?


(j) Have the hazardous materials required for the maintenance of the system been coordinated with facility and / or ship for inclusion in their authorized usage lists?


(21) Pollution Prevention Program


(a) Has the pollution prevention program identified impacts of the system on the environment?


(b) Has the pollution prevention program identified associated source reduction opportunities to include noise, engine emissions, and hazardous materials?


(c) Has the program planned to recycle or dispose of system replaceable and disposable components, such as metals, plastics, electronic components, oils, and coolants and refrigerants, during system life and end of service life?

PDR Page 39 of 87

Risk Character




0 0 0 0 09.c(22)

9.c(22)(a)

9.c(22)(b)

9.c(22)(c)

9.c(22)(d)

9.c(23)

9.c(24)

0 0 0 0 09.c(25)

9.c(25)(a)

9.c(25)(b)

0 0 0 0 0

9.c(25)(c)

T&E, software, programmatic

(22) Test and Evaluation Equipment

programmatic, T&E

(a) Has test unique equipment for test aircraft been identified, and is the mechanical and electrical design sufficiently mature for this phase of the program?

programmatic, T&E

(b) Has the design installation been coordinated with the appropriate aircraft design groups?

programmatic, T&E

(c) Has the data processing system been defined and scoped, and do the data processing system requirements match with the facilities requirements?


(d) Have vendors been identified for instrumentation and data processing hardware and software?


(23) How will the CM process handle changes to the preliminary design?


(24) Is the funding available to support changes to the preliminary design?

logistics, HSI, training, software, programmatic

(25) Maintenance Planning


(a) Is an on-condition status information system defined (e.g., Condition Based Maintenance Plus (CBM+)) and integrated if the Reliability Centered Maintenance (RCM) approach is implemented?


(b) What are the specific criteria for repair and maintenance for all applicable maintenance levels in terms of time, accuracy, repair levels, BIT, testability, reliability, maintainability, nuclear hardening, support equipment requirements (including automatic test equipment), manpower skills, and facility requirements for peacetime and wartime environments?


(c) What are the inter-service maintenance requirements, organic and contractor mix, projected workloads, installation requirements and time phasing for accomplishing depot maintenance requirements?


(I) Have initial estimates of depot capability / capacity and resource requirements been made and documented?

9.c(25)(c)(I)

PDR Page 40 of 87

Risk Character




9.c(25)(d)

9.c(25)(e)

9.c(25)(f)

9.b(25)(g)

9.c(25)(h)

9.c(25)(i)

9.c(25)(j)

9.c(25)(k)

9.c(25)(l)

9.c(25)(m)

9.c(25)(n)


(d) What is the maintenance approach including Level of Repair Analysis (LORA) and will include the results of the analysis be used to determine logical maintenance task intervals, grouping and packaging?


(e) What are the actions and support necessary to ensure that the system attains the specified Ao that is optimized considering RCM, CBM, time-based maintenance and Total Ownership Cost (TOC)?


(f) What are the specific maintenance tasks, including battlefield damage repair procedures, to be performed on the materiel system?


(g) What is the extent, duration, and use of interim contractor support (when applicable), and are there plans for transition to organic support?


(h) What action and support is required for materiel fielding, including environment, safety, and occupational health planning?


(i) What specifies the type of repair (e.g., inspect, repair as necessary, dispose or overhaul)?


(j) Have system maintenance task times been derived for reliability (e.g., Mean Time Between Failure (MTBF)), maintainability (e.g., Mean Time to Repair (MTTR), maintenance task times), availability (e.g., task time limits), reliability and maintainability tests, and performance monitoring / fault detection / fault isolation and diagnostics?


(k) What post production issues have been identified?


(l) Has a preliminary maintenance plan been developed?


(m) Has it been updated to reflect the results of systems engineering and supportability analysis conducted during the systems integration work effort?


(n) Do the R&M thresholds used in establishing the maintenance concept support the system availability and performance requirements in the CDD / CPD?

PDR Page 41 of 87

Risk Character




9.c(25)(o)

9.c(25)(p)

9.c(25)(q)

9.c(25)(r)

9.c(25)(s)

9.c(25)(t)

9.c(25(u)

9.c(25)(v)

9.c(25)(w)

9.c(25)(x)

0 0 0 0 09.c(26)

9.c(26)(a)

9.c(26)(b)

9.c(26)(c)


(o) Does the PSP describe the program's approach to evolving the maintenance and support concepts into an approved maintenance plan?


(p) Have funding requirements for interim support, transition planning, and establishment of organic capability been identified and documented in the LRFS?


(q) Have depot capability / capacity and resource requirements been made and documented?


(r) Is the interim depot ready to accept workload?


(s) Is the contract awarded, if this is a commercial depot?


(t) When will the depot manager certify the depot for support of the system?


(u) When will all organic depot personnel be trained and all required equipment, tools, etc., be in place to perform depot maintenance?


(v) Has a Cost of Repair Estimate (CORE) analysis been completed?


(w) Has a depot maintenance inter-service study been completed?


(x) Are teaming efforts between the depots and original equipment manufacturers being considered?


(26) Testability and Diagnostics


(a) Is the testability and BIT concept defined with the operational concept and the maintenance concept for all levels of maintenance?


(b) What design analyses (e.g., fault tree, failure modes, effects and criticality) been used to determine test point requirements and fault ambiguity group sizes?


(c) Is the LORA and testability analysis completed for each configuration item for each maintenance level to identify the optimum mix of BIT, semi-automatic test equipment and general-purpose test equipment?

PDR Page 42 of 87

Risk Character




9.c(26)(d)

9.c(26)(e)

9.c(26)(f)

9.b(26)(g)

9.c(26)(h)

9.c(26)(i)

9.c(26)(j)

9.c(26)(k)

0 0 0 0 0

9.c(27)

0 0 0 0 0

9.c(27)(a)


(d) Is Preliminary BIT and testability analysis completed by PDR?


(e) Are detailed BIT and testability analysis completed by Critical Design Review (CDR)?


(f) Is the effectiveness of BIT validated with tests?


(g) Does the failure of the BIT circuitry precipitate other hardware or software failures?


(h) Is BIT filtering applied to minimize false alarms?


(i) Are system anomalies and intermittentancies analyzed for possible changes to the BIT design, thresholds, and tolerances and / or filtering?


(j) Can BIT software be revised independently and without change to the operating software?


(k) Are BIT indications and false alarms analyzed for corrective action?

HSI, training, logistics, programmatic, interoperability

(27) Manpower, Personnel and Training (MP&T)


(a) Has a appropriate service training systems plan been developed and validated?


(I) Were the threshold requirements from the CPD / CDD used in the development of the manpower, personnel, and training requirements?

9.c(27)(a)(I)


(II) Does the explanation of manpower requirements clearly articulate qualifications and skill requirements?

9.c(27)(a)(II)


(III) Is there a Required Operational Capability (ROC) / Projected Operational Environment (POE) (or equivalent planning parameters) that addresses this system?

9.c(27)(a)(III)

PDR Page 43 of 87

Risk Character




9.c(27)(b)

9.c(27)(c)

9.c(27)(d)

9.c(27)(e)

0 0 0 0 09.c(28)

9.c(28)(a)

9.c(28)(b)

9.c(28)(c)

logistics, HSI, training, programmatic

(IV) Do the manpower requirements in the appropriate service training systems plan support the ROC / POE?

9.c(27)(a)(IV)


(V) Does the appropriate service training systems plan reflect the most current manpower requirements data available, and are all billet requirements, designators, and ratings identified in the appropriate service training systems plan?

9.c(27)(a)(V)


(VI) Do the tasks identified link to functions that are traced to Joint Mission Essential Task Lists (JMETLS)?

9.c(27)(a)(VI)


(VII) Are training course requirements identified? 9.c(27)(a)(VII)

logistics, HSI, training, T&E, programmatic

(VIII) Are training requirements documented for DT&E and OT&E?

9.c(27)(a)(VIII)


(b) Is there a clear plan on how courses and related materials and devices will be developed for training at each required level of maintenance?

HSI, training, logistics, T&E, programmatic

(c) Is there a plan for the validating and verifying training materials?


(d) Have training requirements been coordinated?

HSI, training, logistics, T&E, programmatic

(e) Does MP&T planning adequately sequence tasks and events to assure personnel are trained to operate and maintain the system during Initial Operational Test and Evaluation (IOT&E)?

training, logistics, HSI, software, programmatic

(28) Training Outline and Curricula Design


(a) Are terminal training objectives defined in detail?


(b) Are specific criteria established to determine the success of training?


(c) Are operator and maintainer training embedded in the Interactive Electronic Technical Manual (IETM)?

PDR Page 44 of 87

Risk Character




9.c(28)(d)

9.c(28)(e)

0 0 0 0 09.c(29)

9.c(29)(a)

9.c(29)(b)

9.c(29)(c)

9.c(29)(d)

9.c(29)(e)

9.c(29)(f)

9.c(29)(g)

0 0 0 0 09.c(30)

9.c(30)(a)

9.c(30)(b)

9.c(30)(c)


(d) Are job performance aids included?


(e) Have safety procedures been incorporated into training curricula?


(29) Training Material


(a) Are technical manuals developed prior to the development of training materials?


(b) Are instructor guides, course curriculum and student guides, as well as audio-visual training aids, developed for classroom training?


(c) Is software developed to disseminate computer based training?


(d) Is the training material evaluated for content, clarity and accuracy, typically in a controlled environment of a pilot course, after development?


(e) Are the training courses adequate?


(f) Do they train on the fielded configuration(s)?


(g) Are training courses conducted in a sufficient timeframe to support IOC and initial fielding?


(30) Training Devices / Simulators


(a) Are training devices to support operator or maintainer training identified?


(b) Is a military characteristics document prepared for each training device, defining its basic physical and functional requirements?


(c) Is maximum embedded on board training capability in deployed equipment used?

PDR Page 45 of 87

Risk Character




9.c(30)(d)

9.c(30)(e)

0 0 0 0 09.c(31)

9.c(31)(a)

9.c(31)(b)

9.c(31)(c)

9.c(31)(d)

0 0 0 0 09.c(32)

9.c(32)(a)

9.c(32)(b)

9.c(32)(c)

9.c(32)(d)

9.c(32)(e)


(d) Are pre-faulted modules or software to simulate faults for diagnostics training used?


(e) Are simulation of scenarios reflecting the actual operating environment used for operator training?


(31) Initial Training Requirements


(a) Is initial training provided in the operation, maintenance, or employment of a system or training aid?


(b) Are contractor test and evaluation activities used for validation of training requirements and initial user training for OT and user introduction?


(c) What are the planned Ready for Training (RFT) dates for each course?

programmatic, HSI, training, logistics

(d) Are training requirements reflected in the LRFS for course and materials development, factory training, training devices and equipment?


(32) Supply Support


(a) Has the Supply Support Management Plan (SSMP) been updated to support systems demonstration?


(b) Does the sequencing and timing of events in the SSMP logically support planned IOC / Material Support Date (MSD)?


(c) Is the provisioning technical documentation being procured adequate to support end items that have parts subject to failure / replacement and require maintenance at any level?


(d) Are accepted sparing analysis and modeling tools being utilized and are the assumptions consistent with the supportability analysis and the prescribed maintenance concept?


(e) Are supply support funding requirements reflected in the LRFS?

PDR Page 46 of 87

Risk Character




0 0 0 0 09.c(33)

9.c(33)(a)

9.c(33)(b)

9.c(33)(c)

0 0 0 0 09.c(34)

9.c(34)(a)

9.c(34)(b)

9.c(34)(c)

9.c(34)(d)

9.c(34)(e)

9.c(34)(f)

9.c(34)(g)

0 0 0 0 09.c(35)


(33) Organic Support


(a) Are Organic Support requirements and funding defined to transition from interim to organic support?


(b) Is inter-service visibility established for optimal organic support selection?


(c) Is a POA&M for organic support developed and implemented?


(34) Supply Chain Management / PBL Management


(a) Does a Business Case Analysis (BCS) support the decision that PBL is not financially viable?


(b) Is contractor support capable of integrating with the defense logistics chain, including logistics Command, Control, Communications, Computers, and Intelligence (C4I) and ebusiness routines?


(c) Has a manufacturing plan been developed and addressed?


(d) Does the short term and longer term full rate production requirements include the time phasing of all resource requirements (e.g., personnel, machines, tooling, measurement system, supply chain, etc.)?


(e) Is a defect / variation prevention program developed?


(f) Do manufacturing processes have defined yield levels and have the process been validated?


(g) Has environmental stress screening been performed to precipitate latent, intermittent or incipient defects or flaws introduced during the manufacturing process?


(35) Warranty Management

PDR Page 47 of 87

Risk Character




9.c(35)(a)

9.c(35)(b)

9.c(35)(c)

9.c(35)(d)

9.c(35)(e)

9.c(35)(f)

9.c(35)(g)

9.c(35)(h)

0 0 0 0 09.c(36)

9.c(36)(a)

9.c(36)(b)

9.c(36)(c)


(a) Are mutually beneficial warranty incentives established to facilitate long term business relationships the provider given incentive to meet specified performance measures?


(b) Is a cost benefit analysis conducted to determine the appropriateness of implementing a warranty plan?


(c) Are warranties considered and integrated in developing the program's logistics support strategy, whether PBL or traditional?


(d) Does the warranty administration and enforcement include defect reporting, analysis and corrective action processed timely and effective?


(e) Is a post award cost effectiveness assessment of the warranty plan periodically performed?


(f) Does the user logistics support summary identify warranty requirements?


(g) Have any issues with warranty administration at the organizational and intermediate levels been identified during early fielding of the system?


(h) Have necessary modifications to the warranty program been made?


(36) Support Equipment


(a) Are the support equipment strategies and diagnostics concepts defined in the preliminary maintenance plan consistent with the LORA, organic repair / contractor support and sparing strategies?


(b) Does the LRFS reflect funds needed to acquire and support, test program set development, maintenance assistance modules, test requirements documents, and metrology / calibration services?


(c) Is there a clear process by which the System Development and Demonstration (SDD) contractor will validate and demonstrate compliance with fault detection and isolation requirements?

PDR Page 48 of 87

Risk Character




9.c(36)(d)

9.c(36)(e)

9.c(36)(f)

9.c(36)(g)

0 0 0 0 0

9.c(37)

9.c(37)(a)

9.c(37)(b)

9.c(37)(c)

9.c(37)(d)


(d) Has an activity been designated to provide life cycle support, including in-service support for Training Systems Plans (TPSs) and logistics support for Peculiar Support Equipment (PSE)?


(e) Has the installation of new support equipment in maintenance ship and shore, depot and training facilities been staffed through the appropriate stakeholders?


(f) Are human engineering and user characteristics (strength, dimensions and perceptual considerations) considered in design of support equipment to ensure safety, efficiency and manpower limitations during sustainment?


(g) Are the deficiencies in the efficiency, cost, and safety of common support equipment outside the program's management authority communicated to relevant item managers?


(37) Technical Data


(a) Is a concept of operations for an Integrated Digital Data Environment (IDDE) developed, implemented, and managed throughout the system life cycle to ensure information and data interoperability with other programs and their interfacing logistics systems?


(b) Are logistics product and technical data for new systems received, managed, and stored in an IDDE to share data?


(c) Are product life cycle operations automated using an approved IDDE to improve logistics and business processes?


(d) Are computer aided design, modeling, and engineering product source data acquired in acceptable digital format per digital data policy and managed according to the IDDE CONOPS?

PDR Page 49 of 87

Risk Character




9.c(37)(e)

9.c(37)(f)

0 0 0 0 09.c(38)

9.c(38)(a)

9.c(38)(b)

9.c(38)(c)

9.c(38)(d)

9.c(38)(e)

9.c(38)(f)

0 0 0 0 0

9.c(39)

9.c(39)(a)

9.c(39)(b)


(e) Are electronic data interchange on line access and automation issues addressed starting with development of the information exchange requirements and continuing through the IDDE concept of operations?


(f) Have authoritative data sources and the associated change authority been identified?

logistics, PQM, software, HSI, programmatic

(38) Product / Technical Data Package and Publications


(a) Has a product / technical data management plan, guided by the IDDE concept of operations, including change control processes and in process reviews, been developed and validated?


(b) Has a designated Government technical data review authority been established?


(c) Has an IDE implementation plan been identified as a proposal requirement of the Request For Proposal (RFP) and / or as a contract deliverable?

logistics, PQM, software, HSI, programmatic

(d) Is there a clear plan for the integration of contractor technical information systems and processes for engineering, manufacturing, and logistics support?


(e) Is the Government authorized access to contractor databases necessary to support systems demonstration?


(f) Does the delivery schedule for the Technical Data Package (TDP) support a competitive production contract?

training, T&E, logistics, software, HSI, programmatic

(39) Computer Resources


(a) Has a computer and software security plan, including safety, been developed?


(b) Are computer and software products and technical data and the supporting infrastructure outlined through an IDDE concept of operations that supports the total life cycle management of associated product?

PDR Page 50 of 87

Risk Character




9.c(39)(c)

9.c(39)(d)

9.c(39)(e)

9.c(39)(f)

9.c(39)(g)

9.c(39)(h)

9.c(39)(i)

9.c(39)(j)

9.c(39)(k)

9.c(39)(l)

9.c(39)(m)

9.c(39)(n)

9.c(39)(o)

0 0 0 0 0

9.c(40)

9.c(40)(a)


(c) Have software functional requirements and associated interfaces been defined?


(d) Has the functional baseline for software been established?


(e) Has the gap analysis been performed on candidate COTS software to identify functionality shortfalls?


(f) Has a software configuration management plan been developed?


(g) Have measures of effectiveness for software been developed for systems demonstration?


(h) Has a Software Support Activity (SSA) been established?

training, logistics, software, HSI, programmatic

(i) Has the SSA been designated and personnel training, and facility requirements identified?

T&E, logistics, software, HSI, programmatic

(j) Have the software testing requirements been identified and integrated into the overall system test program?


(k) How does the TEMP address testing of computer hardware and software?


(l) Have requirements for system firmware and software documentation been identified and procured?


(m) Has a software development plan been developed, and does it reflect program milestones?


(n) Can and has the software maturity been measured?


(o) Have required software data rights been obtained?


(40) Facilities


(a) Have Military Construction (MILCON) requirements been identified in the LRFS?

PDR Page 51 of 87

Risk Character




0 0 0 0 0

9.c(40)(b)

9.c(40)(c)

9.c(40)(d)

9.c(40)(e)

9.c(40)(f)


(b) Have the types of facilities / infrastructure required to support and sustain the new or modified system been identified?


(I) Do the facilities / infrastructure include berthing space for ships (including utilities, dredging, special deck structural requirements for crane loads, and fendering systems)?

9.c(40)(b)(I)


(II) Do the facilities / infrastructure include parking aprons and hangar space for aircraft?

9.c(40)(b)(II)


(III) Do the facilities / infrastructure include support and maintenance facilities?

9.c(40)(b)(III)


(IV) Do the facilities / infrastructure include supply warehouses and transit sheds?

9.c(40)(b)(IV)


(V) Do the facilities / infrastructure include a dry dock capability?

9.c(40)(b)(V)


(VI) Do the facilities / infrastructure include training facilities (for both classrooms and trainers for operational training and maintenance training, including required product / technical data to ensure efficient, effective support of facilities)?

9.c(40)(b)(VI)


(VII) Transient support requirements when the system requires some level of support for continental US and outside continental US activities that are not regular homeports or support sites.

9.c(40)(b)(VII)


(c) Does the PSP include analysis conducted to determine facility requirements?


(d) Are the facilities / infrastructure support requirements usually documented in the PSP, LRFS, and / or the program’s Facilities Management Plan (FMP) or its equivalent?


(e) Is there a facilities requirements document and a schedule to conduct site surveys?


(f) Is the facilities requirement development process integrated with the Supportability Analysis process?

PDR Page 52 of 87

Risk Character




9.c(40)(g)

9.c(40)(h)

9.c(40)(i)

9.c(40)(j)

9.c(40)(k)

0 0 0 0 09.c(41)

9.c(41)(a)

9.c(41)(b)

9.c(41)(c)

9.c(41)(d)

9.c(41)(e)

9.c(41)(f)


(g) Has the program assessed Environmental documentation planning and completion in accordance with NEPA / EO 12114 for new construction or modification of existing facilities?


(h) Has the program assessed (e.g., site surveys and trade studies) all means of satisfying a facility requirement prior to selecting the use of MILCON?


(i) Has the program office identified funding to support construction alterations less than $750,000, and is a contract award in process?


(j) Has Congressional authorization and funding been approved for projects in excess of $750,000 (classified as MILCON)?


(k) Have the estimates of facility requirements and associated costs been refined, and have detailed project documentation and cost estimates been developed?


(41) Packaging, Handling, Storage and Transportation (PHS&T)


(a) Have potential PHS&T related problems been identified, and are risk mitigation plans in place?


(b) Are PHS&T plans adequate to meet statutory and regulatory requirements, if new hazardous materials are being introduced?


(c) Does the LRFS identify PHS&T funding requirements?


(d) Has DoD’s computerized Container Design Retrieval System (CDRS) database been searched to preclude the design of new specialized containers when suitable ones exist in the system?


(e) Has the military packaging, MIL-STD-2073, been considered for items that documented analyses have shown cannot be protected and preserved in a cost effective manner using commercial packaging?


(f) Has the military packaging, MIL-STD-2073, been considered for items delivered during wartime for deployment with operational units?

PDR Page 53 of 87

Risk Character




9.c(41)(g)

9.c(41)(h)

9.c(41)(i)

0 0 0 0 09.c(42)

9.c(42)(a)

9.c(42)(b)

9.c(42)(c)

9.c(42)(d)

9.c(42)(e)

9.c(42)(f)

9.c(42)(g)

0 0 0 0 09.c(43)

9.c(43)(a)


(g) Has the military packaging, MIL-STD-2073, been considered for items requiring reusable containers?


(h) Has the military packaging, MIL-STD-2073, been considered for items intended for delivery-at-sea?


(i) Has the military packaging, MIL-STD-2073, been considered for items where the contractor has determined military packaging is the optimal packaging solution?


(42) Design Interface


(a) Are the interface design documents complete and under configuration control?


(b) Have testability, maintainability and supportability requirements been defined and adequately considered in the preliminary design?


(c) Have the results of FMECA been integrated with the supportability analysis program?


(d) Are there indications that critical safety items are being identified, and is creation of a composite critical safety item list being considered?


(e) Do design processes include adherence to specific derating guidelines, particularly for electronic and electrical components?


(f) Is qualification testing planned to support design limit / life testing during system demonstration?


(g) Do the parts and material selection processes ensure items are qualified to the worst case Design Reference Mission Profile (DRMP) and design environment?


(43) Manufacturing Planning


(a) Has a manufacturing plan been developed that includes a defect variation prevention program?

PDR Page 54 of 87

Risk Character




9.c(43)(b)

9.c(43)(c)

0 0 0 0 09.c(44)

9.c(44)(a)

9.c(44)(b)

9.c(44)(c)

9.c(44)(d)

9.c(44)(e)

9.c(44)(f)

9.c(44)(g)

9.c(44)(h)

9.c(44)(i)


(b) Has a manufacturing plan been developed that includes manufacturing processes that have defined yield levels, and have been validated?


(c) Has a manufacturing plan been developed environmental stress screening to precipitate latent, intermittent or incipient defects or flaws introduced during the manufacturing process?


(44) Parts and Materials Selection


(a) Are guidance and / or requirements documented in the parts and materials design guide before the start of design, which addresses parts selection, derating and testability factors, and is adherence to the guidelines verified at design reviews?


(b) Does the order of precedence for parts selection emphasizes the use of qualified manufacturers' lists parts, particularly for applications requiring extended temperature ranges?


(c) Is a preferred parts list required prior to detailed design?


(d) Have shelf and operating life requirements been identified?


(e) Is identification of COTS / NDI reliability required?


(f) Are parts and materials selected qualified to the worst case DRMP and detail design environments?


(g) Is uprating or upscreening of parts not a best practice so they should not be performed?


(h) Is parts derating required for all electronic and electrical components, and are electrical parameters of parts characterized to requirements derived from the DRMP to ensure that all selected parts are reliable for the proposed application?


(i) Are highly integrated parts (e.g., application specific integrated circuits) used to reduce the number of individual discrete parts and chips?

PDR Page 55 of 87

Risk Character




9.c(44)(j)

9.c(44)(k)

9.c(44)(l)

9.c(44)(m)

9.c(44)(n)

9.c(44)(o)

9.c(44)(p)

9.c(44)(q)

9.c(44)(r)

0 0 0 0 09.c(45)

9.c(45)(a)

9.c(45)(b)

9.c(45)(c)


(j) Are highly integrated parts (e.g., application specific integrated circuits) used to reduce the number of interconnections?


(k) Are highly integrated parts (e.g., application specific integrated circuits) used to reduce size, power consumption, and cooling requirements?


(l) Are highly integrated parts (e.g., application specific integrated circuits) used to reduce failure rates?


(m) Has the critical items list been developed?


(n) Does the CI list include any item of high technical risk with no workaround, and items with schedule or delivery risk?


(o) Does the CI list include sole source items?


(p) Does the CI list include high failure rate items, and safety of flight items?


(q) Do COTS / NDI parts and their applications meet DRMP?


(r) Does selection of parts, maintenance process and materials consider use of the least hazardous materials and process consistent with performance, economy and life cycle costs?


(45) Commodity Management


(a) Has the program manager pursued the use of standard systems, subsystems, and support equipment against specific capabilities, technology growth, and cost effectiveness?


(b) Does the acquisition strategy identify common systems integrated into the program?


(c) Has the program manager established a process to reduce the proliferation of non-standard parts and equipment within and across system designs?

PDR Page 56 of 87

Risk Character




0 0 0 0 0

9.c(46)

9.c(46)(a)

9.c(46)(b)

9.c(46)(c)

9.c(46)(d)

0 0 0 0 0

9.c(46)(e)

9.c(46)(f)

9.c(46)(g)

9.c(46)(h)

0 0 0 0 09.c(47)

9.c(47)(a)

9.c(47)(b)

logistics, PQM, technology, HSI, programmatic, interoperability

(46) Root Cause Corrective Action


(a) Has a process been implemented to assess achieved Reliability, Maintainability, Availability (RMA) performance by collection and analysis of user data?


(b) Are system thresholds for RMA being achieved?


(c) Have logistics problems been identified using RMA data and has a POA&M been developed for corrective actions?


(d) Is a Failure Reporting Analysis and Corrective Action System (FRACAS) established and are failures analyzed and trended for Integrated Logistics Support (ILS) visibility?


(e) Is a FRACAS review performed on engineering development models, pre-production units, and production units?


(I) Are mishap reports associated with material and design deficiencies integrated into the FRACAS?

9.c(46)(e)(I)


(f) Are design review requirements including supportability, flowed to design engineering from in-service data?


(g) Do technical reviews include an assessment of system supportability requirements?


(h) Are readiness reviews performed periodically throughout the life cycle and do they include supportability factors?


(47) Obsolescence


(a) Have the technical reviews included discussions regarding the system's obsolescence?


(b) Have the various analyses and reports addressed solutions for obsolescence issues?

PDR Page 57 of 87

Risk Character




0 0 0 0 0

9.c(48)

9.c(48)(a)

9.c(48)(b)

0 0 0 0 0

9.c(49)

9.c(49)(a)

9.c(49)(b)

9.c(50)

9.c(51)

9.c(52)

0 0 0 0 09.d

9.d(1)

9.d(2)

logistics, HSI, technology, programmatic, interoperability

(48) Shipboard Interface / Integration


(a) Have the technical reviews included discussions regarding shipboard interface / integration of the system?


(b) Have the various analyses and reports addressed solutions for shipboard interface / integration issues?

logistics, software, HSI, technology, programmatic

(49) Platform Diagnostics Integration


(a) Have the technical reviews included discussions regarding platform diagnostics integration of the system?


(b) Have the various analyses and reports addressed solutions for platform diagnostics integration issues?


(50) Is the preliminary design testable?


(51) Are there plans in place to cover verification via other means as required (analysis, simulation, etc.), and is there acceptance among all stakeholders as to these approaches?


(52) Is there sufficient functional detail to enable detailed design (i.e. development of program performance specifications) from which coding can occur for computer / software CIs?


d. Are system constraints (system budgets) addressed for the overall system, and each CI?


(1) Have Physical Interface (hardware and human) requirements been considered in the preliminary design, and have proper tradeoffs been made?


(2) Is the Cost Analysis Requirements Description (CARD) consistent with the system allocated baseline and do cost estimates reflect the CARD content?

PDR Page 58 of 87

Risk Character




9.d(3)

9.d(4)

9.d(5)

9.d(6)

9.d(7)

9.d(8)

9.d(9)

9.d(10)

9.d(11)

9.d(12)

9.d(13)

9.d(14)

9.d(15)

9.d(16)

9.e


(3) Has development cost been considered in the preliminary design?


(4) Have production cost budgets been established, and have these been considered in the preliminary design?


(5) Have operations and support costs been considered in the preliminary design?


(6) Have weight budgets been establish for all CIs?


(7) Has CI weight and its impact of overall system weight been considered and properly traded?


(8) Has the volume budget and its impact of overall system weight been considered and properly traded?


(9) Has CI volume impact and its impact of overall system weight been considered and properly traded?


(10) Has power budget and its impact of overall system weight been considered and properly traded?


(11) Has CI power impact and its impact of overall system weight been considered and properly traded?


(12) Has cooling budget and its impact of overall system weight been considered and properly traded?


(13) Has CI cooling impact and its impact of overall system weight been considered and properly traded?


(14) Have the requirements for available technology insertion and system growth been allocated to the CIs and reflected in the preliminary design?


(15) Has risk been considered at the CI level?


(16) Have all requirements and systems constraints been captured in a preliminary product specification for each CI?


e. Is the preliminary design of each CI consistent with the subsystem test planning and approach?

PDR Page 59 of 87

Risk Character




0 0 0 0 09.f

9.f(1)

9.f(2)

9.f(3)

9.f(4)


f. T&E Equipment Design

programmatic, T&E

(1) Has test unique equipment for test aircraft been identified, and is the mechanical and electrical design sufficiently mature for this phase of the program?

programmatic, T&E

(2) Has the design installation been coordinated with the appropriate aircraft design groups?

programmatic, T&E

(3) Has the data processing system been defined and scoped, and do the data processing system requirements match with the facilities requirements?


(4) Have vendors been identified for instrumentation and data processing hardware and software?

PDR Page 60 of 87

Risk Character




0 0 0 0 0

10

0 0 0 0 0

10.a

10.a(1)

10.a(2)

10.a(3)

10.a(4)

10.a(5)

10.a(6)

10.a(7)

0 0 0 0 010.b

10.b(1)

0 0 0 0 0

10.c

Level 1, HSI, logistics, training, software, T&E, technology, risk, programmatic, interoperability

10. Life Cycle Logistics

HSI, logistics, training, software, T&E, technology, programmatic, interoperability

a. Maintenance Planning


(1) Is Performance Based Logistics (PBL) being considered as the preferred support strategy?


(2) Is the product support integrator PBL performance monitored / managed?


(3) Has the transition of the program's legacy systems and their existing support structures to the PBL approach progressed as planned?

HSI, logistics, T&E, training, software, programmatic, interoperability

(4) Does the PBL Performance Based Agreement (PBA) reflect the war fighter requirements and associated KPP?

HSI, logistics, training, software, T&E, technology, programmatic, interoperability

(5) Does the PBL contract include exit criteria should scenarios arise that result in cessation of the PBL contract, and does the exit criteria include drawings, technical data, Automated Test Plans (ATPs), SE, training, etc?


(6) Will PBL supportability BCAs continue throughout the life cycle process with oversight to ensure reassessment at appropriate supportability trigger points?


(7) Has a PBL strategy been developed?


b. Logistics Requirements and Funding


(1) Are the funding shortfalls to the PBA requirements and impacts identified, prioritized, fully documented and addressed to the program manager and resource sponsor?


c. Management Planning

PDR Page 61 of 87

Risk Character




10.c(1)

10.c(2)

10.c(3)

10.c(4)

10.c(5)

10.c(6)

10.c(7)

10.c(8)

10.c(9)

10.c(10)

10.c(11)

0 0 0 0 010.c(12)


(1) Has an IPT been formed to evaluate the PBL Candidate?


(2) Have all stakeholders, including Defense Logistics Agency (DLA), appropriate service inventory control point, and Foreign Military Sales (FMS), been invited to participate?


(3) Does the PBL strategy procure the desired outcomes?


(4) Does the PBL strategy incentivize performance?


(5) Does the PBL strategy clearly define performance metrics?


(6) Does the PBL strategy consider the logistics footprint?


(7) Does the PBL strategy validate the exit clauses are sufficient to ensure re-establishment of organic or commercial support capability?


(8) Does the PBL strategy include technical requirements?


(9) Does the PBL validate that support is transparent to the user?


(10) Is the PBL contract agreement structured to provide cost effective performance outcomes consistent with top level metrics, e.g. operational availability, operational reliability, cost per unit usage, logistics footprint, and logistics response time?


(11) Is planning established and implemented for the transition of the program’s legacy systems and their existing support structures to the PBL approach, including the use of a product support integrator to facilitate the transition?


(12) PBL Business Case Analysis (BCA)

PDR Page 62 of 87

Risk Character




10.c(12)(a)

10.c(12)(b)

10.c(12)(c)

10.c(12)(d)

0 0 0 0 010.c(13)

10.c(13)(a)

10.c(13)(b)

10.c(14)

10.c(15)

10.c(16)

10.c(17)

10.c(18)

10.c(19)

10.c(20)

0 0 0 0 010.d


(a) Is the PBL BCA used to support individual PBL decisions?


(b) Does the PBL BCA include the estimated costs, and does it describe the benefits between alternative product support strategies (e.g., buying a predetermined level of availability to meet war fighter's objectives)?


(c) Are BCA reviews scheduled in time to support programmatic reviews?


(d) Does the BCA support product support decision?


(13) Are the PBL product support providers identified, and are agreements finalized?


(a) Are the agreements long term, and do they include PBL management planning, identify all stakeholder roles and responsibilities, identify sources and data to collect and use, and identify review / reporting requirements and dispute resolution?


(b) Are BCAs used to support individual PBL decisions made between alternatives?


(14) Is public and private partnering optimized?


(15) Are contract clauses sufficient to meet surge requirements and exit criteria included to ensure reestablishment of organic or commercial support capability as necessary?


(16) Are systems established for data collection and for assessment of performance metrics?


(17) Has potential FMS participation been considered?


(18) Has the contract Memorandum of Agreement (MOA) / Memorandum of Understanding (MOU) been awarded?


(19) Is the PBL performance continuously assessed?


(20) Is there a data system to track PBL metrics?


d. Supply Support

PDR Page 63 of 87

Risk Character




10.d(1)

10.d(2)

10.d(3)

0 0 0 0 010.e

10.e(1)

10.e(2)

0 0 0 0 0

10.f

10.f(1)

0 0 0 0 0

10.f(2)

10.f(2)(a)

10.f(2)(b)

10.f(2)(c)


(1) Does the supportability analyses with the associated BCA assess the sparing approach (e.g., PBL or legacy support posture)?


(2) Have the PBL contractors been provided with Clearance and Access Verification System (CAVS) for electronic reporting of requisitions and asset status?


(3) Has the delivery timeline for shipment been identified?

logistics, software, HSI, interoperability

e. Data Package


(1) What data does the Government own?


(2) What ensures life of program Government access to data, including provisions for transfer of data to Government or other support agents on contract exit in PBL environment?

HSI, PQM, logistics, risk, technology, T&E, programmatic, interoperability

f. Diminishing Manufacturing Sources and Material Shortages (DMSMS)

HSI, PQM, logistics, programmatic,

(1) Has a formal DMSMS program been established and documented consistent with DoD 4140.1-R, DoD Supply Chain Material Management Regulation?

logistics, PQM, technology, HSI, programmatic,

(2) Is the DMSMS strategy integrated with the program’s technology roadmap, as well as the industry technology roadmaps for embedded microelectronics?


(a) Does the road mapping process consider the identification of critical items and technologies?


(b) Does the road mapping process consider identification of emerging technologies?


(c) Does the road mapping process consider DMSMS forecasts and impacts integrated into technology refresh / insertion planning?

PDR Page 64 of 87

Risk Character




10.f(3)

10.f(4)

10.f(5)

10.f(6)

0 0 0 0 010.f(7)

10.f(7)(a)

10.f(7)(b)

10.f(7)(c)

0 0 0 0 010.f(8)

10.f(8)(a)

10.f(8)(b)

0 0 0 0 010.f(9)

10.f(9)(a)


(3) Are the DMSMS management approach (e.g., the level of indenture) and strategy (e.g., organic, commercial, PBL, field activity managed) defined, and implemented?


(4) Are the DMSMS key activities tied to the Integrated Master Schedule (IMS) and do they identify relationships and interdependencies between tasks?


(5) Are active microelectronics managed at the piece part level unless otherwise determined by a BCA?


(6) Have DMSMS forecasting and management tools and / or service providers been researched and selected, and has the Bill of Material (BOM) been loaded into the system?


(7) Have identification and forecasting for obsolescence timelines, impact, and mitigation been conducted?


(a) Do they consider product (revisions and generation / technology changes), and supplier base?


(b) Do they consider contract period and life cycle?


(c) Is an on-going review of the parts lists and BOM to identify obsolescence or discontinuance issues conducted?


(8) Has a strategy for DMSMS design and manufacturing documentation been developed?


(a) Does the strategy consider design disclosed items, including sub-tier hardware indenture levels?


(b) Does the strategy consider form, fit, function, and proprietary design items, including sub-tier hardware indenture levels?


(9) Does the design approach minimize the impact of DMSMS by addressing open system architecture?


(a) Does the design approach minimize the impact of DMSMS by addressing the order of precedence for parts selection?

PDR Page 65 of 87

Risk Character




10.f(9)(b)

10.f(9)(c)

10.f(9)(d)

10.f(9)(e)

10.f(9)(f)

10.f(9)(g)

10.f(9)(h)

10.f(9)(i)

10.f(10)

10.f(11)

10.f(12)


(b) Does the design approach minimize the impact of DMSMS by addressing the use of qualified manufacturers lists parts, particularly for applications requiring extended temperature ranges)?


(c) Does the design approach minimize the impact of DMSMS by addressing the selection of parts relatively new in their life cycle?


(d) Does the design approach minimize the impact of DMSMS by addressing use of custom parts?


(e) Does the design approach minimize the impact of DMSMS by addressing the requirement for a preferred parts list and parts control prior to detailed design to minimize obsolescence issues?


(f) Does the design approach minimize the impact of DMSMS by addressing identification of shelf and operating life requirements?


(g) Does the design approach minimize the impact of DMSMS by addressing identification of technology life expectancies?


(h) Does the design approach minimize the impact of DMSMS by addressing association with tech refresh / block upgrade?


(i) Does the design approach minimize the impact of DMSMS by addressing design reviews to verify DMSMS approaches and solutions?


(10) Is the DMSMS Business Case Analysis (BCA) performed as part of trade studies to determine return on investment on mitigation actions and to support DMSMS decisions?


(11) Has the PBL addressed a comprehensive obsolescence and DMSMS plan?

HSI, PQM, logistics, risk, programmatic

(12) Is an obsolescence life cycle (versus contract period) mitigation strategy defined?

PDR Page 66 of 87

Risk Character




10.f(13)

0 0 0 0 010.f(14)

10.f(14)(a)

10.f(14)(b)

10.f(14)(c)

10.f(14)(d)

10.f(15)

10.f(16)

0 0 0 0 0

10.f(17)

10.f(17)(a)

10.f(17)(b)

10.f(17)(c)

logistics, PQM, technology, HSI, programmatic

(13) Are systems that utilize the same components and technologies identified, and are commodity management and preferred material processes established to standardize use of like material across programs?


(14) Funding


(a) Has DMSMS TOC and cost avoidance been estimated?


(b) Is the current and out year budget established and planned based on DMSMS forecast, tracking, and mitigation efforts?


(c) Are funding shortfalls (appropriation, amount, timing) and impact identified, prioritized, and documented?


(d) Do budget planning decisions for DMSMS reference the sponsor’s decision and are the decisions reflected in the LRFS?


(15) Has the program defined DMSMS metrics and tracks DMSMS cases, trends, and associated solutions and cost?


(16) Has an exit strategy been developed and is it contained in contractual PBL documentation that provides DMSMS configuration data access necessary to transition product support capability?

PQM, T&E, logistics, HSI, technology, programmatic, interoperability

(17) Are contractual data requirements defined?

HSI, PQM, logistics, programmatic, interoperability

(a) Do the contractual data requirements include contractor vs. Government life cycle DMSMS tasks and responsibilities?

logistics, PQM, HSI, technology, programmatic

(b) Do the contractual data requirements include DMSMS incentives and awards, decision on ownership of product and technical data package rights and COTS licensing agreements?

logistics, PQM, HSI, technology, programmatic

(c) Do the contractual data requirements include PBL / TSPR strategy for legacy system DMSMS?

PDR Page 67 of 87

Risk Character




10.f(17)(d)

10.f(17)(e)

10.f(17)(f)

10.f(17)(g)

10.f(17)(h)

10.f(17)(i)

0 0 0 0 0

10.f(17)(j)

PQM, HSI, logistics, risk, programmatic

(d) Do the contractual data requirements include DMSMS planning and mitigation requirements?


(e) Do the contractual data requirements include system architecture design to minimize obsolescence costs?


(f) Do the contractual data requirements include DMSMS production, repair, and procurement capability including hardware and software, support and test equipment, tooling and fixtures and chip and die availability and storage?

PQM, software, HSI, logistics, programmatic

(g) Do the contractual data requirements include supply chain monitoring and management including contractor and vendor notification of pending parts obsolescence and part and firmware changes?


(h) Do the contractual data requirements include configuration management to the appropriate obsolescence mitigation levels?


(i) Do the contractual data requirements include DMSMS database establishment and maintenance through an IDDE concept of operations that supports the total life cycle management of the product?

PQM, T&E, software, HSI, logistics, risk, technology, programmatic

(j) DMSMS Mitigation

PQM, software, HSI, logistics, technology, risk, programmatic

(I) Does the technical data package that supports the DMSMS mitigation strategy include specifications, technical manuals, engineering drawings, and product data models that provide appropriate level of detail for reprocurement, maintenance and manufacture of the product?

10.f(17)(j)(I)

PQM, T&E, logistics, risk, technology, HSI, programmatic

(II) Does the technical data package that supports the DMSMS mitigation strategy include special instructions for items such as unique manufacturing, quality and test processes, and preservation and packaging?

10.f(17)(j)(II)

PDR Page 68 of 87

Risk Character




10.f(17)(k)

10.f(17)(l)

10.f(18)

10.f(19)

10.f(20)

10.f(21)

10.f(22)

10.f(23)

0 0 0 0 010.g

10.g(1)

PQM, logistics, technology, HSI, programmatic

(III) Does the technical data package that supports the DMSMS mitigation strategy include Very High Speed Integrated Circuit Hardware Description Language (VHDL) documentation of digital electronic circuitry?

10.f(17)(j)(III)

PQM, risk, logistics, HSI, technology, programmatic

(IV) Does the technical data package that supports the DMSMS mitigation strategy include the version, release, change status, and other identification details of each deliverable item?

10.f(17)(j)(IV)

PQM, logistics, technology, HSI, programmatic

(k) Are the program, design, and production readiness reviews of contractor DMSMS management effectiveness?


(l) Is provisioning screening required for maximum use of existing supply items?


(18) Are the contractors’ DMSMS programs assessed to ensure that program requirements are met?

PQM, HSI, logistics, programmatic, interoperability

(19) Are the DMSMS considerations incorporated into the PSP and post production support plan?

PQM, HSI, software, logistics, risk, programmatic

(20) Are items that are single source and those for which the Government cannot obtain data rights and the associated corrective action plans identified?

logistics, PQM, technology, HSI, programmatic

(21) Are strategies to resolve potential DMSMS problems (e.g., production or repair capabilities, software upgrades and maintenance, support equipment) established, and are predictive cost effective industry solutions used to reduce DMSMS risks and enhance performance?


(22) Is a program manager reprocurement engineering support agreement in place?


(23) Does the program monitor the usage and anticipated demand vs. items available for DMSMS mitigation planning throughout the items life cycle?

software, T&E, logistics, HSI, programmatic

g. Performance Requirements


(1) What are the war fighter needs from the support system to meet sustained operational requirements?

PDR Page 69 of 87

Risk Character




10.g(2)

10.g(3)

10.g(4)

10.g(5)

10.g(6)

10.g(7)

10.g(8)

10.g(9)

0 0 0 0 0

10.h

0 0 0 0 0

10.h(1)

10.h(1)(a)


(2) Do war fighter needs address reduced footprint and TOCs as well as improved deployability and sustainability?


(3) Are war fighter needs reflected in the performance agreements, capabilities documents, and specification documents?


(4) Are performance measures / metrics (objectives and thresholds) specified to meet user oriented performance requirements (e.g., reliability, operational availability, mission capable rate, customer wait time, cycle time, footprint, cost / operating cycle, life cycle cost), and the target price for the set level of performance?


(5) Are operating and support objectives defined where feasible considering performance histories of prior systems or systems of similar capabilities?


(6) Are the performance parameters (e.g., availability, reliability, maintainability, manpower) that are included in the requirements / capabilities documents are part of design requirements for subsequent acquisition phases?


(7) Do requirements improve on logistics footprint reductions, limitations and deployment requirements compared to prior or similar systems?


(8) How do the requirements address the need to reduce multiple configurations?


(9) Are performance agreements and war fighter requirements measurable and aligned with capabilities documents?

software, T&E, logistics, HSI, technology, programmatic

h. System Development and Demonstration Phase (SDD)


(1) Key logistics criteria during SDD


(a) Do mission capabilities provide more discrete identification of the taxonomy and metrics driving performance based outcomes?

PDR Page 70 of 87

Risk Character




10.h(1)(b)

10.h(1)(c)

10.h(1)(d)

10.h(1)(e)

10.h(1)(f)

10.h(1)(g)

10.h(1)(h)

10.g(1)(i)

10.g(1)(j)


(b) Do availability requirements reflect a detailed assessment of the requirements for the system to operate successfully in the mission operational environment, and the necessary support requirements to achieve that objective?


(c) Does reliability define the logistics reliability targets and the corresponding sustainment infrastructure necessary to ensure achievement of the reliability objectives given the operational environment and combatant commander availability requirements?


(d) Does maintainability provide comprehensive identification of both projected maintenance strategy, including diagnostics, prognostics, maintenance duration targets, and similar measures?


(e) Are manpower and personnel requirements, both organic and contractor sourced?


(f) Is there continued refinement of Life Cycle Cost Estimates (LCCEs)?


(g) Is support related performance and acceptance criteria demonstrated during planned testing and through modeling and simulation?


(h) Has the collection, analysis, and evaluation of system performance and maintenance performance data determined the system's need for and the prescribe changes to the system configuration, maintenance support structure, and maintenance resource requirements?


(i) Is the utilization of on-board (embedded) monitoring sensors, diagnostics, and prognostics integral to this process?


(j) Continued inclusion of logistics support considerations in detailed design reviews to include life cycle costs, and characteristics such as openness of design, upgradeability, modularity, testability, and commercial technology insertion.

PDR Page 71 of 87

Risk Character




10.g(1)(k)

10.g(1)(l)

10.g(1)(m)

10.g(1)(n)

10.g(1)(o)

10.g(1)(p)

10.g(1)(q)

10.g(1)(r)

10.g(1)(s)

10.g(1)(t)

10.g(1)(u)

0 0 0 0 010.g(2)


(k) Has the continued inclusion of logistics support considerations in detailed design reviews included life cycle costs, and characteristics such as openness of design, upgradeability, modularity, testability, and commercial technology insertion?


(l) Has iterative refinement of logistics support considerations correspondent with the evolutionary acquisition strategy been employed?


(m) Has verification of support related design characteristics and product support strategy and infrastructure begun?


(n) Was identification of product support integrator, potential support providers (public and private), and potential partnering opportunities considered?


(o) Was the depot level maintenance core capability assessment and the identification of workloads required to sustain those capabilities conducted?


(p) Was identification of potential organic depot level sources of maintenance considered?


(q) Was a PBL BCA conducted to determine the relative cost vs. benefits of different support strategies?


(r) Was a PBL BCA conducted to determine the impact and value of performance / cost / schedule / sustainment trade-offs?


(s) Was a PBL BCA conducted to determine the data required to support and justify the PBL strategy?


(t) Was a PBL BCA conducted to determine the product support integrator performance outcomes / requirements, e.g. mission readiness, logistics footprint, response times, etc?


(u) Has development of performance based logistics product support concept to include development of war fighter and support provider agreements?


(2) Key logistics information for SDD

PDR Page 72 of 87

Risk Character




10.g(2)(a)

10.g(2)(b)

10.g(2)(c)

10.g(2)(d)

10.g(2)(e)

0 0 0 0 010.g(3)

10.g(3)(a)

10.g(3)(b)

10.g(3)(c)

10.g(3)(d)

10.g(3)(e)

10.g(3)(f)


(a) Did key logistics information for SDD include updated support strategy?


(b) Did key logistics information for SDD include key logistics parameters?


(c) Did key logistics information for SDD include sustainment funding requirements?


(d) Did key logistics information for SDD include logistics testing criteria, PBL BCA, auditable depot level maintenance core capability and workload assessment (to be completed bi-annually)?


(e) Did the key logistics information for SDD include an annual determination of the distribution of maintenance workloads?


(3) Program Logistician Responsibilities during SDD


(a) Has the program logistician responsibilities for SDD included updating support strategy within the ASR and updating logistics criteria and parameters with the ABP?


(b) Were logistics and overall sustainment requirements defined as referenced in the CPD?


(c) Are logistics parameters and test points in the TEMP?


(d) Acceptable performance in development, test and evaluation, and operational assessment, mature software capability, acceptable interoperability, and acceptable operational supportability?


(e) Has the system demonstration that it is affordable throughout the life cycle, optimally funded, and properly phased for rapid acquisition?


(f) Are the two major focus areas for sustainment during this phase designing the critical aspects of supportability through application of the Systems Operational Effectiveness (SOE) model and defining the initial framework with options for the long term performance based support?

PDR Page 73 of 87

Risk Character




0 0 0 0 0

11

11.a

11.b

11.c

11.d

11.e

11.f

11.g

11.h

11.i

Level 1, HSI, risk, logistics, training, T&E, technology, programmatic

11. Program Risk Assessment


a. Have risk items in the preliminary design been defined and analyzed?

risk, logistics, HSI, technology, programmatic

b. Is the risk assessment process tightly coupled with the technical effort and reflective of the technical risks inherent in the preliminary design?

HSI, training, logistics, risk, technology, T&E, programmatic

c. Has the risk assessment addressed future risks to detailed design, developmental test, operational test, training, and production / fielding of the system?


d. Is there adequate acceptance among the technical team as to risks and mitigations?


e. Is the technical risk assessment being shared at all levels of the program team?


f. Have supportability and logistics risk items been defined, analyzed, and included in the program risk assessment?


g. Have cost and schedule impacts for supportability and logistics risk mitigation been documented and identified in the LRFS?

risk, HSI, logistics, T&E, programmatic

h. Are SE requirements for the T&E program understood, and is testing for unique SE included in the test plans?

risk, HSI, logistics, T&E, programmatic

i. Are Government and contractor T&E facility requirements solidified, and will facilities be available to meet the schedule?

PDR Page 74 of 87

Risk Character




0 0 0 0 0

12

12.a

12.b

12.c

0 0 0 0 0

12.d

12.d(1)

12.d(2)

12.d(3)

12.d(4)

12.d(5)

12.d(6)

12.d(7)

12.d(8)

12.d(9)

Level 1, software, HSI, logistics, risk, T&E, technology, programmatic, interoperability

12. Completion / Exit Criteria


a. The PDR is considered complete when all draft RFAs are signed off, and an acceptable level of program risk is ascertained, and were all PDR RFAs properly completed (closed)?

HSI, risk, logistics, T&E, technology, programmatic

b. Were the proper technical disciplines represented at the review?

HSI, risk, logistics, T&E, programmatic

c. Were all required flight clearance performance monitors involved, and do they concur with the preliminary design?

HSI, software, logistics, T&E, technology, risk, programmatic

d. Were PDR logistics and supportability issues captured in RFAs and properly adjudicated and assigned?


(1) Does the status of the technical effort and design indicate OT success (operationally suitable and effective)?


(2) Can the preliminary design satisfy the CDD / CPD?


(3) Can the logistics and supportability concepts for the preliminary design satisfy the CDD / CPD?


(4) Are adequate processes and metrics in place for the program to succeed?

HSI, T&E, logistics, risk, programmatic

(5) Are the risks known and manageable for DT / OT?

HSI, logistics, technology, programmatic

(6) Is the program schedule executable within the anticipated cost and technical risks?


(7) Are the system requirements understood to the level appropriate for this review?


(8) Is the program properly staffed?


(9) Is the program Non-Recurring Engineering (NRE) executable with the existing budget?

PDR Page 75 of 87

Risk Character




12.d(10)

12.d(11)

12.d(12)


(10) Is the preliminary design producible within the production budget?


(11) Was verification that the integrated architecture System Views (SVs) and Technical Views (TVs) conducted, and are the views consistent with, the appropriate Operational Views (OVs), the CPD, the ISP and NR-KPP?


(12) Has verification that system data has been entered / updated in the Net-Centric Operations and Warfare implementation baseline?


Preliminary Design ReviewProgram Risk Assessment Checklist

Risk Character

Legend: R Y GR = Red, Y = Yellow, G = Green, U = Unknown / Unavailable, NA = Not Applicable

SpecialInterest




Program Risk Assessment Checklist 0


0

Risk Character

U NA ItemComments / Mitigation

U = Unknown / Unavailable, NA = Not Applicable



Risk Character


SpecialInterest






0

Risk Character





Risk Character


SpecialInterest






0

Risk Character





Risk Character


SpecialInterest






0

Risk Character





Risk Character

Legend: R Y G

0 0 0 0

0 0 0 0

0 0 0 0

0 0 0 0

0 0 0 0

0 0 0 0

0 0 0 0



Level 1, software, T&E, logistics, HSI, training, risk, programmatic

1. Timing / Entry Criteria

Level 1, software, T&E, PQM, training, risk, logistics, HSI, programmatic, interoperability

2. Planning

Level 1, software, T&E, training, logistics, risk, technology, HSI, programmatic, interoperability

3. Program Schedule

Level 1, EVM, logistics, PQM, software, HSI, training, T&E, technology, risk, programmatic, interoperability

4. Management Metrics

Level 1, HSI, software, T&E, logistics, training, technology, risk, programmatic, interoperability

5. Program Staffing

Level 1, software, logistics, PQM, training, HSI, risk, technology, T&E, programmatic, interoperability

6. Process Review

Level 1, HSI, T&E, logistics, training, risk, software, programmatic, interoperability

7. Requirements Management

Return to PDR

0 0 0 0

0 0 0 0

0 0 0 0

0 0 0 0

0 0 0 0

Totals: 0 0 0

Level 1, software, risk, logistics, T&E, training, HSI, programmatic, interoperability

8. Net-Centric Operations and Warfare Consolidated Compliance Checklist

Level 1, T&E, HSI, PQM, software, technology, risk, logistics, training, programmatic, interoperability

9. System Preliminary Design

Level 1, HSI, logistics, training, software, T&E, technology, risk, programmatic, interoperability

10. Life Cycle Logistics

Level 1, HSI, risk, logistics, training, T&E, technology, programmatic

11. Program Risk Assessment

Level 1, software, HSI, logistics, risk, T&E, technology, programmatic, interoperability

12. Completion / Exit Criteria





0

Risk Character

U NA Item Comments/Mitigation

0 01 0

0 0

2 0

0 0

3 0

0 0

4 0

0 0

5 0

0 0

6 0

0 0

7 0


0 0

8 0

0 0

9 0

0 0

10 0

0 0

11 0

0 0

12 0

0 0

Documents

PDR Risk Checklist - · XLS file · Web viewc. Has the risk assessment addressed future risks to detailed design, developmental test, operational test, training, ... LORA and testability