Niwot Ridge Capabilities

© 2015 Niwot Ridge Consulting, L.L.C., 4347 Pebble Beach Drive, Longmont, Colorado 303.241.9633, [email protected]

N iwo t R i d g e , L . L . C .

INCREAS ING THE PROBAB I L I T Y O F PROGRAM SUCCES S

Successful execution of Enterprise IT, Aerospace, Defense, and Government Software Intensive Systems (SIS) requires management discipline to identify what “Done” looks like, provide measures of progress toward “Done,” identify and remove risks and impediments to reaching “Done,” and assure timely corrective actions to maintain the planned progress towards “Done.”

Performance–Based Project Management® assures people, processes, and tools are integrated to be capable of successfully delivering the program success using seven core capabilities.

Capability Beneficial Outcome

➊

Integrated Master Plan & Integrated

Master Schedule

Starting with the Concept of Operations, the Capabilities Based Plan, and other program mission documents, informs the Integrated Master Plan (IMP) and a DI-‐MGMT-‐81861 compliant risk adjusted, resource loaded, Integrated Master Schedule (IMS) with Work Packages (WP) showing the credible path to success.

➋

Earned Value Management

Systems (EVMS)

EVM requirements elicitation, EVMS product selection, management of installation and startup, and integration with Accounting and Procurement systems. Develop the System Description (SD), Work Instructions (WI), training and mentoring needed for a successful deployment of the EVMS and its application to program and compliance with ANSI-‐748-‐C.

➌

Programmatic and Technical

Risk Management

Cost, schedule, and technical risk management integrated with the Performance Measurement Baseline establishes the credible Estimate At Completion (EAC) and Management Reserve (MR).

➍

Proposal Management

for management volume

Starting with proposal “win themes,” develop the Integrated Master Plan for deliverables and the supporting Integrated Master Schedule, Technical Performance Measures, Risk Handling strategies, Measures of Effectiveness, and Measures of Performance to assure proposal is credible and winning.

➎

Six Business System Rule Integration

Enterprise system integration capabilities connects business Accounting, Procurement, and Earned Value Management Systems for Cost Plus or Fix Price Incentive contracts valued at $20M or more, in accordance with DFARS §234.201(1)(iii) and (iv).

➏ Agile Software Development

As the US Government moves toward agile development processes to software intensive systems, integrating agile with Earned Value Management is needed. Assures DCMA compliance while applying the best of agile development methods in ERP and software intensive systems.

➐

DCMA/DCAA Validation and Surveillance

For contract awards greater than $20M, planning, development, deployment, training, and support activities for the EVMS, start with contract award, and continue through Integrated Baseline Review (IBR), then onto program execution. Hands-‐on Control Account Manager (CAM) training and mentoring process assures staff prepared for a successful DCMA validation process.


❶ – IN TEGRATED MASTER PLAN / IN TEGRATED MASTER SCHEDULE

PROJECT SUCCESS REQUIRES A CLEAR AND CONCISE DESCRIPTION OF “DONE” Hands–on approach constructs, baselines, reviews, and executes, with full client participation, the program using the Integrated Master Plan and Integrated Master Schedule (IMP/IMS). The IMP/IMS progress is defined through risk adjusted physical percent complete rather than measuring the passage of time and consumption of resources.

This approach starts with Vertically planning each Program Event (PE), through Significant Accomplishments (SA), to Accomplishment Criteria (AC), to Work Packages, to provide visibility to the increasing maturity of all project deliverables. Only then, does planning take place Horizontally for the dependencies between the Work Packages and Planning Packages.

Program Events are more than milestones, they are maturity assessment points in the program, where pre-‐defined deliverables are assessed to assure Technical Performance is being met against pre-‐defined metrics. As well, pre-‐defined levels of risk are being retired or mitigated as planned. Event based planning and scheduling provides an alternative approach with several important benefits: § Measures of increasing maturity of products and services produced by work efforts are

directly connected to the value produced by the program for the customer. This is the foundation of Capabilities Based Planning. The program plan states what capabilities will be available at what time, how these capabilities are described and assessed, and how the Measures of Effectiveness and Performance of a capability are made visible in the plans and schedules.

§ By isolating all work to Accomplishment Criteria “containers,” the measurements of progress is described through the Exit Criteria for this collection of work. This provides direct visibility to progress through measures of Physical Percent Complete.


❷ – EARNED VALUE MANAGEMENT SY ST EM (EVMS)

EARNED VALUE MANAGEMENT IS THE ‘BEST TOOL’ FOR MANAGING LARGE, COMPLEX ACQUISITION PROGRAMS – DR. ASHTON CARTER (USD, AT&L) 26 NOVEMBER 2009

Earned Value Management consulting and compliance practice starts with program management, process development, and EVM engine selection. With the Earned Value Management System (EVMS) defined, the Program Manager, Control Account Managers, Accounting staff, and Business Management staff are trained and mentored to assure they are prepared for the application of EVM. The System Description and Work Instructions are developed and tailored to fit the business processes. The DCMA self-‐assessment and system validation review is then started to complete the compliance process.

For contract awards greater than $20M, an Earned Value Management system is required for contract execution and for awards greater than $50M, a Validated Earned Value Management System is required. ANSI-‐748-‐C defines 32 guidelines for EVM compliance as stated in FAR §34.2 and DFARS §234.001, §252.234-‐7002, and PGI §234.201 (7)(iii)

This integrated management system allows program managers to: (1) Quantify and measure program / contract performance; (2) Provide an early warning system for deviation from baseline; (3) Handle risks associated with cost and schedule overruns; and (4) Provide forecasts of final cost and schedule dates.

These services develop work processes needed to integrate the EVMS with the accounting system, assure the program staff is trained and capable of managing the contract with EVM, and ready to produce the monthly reporting required by DI-‐MGMT-‐81861. Starting with 11 critical Guidelines (colored shown below) to establish a credible program controls foundation, before moving to the second tier of compliance Guidelines.

ORGANIZATION PLANNING AND BUDGETING ACCOUNTING MANAGERIAL ANALYSIS REVISIONS

1. Define WBS 6. Schedule The Work 16. Record Direct Costs 22. Periodically Collect

Control Account Sums

28. Incorporate Changes

2. Identify Organizations

7. Identify Products And Milestones

17. Summarize Costs In The Wbs 23. Determine Variance 29. Reconcile Budgets

3. Integrate Subsystems

8. Set Time Phased Budget 18. Summarize The Cost In The OBS

24. Budget And Actual Indirect Costs

30. Control Retroactive Changes

4. Identify Overhead 9. Identify Significant Cost Elements

19. Record Indirect Costs

25. Sum Data And Variances

31. Perform Only Authorized Work

5. Integrate WBS And OBS

10. Establish Discrete Work Packages

20. Identify Equivalent And Lot Costs 26. Manage Action Plans

32. Document PMB Changes

11. Sum Work And Planning Packages

21. Performance Material Accounting

27. Revise Estimate At Completion

12. Identify Level Of Effort Work

13. Set Overhead For Organization

14. Identify Management

Reserve And Unallocated Budget

15. Identify Target Costs And Budgets

These 11 critical success Guidelines, of the 32 in ANSI-‐748-‐C, are the start of the EVMS deployment process. With these 11, program performance management processes are in place to increase the probability of project success. DCMA validation starts with 16 critical Guidelines (1, 3, 6, 7, 8, 9, 10, 12, 16, 21, 23, 26, 27, 28, 30, or 32). These are considered “show stoppers” if they are not met.


❸ – PROGRAMMATIC AND TECHNICAL R ISK MANAGEMENT

“RISK MANAGEMENT IS HOW ADULTS MANAGE PROJECTS” – TIM LISTER, IBM FELLOW

Continuous risk management process is based on industry standard approaches to integrating risk, cost, schedule, and technical

performance measures into a credible Performance Measurement Baseline.

Risk Management is the critical success factor for all programs. Even more so, when executing government funded programs that produce mission critical outcomes.

The application of Continuous Risk Management (CRM) processes assures the program has made the proper adjustments to the Performance Measurement Baseline to include risk handling and development of a credible Management Reserve for handling of event based risks that occur during the life of the program. This includes: 1. Identifying Risks – before risks can be managed, they must be identified. Identification surfaces

risks before they become problems that adversely affect a project. The Software Engineering Institute (SEI) has developed techniques for surfacing risks by the application of a disciplined and systematic process that encourages project personnel to raise concerns and issues for subsequent analysis. SEI paradigm can be applied all project domains, not just software development.

2. Analyzing Risks – through the conversion of risk data into risk decision‒making information. This provides the basis for the project manager to work on the “right” risks. This analysis examines the risks in detail to determine the extent of the risks, how they relate to each other, and which ones are the most important.

3. Planning Risk Handling Strategies – by turning risk information into decisions and actions (both present and future). Planning involves developing actions to address individual risks, prioritizing risk actions, and creating an integrated risk management plan. This planning effort decides what if anything should be done with the risk. Planning produces risk actions for individual or sets of risks. Risks are planned by the people who have the knowledge, expertise, background, and resources to effectively deal with the risks.

4. Tracking the Status of Risk-‐handling Strategies – by monitoring the status of risks and actions taken to ameliorate them. Appropriate risk metrics are identified and monitored to enable the evaluation of the status of risks themselves and of risk mitigation plans. Tracking serves as the “watch dog” function of risk management.

5. Making Decisions Based on the Performance of the Risk-‐Handling Activities – using the Tracking status reports produced in the fourth step and deciding what to do with the risks, based on the reported data. The person accountable for a risk normally makes the control decisions for that risk.

6. Communicating the Risk Information – takes place across with all levels within the project and organization, within the customer organization, and most especially, across that threshold between the development organization, the customer, and, where different, the user. Because communication is pervasive, the approach addresses it as integral to every risk management activity and not as something performed outside of, and as a supplement to, other activities.


Program ExecutionPMB for IBRProposal SubmittalDRFP & RFP

Performance Measurement Baseline

Tasks (T)

BOE

% Complete

Statement of Work

Program Deliverables

IMP

Accomplishmen ts (A)

Criter ia (C)

EVMS

Events (E)

Budget Spreads by CA & WPCAIV

Capabilit ies Based Requirements

X BCWS =

Probabilistic Risk Analysis

=

Time keeping and ODC =

Technical Performance Measure

BCWP

ACWP

Cost & Schedule Risk Model

BCWS

De c r e a s in g t e c h n i c a l a n d p r o g r a mma t i c r i s k u s in g R i s k Ma n a g e me n t Me t h o d s

IMS

Physical % Complete

Continuity and consistency from DRFP through Program Execution

❹ – PROPOSA L MANAGEMENT THE KEY TO PROGRAM SUCCESS IS TO “EXECUTE AS YOU PROPOSE”

Performance–Based Project Management® assures highest probability of project success, starting with the proposal, continuing through Integrated Baseline Review (IBR), and onto program execution.

Success of any proposal and resulting contract award starts with a credible description of how the contractor is going to deliver on the needed capabilities, the technical and operational requirements that fulfill these capabilities, how risks associated with the deliverables are managed, and how measures of progress to plan are used to stay on budget, stay on schedule, and deliver technical compliance.

1. During the Draft RFP and Final RFP stage, confirm all deliverables are clear and concise with Measures of Effectiveness (MOE) and Performance (MOP) for those deliverables. This approach assures the contractor has a clear and concise understanding of what Done looks like during the proposal phase. Without this clarity, corrective actions must be taken to improve the integrity of the proposal. This is done by decomposing the Statement of Work (SOW), identifying the deliverables, connecting these with the needed capabilities, and establishing the Technical Performance Measures.

2. Proposal development and submittal is based on the Integrated Master Plan / Integrated Master Schedule (IMP/IMS), a resource loaded Performance Measurement Baseline, risk adjusted durations and cost basis, Event–Based risk handling strategies with Management Reserve (MR), and an integrated technical, operational, cost, and schedule performance measurement process.

3. Once awarded, the Performance Measurement Baseline is established at the Work Package level using Earned Value’s Budgeted Cost for Work Scheduled (BCWS). Technical Performance Measures (TPM) are established for each deliverable to assure actionable information is provided to the decision makers in meaningful units of measure.

4. During execution, fine–grained assessments of performance to plan are made to assure progress is made against the Performance Measurement Baseline. The Integrated Master Schedule (IMS) provides guidance for the sequence of work and measures of technical progress, while adhering to cost and schedule. The result is a risk adjusted Estimate At Completion (EAC) using the Earned Value Management data, based on Monte Carlo Simulation and Event–Based Risk modeling.

Performance-‐Based Project Management® provides end-‐to-‐end visibility to assure program success


Under the Business System Rule, the DCMA contracting officer … must determine the acceptability of the contractor’s systems, approve or disapprove the system, and pursue correction of any deficiencies.

❺ – S I X BUS INE S S SY ST EM RULE

‘CONTRACTOR BUSINESS SYSTEMS AND INTERNAL CONTROLS ARE THE FIRST LINE OF DEFENSE AGAINST WASTE, FRAUD, AND ABUSE. WEAK CONTROL SYSTEMS INCREASE THE RISK OF UNALLOWABLE AND UNREASONABLE

COSTS ON GOVERNMENT CONTRACTS’ – DEFENCE CONTRACT MANAGEMENT AGENCY

Provide integration of the Earned Value Management, Procurement, and Accounting system capabilities needed to deliver program performance visibility to the decision makers. Integration starts with program’s Performance Measurement Baseline (PMB) established in the Earned Value Management System, integrated with a DCAA validated accounting system, and a CPSR compliant procurement system, to produce the Integrated Program Management Report (IPMR), per DI-‐MGMT-‐81861 for monthly submittal to the government.

DFARS 252.242-‐7005, 12 Feb 2012 Business System Guidance

Accounting DFARS §242.75, §252.242-‐7006, PGI 242.7502 Cost Estimating DFARS §215.407-‐5-‐70, §252.215-‐7002, PGI, 215.407-‐5-‐70(e)(3) Procurement DFARS §244.1, §252.244-‐7001, PGI 244.305-‐70 Material

Management and Accounting

DFARS §242.72, §252.242-‐7004, PGI 242.7203

Property Control DFARS §245, §252.245-‐7003, PGI 245.105 Earned Value Management DFARS §234.001, §252.234-‐ 7002, PGI 234.201 (7)(iii)

The six business systems apply to contracts, awarded after May 2011. Circumstances where these individual clauses are required vary. A contractor can expect to encounter one or more of the clauses on negotiated, cost—based DOD contracts in excess of $700,000. The Business System rule is applicable to accounting systems with Cost-‐reimbursement, incentive type, time-‐and-‐materials, or labor-‐hour contracts; and Contracts with progress payments made on the basis of costs incurred by the contractor or on a percentage or stage of completion. As well, Earned Value Management is applicable on cost or incentive contracts valued at $20,000,000 or more, and for other contracts for which EVMS will be applied in accordance with DFARS §234.201(1)(iii) and (iv).

Contracts are guided through Self-‐Assessment and Validation Review based on the 19 DCAA guidelines and the 32 DCMA guidelines. The focus is on Earned Value and its integration with the Accounting and Procurement systems and processes in compliance with Memorandum for Regional Directors, DCAA, April 24, 2012; Audit Guidance on Auditing Contractor Business Systems and Contractor Compliance with DFARS 252.242-‐7006, Accounting System Administration.


❻ – AG I L E DEVE LOPMENT IN ENTERPR I S E IT AND FEDERAL GOVERNMENT THE NATIONAL DEFENSE AUTHORIZATION ACT, 2010, §804 HAS BROAD IMPLICATIONS FOR DOD PROGRAMS

ACROSS THE IT ACQUISITION LIFECYCLE DESCRIBING HOW TO FOSTER CONTINUOUS USER INVOLVEMENT, MULTIPLE AND RAPID EXECUTION INCREMENTS, SUCCESSIVE PROTOTYPING AND MODULAR OPEN-‐SYSTEMS

Agile Means …“Being able to turn inside the loop of unfolding events.” 1

Delivering Department of Defense Software Intensive Systems (SIS) on time, within budget and with the required functionality with traditional systems processes has been problematic. This problem increases as the complexity of these systems grows and more functionality within systems is relegated to software. In commercial software development, Agile is proposed to solve some problems encountered in emerging requirements and customer feedback that are the root cause of delays in delivery. The difficulty in applying Agile to DOD acquisition is the existing compliance requirement with Earned Value Management called out in DFARS §252.234-‐7001 and 7002.

Our approach starts with establishing Agile processes to SIS development based on Earned Value Management to measure physical percent complete using Measures of Effectiveness (MOE), Measures of Performance (MOP), Technical Performance Measures (TPM), and Key Performance Parameters (KPP), applied to baselined Rolling Waves, and emergent Planning Packages in future Rolling Waves.

GL # EVM Criteria Agile Approach

1 Define the Work Breakdown Structure (WBS) Features and Stories define tasks in the backlog

2 Identify Organization Structures (OBS) Self organizing teams

5 Integrate WBS and OBS Teams aligned with deliverables in sprints

6 Schedule The Work in a Logical Order Iterations and Releases produce outcomes

7 Identify Products & Milestones Working software at the end of each iteration

8 Set time phased budget for planned work Fixed length iterations and releases

16 Record direct costs for work performed Fixed staff = Level of Effort = Fixed cost

23 Determine variances from plan Velocity measures missed features in iteration

25 Sum data and variance data for analysis Missed features moved to next iteration

26 Manage action plans based on analysis Replan missed features, adjust velocity

28 Incorporate changes to baseline plans Replan missed features, adjust velocity

Integrating agile into formal program management processes is starlight forward once it is recognized that Agile’s core principle is the delivery of working software. Earned Value management principles are based on the assessment of the delivery of products that meet the planned Capabilities, accessed through Measures of Effectiveness, Measures of Performance, and compliance with Technical Performance Measures.

Once these two principles are connected, Agile and more formal performance management process can then share the same goal – delivery of needed Capabilities at the needed time for the planned budget. 2

1 Dr. Ashton Carter, Under Secretary of Defense for Acquisition, Technology and Logistics, Sep/Oct, 2010 Defense AT&L. 2 Best Practices for Implementing Agile Methods: A Guide for Department of Defense Software Developers, IBM Center for The Business of 2 Best Practices for Implementing Agile Methods: A Guide for Department of Defense Software Developers, IBM Center for The Business of

Government.


❼– DCMA / DCAA VAL IDAT ION AND SURVE I L LANCE RECEIPT OF A $20M OR GREATER COST REIMBURSEMENT CONTRACT IS SUBJECT TO FAR §34.2 AND THE REQUIREMENT FOR SELF-‐ASSESSMENT (SA) OR A FULL EVMS VALIDATION REVIEW INCLUDING THE SA

The Self-‐Assessment, Validation, and Surveillance process determines if the contractors: § Processes, procedures, and methods are compliant with the 32 Guidelines of ANSI-‐748-‐C. § Descriptive documents containing the contractor’s policies and procedures are used in actual

management activities on the program. § Earned Value data generated by the Earned Value Management System is evidenced in use for the

management of the program. § Management knowledge of the EVMS roles and responsibilities of it operating personnel are capable

of maintaining compliance with ANSI-‐748-‐C.

These work processes assure four phases on the path to Validation are successfully completed with minimal disruption to the contractor’s daily routine.

These four phases are: 1. Data Item List Package (DILP) submittal – this occurs when there is 2 months of clean

data available from the Earned Value Management System. 2. EVMS Self–Assessment (SA) Package submittal – usually 4 to 5 weeks after submittal of

the DILP. 3. Readiness Assessment (RA) – usually 30 days after submittal of DILP. 4. Compliance Review (CR) – usually six months after submittal of DILP.

The primary role in guiding our clients through each of these Assessment and Review stages, using our step-‐by-‐step process to assure the client reaches success, includes:

Major Phase Detailed Work Activities During Each Phase

Risk Assessment

§ Program phase dependent risk assessment of processes, tools, and procedures. § Discrepancy Report and Corrective Action Report, cost, schedule, and technical performance variance assessment.

System Description and Earned Value Management

System

§ Confirm all sections are present and cover the ANSI-‐748-‐C topics. § Assure all data items can be traced to sections of the System Description. § Confirm Work Instructions definitize the guidance in the System Description. § Document compliance at the level of application for all data and processes. § Programmatic procedures for common processes, flexibility, and unique instances. § Collect actual data generated by program to support compliance with EVMS and SD. § Assemble all data, processes, and access procedures in a single Story Board Process.

Training

§ Use of System Description confirmed. § Control Account Manager (CAM) interview questions for all ANSI-‐748-‐C sections. § Document review requirements and CAM training for document usage. § First round mock interviews for CAMs with SD, WI, and live data.

Self Assessment

§ Verification of process implementation using Story Boards and data traces. § Document trace of system compliance with ANSI-‐748-‐C Guidelines. § Conduct Cross Reference Checklist for EVMS. § Documentation of proper system implementation. § Follow up discrepancies with corrective actions. § Mock Control Account Manager Interviews. § Complete Interview Findings Form (IFF) for PM, CAMs, Accounting, Business Management, and Risk. § Re-‐verification of closed discrepancies.

Data Trace, Collection, Verification, Storage, and

Access Procedures

§ Data collection for validation review. § Review for accuracy, integration, and consistency with the EVMS. § CAM focus on documents – WAD, Planning and Budgeting, IMS, VAR, EAC, revisions. § Focus on GL’s assigned – all budget and change logs traceable to Performance Measurement Baseline. § Focus on data reference integrity and traceability to IPMR and all change control logs. § Verify 9 EVM Processes and 32 Guidelines before final validation.

Documents

Niwot Ridge Capabilities