NDIA Top SE-SW Issues – 2010 Preliminary Results 1 NDIA Top SE and SW Issues – 2010 Report on NDIA SE and SW Workshops Held March 16-18, 2010 Report to

NDIA Top SE-SW Issues – 2010Preliminary Results

1

NDIA Top SE and SW Issues – 2010

Report on NDIA SE and SW Workshops

Held March 16-18, 2010

Report to NDIA SE Division

June 15, 2010

Facilitators – SED Vice Chairs:• SW Issues - Geoff Draper, Harris• SE Issues - Hal Wilson, Northrop Grumman


2

Task Description

Identify Top Systems Engineering and Software Engineering problems or issues prevalent within the defense industry• Assess progress and update issues from 2006 workshops

Document issues• Description and current state• Rationale and SW impacts• Develop recommendations (short term and long term)

Generate task report• Submit to OSD by May 2010


3

Task Group Activities

• Identify Candidate SW Issues

• Top SE-SW Issues (2006)

• Status of • OSD/

Industry• Initiatives

• Prioritize SW Issues

and Actions

• Detailed sub-issues• Effects and root

causes• Consensus review

• Document Top SW

Issues for OSD

Report

• Candidate issues

• Task reports

• Issues• Detailed

description• Recommendations

• Top SE/SW Issues(2010)

• Background• Brainstorming• Group discussion

NDIA reports on Top SE and SW issues in progress.Due to be submitted to OSD May 2010.


4

NDIA Top Software Issues – 2010

Preliminary Results -

NDIA Top SW Issues Workshop

March 16-17, 2010


5

2010 - Common Themes Noted

The need for rapid development has increased – rapid response to mission needs (faster, better, cheaper). Need to find ways to be more responsive, quicker.

The need for better SW architectures and modular components has been reinforced. Architectures are more difficult when accounting for legacy upgrades, SoS, and interoperability.

Agile terminology is often misused and could be misleading Stakeholder commitment to Agile development processes is not widely understood (acquisition agility is not the same as Scrum).

SW community must be engaged in advanced development planning.


6

2010 NDIA Top SW Issues

1. Rapid acquisition: The increasing demands of the war fighter are requiring development of software and systems more rapidly than acquisition models and development processes allow.• Issues: responsiveness; lack multi-disciplinary approach; process

mismatch; prioritization; adaptation; fielding prototypes; sustainability trade-offs.

2. Start programs right: Inadequate/insufficient program planning and start-up has a negative impact on the program’s SW success.• Issues: success-oriented plans; rework from late consideration of SW

impacts; inconsistent use and maintenance of SDP; ineffective org structures, IMP/IMS; changing baselines; inadequate expertise; lack SW engagement in technical trades and risks


7

2010 NDIA Top SW Issues

3. Execute programs right: Effective engineering practices are not well-defined or consistently implemented for key software processes with emerging mission needs and supply chain dependencies.• Issues: SoS, SW/SE I&T, system assurance, COTS/NDI integration;

methodologies, scalability to complex systems.

4. Sustainment: There is insufficient attention given to the overall software life cycle activities including sustainment and changing threats.• Issues: sustainment resources are unable to keep up with changing

mission scenarios; sustainment is not always considered adequately in system life cycle planning, engineering, and budgets; trades for sustainability may be negatively impacted by competitive prototyping.


8

NDIA Top SE Issues – 2010

Preliminary Results -

NDIA Top SE Issues Workshop

March 17-18, 2010


9

2010 NDIA Top SE Issues

1. Rapid Fielding: Increasingly urgent demands of the warfighter require effective capabilities be fielded more rapidly than the conventional acquisition processes and development methodologies allow.• Issues: quick response capability encumbered by tradition acquisition

processes; SE process rigor may vary by acceptable risk; trades should consider impacts across entire SE life cycle; critical success factors for rapid response are not well documented or used in decision-making

Recommendation:a) NDIA Systems Engineering Division to convene a joint

government/industry study, with cross-functional representation including T&E, to develop risk-driven guidance on tailoring SE processes and activities and DoD acquisition requirements from the Defense Acquisition System to achieve rapid acquisition and deployment.


10


2. Human Capital: The quantity and quality of Systems Engineering expertise is insufficient to meet the demands of the government and defense industry.• Issues: SE skills, learning, and development; accelerating growth of

quality SEs; interfaces between SE and SW.

Recommendations:a) Develop systems engineering expertise through role

definition, selection, training, career incentives, and broadening “systems thinking” into other disciplines.

b) Establish systems engineering job codes or other indicators within government service to emphasize systems engineering in the recruiting of new hires.


11

2010 NDIA Top SE IssuesIssue 2, Human Capital: Recommendations (con’t.)

c) OSD should establish policy/guidance for services to include systems engineering expertise in RFP/proposal efforts and program start-up teams with reach-back capability. Consideration should be given to integrating this initiative with the OSD transition team concept. Discrete budgets should be allowed for the SE effort and fenced from cuts without a thorough review of the implications.

d) NDIA SED Education and Training Committee should investigate and recommend specific methods and techniques to accelerate the training and development of Systems Engineers to include purposeful OJT rotation programs to compliment course work.

e) Provide contract incentives to increase the development of systems engineers on programs.


12


3. Effective SE: Systems engineering practices known to be effective are not consistently applied or properly resourced to enable early system definition• Issues: SE best practices not consistently followed; competitive pricing can

lead to tradeoffs in SE process fundamentals; SE value not consistently understood; dependencies on SE resources and experience

Recommendations:a) Identify key SE information that decision makers need and

develop consistent methods for presenting that information. b) Establish a working group to determine why proven tools and

techniques are not consistently applied. c) To create a receptive audience to gain full value from these

recommendations to be effective, implementation of Issue 2 recommendations are encouraged.


13

4. Informed decision-making: Decision makers do not have the right information at the right time to support informed and proactive decision making that ensures effective and efficient program planning, management and execution.• Issues: effective use of leading indicators to manage program success; lack

a quantitative basis for decision-making; address systemic program technical issues.

Recommendations:a) NDIA will create a roadmap that programs can use to

facilitate the definition and implementation of a technical measurement process to support engineering management decisions. The roadmap should be coordinated with INCOSE Measurement Working Group and PSM.



14


Issue 4, Informed Decision Making, Recommendations (con’t.)

c) NDIA should create a research agenda to investigate causes and effects for program systems engineering success and characterization of SE methods, practices, and solution approaches. Investigate the identification of context factors and how they dynamically interact. Include NAVAIR.

d) NDIA should work with other parties to explore & potentially create a data collection and analysis capability in order to develop technical management benchmarks. Need to create the value proposition that justifies the considerable effort likely required (e.g., OSD/Systems Engineering root cause analysis data base, USC as part of the Systems Engineering Research Center, SEI - Software Engineering Institute, PSM - Practical Software and Systems Measurement).


15


5. SE technical integrity and rework: Lack of technical authority can impact the integrity of developed systems and result in cost/schedule/system performance impacts as the technical solution is iterated and reworked in later stages of the development.• Issues: lack empowerment for proper SE; critical issues and tradeoffs not

adequately addressed; excessive rework from not resolving issues early; independent technical authority to escalate SE technical issues.

Recommendationsa) Add a requirement to the SEP guidance that the SEP must

explicitly identify the technical authority (TA) for the program: 1) the TA role, authority, responsibility, and the performance reporting

chain

2) TA conduct and chairing of technical reviews

3) the mechanism for elevating technical issues beyond the program office for resolution.


16


Issue 5, SE technical integrity and rework, Recommendations (con’t.)b) There is a need to establish event-based technical reviews with

enforced entry criteria and go deeper into technical details. 1) The government PMO (and contractor’s programs) must staff

these reviews with experienced engineers who can operate somewhat independently of the Program Office and ensure that appropriate technical authority is being applied to the program. The matrix organization (people not directly doing the work) must support peer reviews.

2) NDIA review and update guidance, as needed, for entry/exit criteria and application for consistent rigor in technical reviews


17


Issue 5, SE technical integrity and rework, Recommendations (con’t.)c) Create a career path for senior technical people with broad

expertise and wide experience who can see strategic viewpoints need a career path that leads to seniority and authority equivalent to the Program Manager and PEO. 1) In practice these engineers are by definition (rules) always

subordinated to the Program Manager; but there needs to be a technical authority chain of command to balance program authority, specifically the Chief Systems Engineer on the program. The Chief Engineer is presumed to possess the adequate skill set, the experience of a systems thinker / systems integrator, and is recognized as the technical authority for the program.

2) NDIA SE Division discuss with NDIA PM Systems Committee and OSD to determine if there are actions that can be taken through NDIA to support this.

Documents

NDIA Top SE-SW Issues – 2010 Preliminary Results 1 NDIA Top SE and SW Issues – 2010 Report on NDIA SE and SW Workshops Held March 16-18, 2010 Report to