1

Jo Ann LaneUniversity of Southern CaliforniaCenter for Systems and Software Engineeringjolane@usc.edu

Dr. Paul CarlockNorthrop Grumman CorporationMission Systems paul.carlock@ngc.com

COCOMO Forum 2006

COSOSIMO: How Well Does It Capture the Unique Aspects of System of Systems Engineering

Processes?

COSOSIMO: How Well Does It Capture the Unique Aspects of System of Systems Engineering

Processes?

2

• What is System of Systems Engineering (SoSE) and what makes it unique

• System of Systems Enterprise Systems Engineering (SoS ESE) and Enterprise Architecture Management Framework (EAMF) Overview

• Comparison of SoS ESE EAMF with SoSE unique features

• SoS ESE EAMF track record with respect to SoS success• COSOSIMO overview and relationship to SoS ESE EAMF• COSOSIMO parameter consistency with respect to SoS

ESE key features• Conclusions and impact on COSOSIMO evolution

OutlineOutline

3

SoSE and

What Makes it Unique

SoSE and

What Makes it Unique

4

• USAF SAB Report on SoSE for Air Force Capability (USAF 2005): The process of planning, analyzing, organizing, and integrating the capabilities of a mix of existing and new systems into a system-of-systems capability that is greater than the sum of the capabilities of the constituent parts. This processes emphasizes the process of discovering, developing, and implementing standards that promote interoperability among systems developed via different sponsorship, management, and primary acquisition processes.

• National Centers for Systems of Systems Engineering (NCOSOSE): The design, deployment, operation, and transformation of metasystems that must function as an integrated complex system to produce desirable results. These metasystems are themselves comprised of multiple autonomous embedded complex systems that can be diverse in technology, context, operation, geography, and conceptual frame. (http://www.eng.odu.edu/ncsose/what_is_SOSE.shtml)

What is SoSE?What is SoSE?

5

• Key areas where SoSE activities differ from traditional SE – Architecting composability vs. decomposition (Meilich 2006)

– Added “ilities” such as flexibility, adaptability, composability (USAF 2005)

– Net-friendly vs. hierachical (Meilich 2006)

– First order tradeoffs above the component systems level (e.g., optimization at the SoS level, instead of at the component system level) (Garber 2006)

– Early tradeoffs/evaluations of alternatives (Finley 2006)

– Human as part of the SoS (Siel 2006, Meilich 2006, USAF 2005)

– Discovery and application of convergence protocols (USAF 2005)

SoSE Compared to Traditional SE ActivitiesSoSE Compared to Traditional SE Activities

6

• Key areas where SoSE activities differ from traditional SE (continued)– Organizational scope defined at runtime instead of at system

development time (Meilich 2006)– Dynamic reconfiguration of architecture as needs change (USAF

2005)– Modeling and simulation, in particular to better understand

“emergent behaviors” (Finley 2006)– Component systems separately acquired and continue to be

managed as independent systems (USAF 2005)– Intense concept phase analysis followed by continuous

anticipation; aided by ongoing experimentation (USAF 2005)

SoSE Compared to Traditional SE Activities (continued)

SoSE Compared to Traditional SE Activities (continued)

7

• Key Challenges for SoSE– Business model and incentives to encourage working

together at the SoS level (Garber 2006)– Doing the necessary tradeoffs at the SoS level (Garber

2006)– Human-system integration (Siel 2006, Meilich 2006)– Commonality of data, architecture, and business strategies

at the SoS level (Pair 2006)– Removing multiple decision making layers (Pair 2006)– Requiring accountability at the enterprise level (Pair 2006)– Evolution management (Meilich 2006)– Maturity of technology (Finley 2006)

For the most part, SoSE appears to be SE+

SoSE Compared to Traditional SE Activities (continued)

SoSE Compared to Traditional SE Activities (continued)

8

SoS ESE OverviewSoS ESE Overview

9

SoS ESESoS ESE

“System of Systems (SoS) Enterprise Systems Engineering (ESE) (also called “Agency-Level Systems Engineering” for federal enterprises) is the set of processes and activities devoted to capability-delivery design and integration throughout an enterprise’s mission planning.

It translates and implements the enterprise’s goals and objectives into a comprehensive and coherent Enterprise Architecture, a Strategic Plan for “System of Systems” Evolution or transformation.

Essentially, it is the enterprise’s Strategic Planning and Control Process.”

• SoS ESE was developed to respond to the Information Technology Management Reform Act of 1996 and the Government Performance and Results Act (GPRA) of 1993.

• Low risk transformation of complex SoS Enterprises requires the discipline and rigor of SoS ESE.

• SoS ESE has a solid technical foundation that satisfies federal mandates for enterprise strategic planning and control.

By way of definition…

10

The Enterprise Architecture (EA) incorporates both technical and programmatic information to– Define the strategic plan and roadmap for transformation– Support informed investment decision making

The Enterprise Architecture Management Framework (EAMF) provides the disciplined processes to evolve, maintain, and ensure the proper implementation of the Enterprise Architecture both efficiently and successfully

– Combined focus on strategic, tactical, and operational objectives– Clear roles and responsibilities are assigned to each partner in agency transformation– Governance support mechanisms for managing transformation activities

Integrates and leverages Best Practices from successful large-agency transformation efforts already accomplished

An approach to managing enterprise transformation that Utilizes a comprehensive, integrated, mission service-based enterprise architecture Defines an enterprise architecture management strategy that effectively employs all elements of the

acquisition framework to minimize risk and expedite delivery of benefits Is flexible to accommodate and leverage proven best practices of mature corporate, mission, and

cultural process reengineering methodologies

The SoS ESE Methodology

Architecting for Enterprise-Level TransformationArchitecting for Enterprise-Level Transformation

11

Three-Level SoS ESE MethodologyThree-Level SoS ESE Methodology

Tight linkage to the Organization’s Acquisition Strategy is critical to establishing a transformation partnership

12

• Integrated technical and programmatic description of the enterprise• Includes technical, cost, and schedule data, and all other data necessary to

define a strategic plan– Supports development of all architecture views (DoDAF, business, temporal,

security, data, cost, …)

• Lays out the transition of capabilities aligned with agency and stakeholder values and priorities

– Mission service-based derived from community needs• Mission service provision through capabilities• Capability provision through systems, people,

facilities, support activities

• Temporal in nature– Capability provision “how”

changes with time

The enterprise architecture IS the strategic plan for enterprise transformation

Characterizing the Enterprise Architecture (EA)Characterizing the Enterprise Architecture (EA)

DoDAF Views

13

Characterizing the Enterprise Architecture Management Framework

Characterizing the Enterprise Architecture Management Framework

Seamless life cycle acquisition management process that extends from identification of need to capability retirement

14

The EAMF ensures proper implementation of the enterprise architecture by explicitly integrating systems engineering into the acquisition process

Enterprise Architecture Management Framework (EAMF)

Enterprise Architecture Management Framework (EAMF)

15

Enterprise Architecture TaxonomyEnterprise Architecture Taxonomy

The enterprise architecture balances User Community needs against resources to achieve successful transformation

16

Industry Best Practices in the SoS ESE Methodology

Industry Best Practices in the SoS ESE Methodology

17

Performance Modeling and Measurement

System Needs and Requirements Management

Proven SE Provides the FoundationProven SE Provides the Foundation

Requirements Analysis Architecture Development Technology Insertion Risk Management CM IV&V Readiness …

Systems Engineering Activities

18

Value Engineering ensures a customer focus andtight alignment of enterprise strategic planning and implementation

Value Engineering Provides the Strategic EdgeValue Engineering Provides the Strategic Edge

• SE transformation discipline

– Quick, decisive performance improvements

– Aligns strategy with capabilities

– Builds ownership and acceptance

• Key areas:– Strategy development– Capability analysis– Change management– Communications

19Mission Engineering comprehensively defines capabilities to support effective systems engineering

Mission Engineering Enables Dynamic ResponsivenessMission Engineering Enables Dynamic Responsiveness

• SE discipline for visual definition and linkage to requirements

– Transforms capabilities into requirements

– Captures • Mission and business

processes• Business rules• Design approaches

– Vehicle for collaborative hand-off among performing organization elements

• Key activities– Community requirements– Capability requirements– Systems requirements

EnterpriseActivity Roadmap

(OperationsAnalysis)

Multi-Dimensional Requirements

View

20Melding of proven SE disciplines provides a rapid Enterprise Architecture definition

Application Within the EAMFApplication Within the EAMF

• Integrated disciplines provide critical artifacts across the EAMF to update EA

– Tight interface to customers and priorities

– Flexible for varying situations

– Responsive to short timeline needs

• Provides the basis for a decision support framework

– Technical and programmatic data

– All stakeholders involved

21

SoS ESE EAMF Track RecordSoS ESE EAMF Track Record

• Successes– FAA (1998-2001): Led to

Congress trusting the FAA architecture planning process and increasing the budget by 42% (9/11 changed priorities)

– National System for Geospatial-Intelligence: Using to guide enterprise architecture development. Incorporates scenario-based engineering into the framework.

– Other classified programs

• Reasons why ESE EAMF has not worked well on some other programs– Lack of senior management

commitment to processes– Interdependencies not

accurately reported and tracked

– Inaccurate tracking of schedule/budget issues

• The EA, as a strategic plan, is only as good as the data it contains and the commitment of the organization to keep it current and correct

22

COSOSIMO Overview and Relationship to SoS ESE EAMF

COSOSIMO Overview and Relationship to SoS ESE EAMF

23

Planning, Requirements Management,

and Architecting (PRA)

Source Selection and Supplier

Oversight (SO)

SoS Integrationand Testing

(I&T)

Size Drivers

Cost Drivers

SoSDefinition andIntegrationEffort

COSOSIMO Reduced Parameter Sub-Model Overview

COSOSIMO Reduced Parameter Sub-Model Overview

24

COSOSIMO/SoS ESE EAMF RelationshipCOSOSIMO/SoS ESE EAMF Relationship

PRA

COSOSIMO

SO I&T

Strategic Analysis

CapabilityAnalysis

AlternativeAnalysis Implement

OperationsAnd

Sustainment

25

COSOSIMO Parameter Consistency With Respect to

SoS ESE Key Features

COSOSIMO Parameter Consistency With Respect to

SoS ESE Key Features

26

PRA

Size Drivers• # SoS-related requirements• # SoS interface protocols

Cost Drivers• Requirements understanding• Level of service requirements• Stakeholder team cohesion• SoS team capability• Maturity of LSI processes• Tool support• Cost/schedule compatibility• SoS risk resolution

LSI PRAEffort

Associated EAMF Key Features• Tight interface to customers and

priorities• Flexible for varying situations• Responsive to short timelines

needs• Provides the basis for a decision

support framework• Integrated technical and

programmatic (including cost) data

• All stakeholders involved

COSOSIMO: PRA Sub-ModelCOSOSIMO: PRA Sub-Model

27

SO

Size Drivers• # independent component

system organizations

Cost Drivers• Requirements understanding• Architecture maturity• Level of service requirements• SoS team capability• Maturity of LSI processes• Tool support• Cost/schedule compatibility• SoS risk resolution

LSI SOEffort

Associated EAMF Key Features• Identification of viable

alternatives• Comprehensive analysis of

alternatives• Selection based on stakeholder

priorities• Consistency with both current

and projected budget profiles

COSOSIMO: SO Sub-ModelCOSOSIMO: SO Sub-Model

28

I&T

Size Drivers• # SoS interface protocols• # SoS scenarios• # unique component systems

Cost Drivers• Requirements understanding• Architecture maturity• Level of service requirements• SoS team capability• Maturity of LSI processes• Tool support• Cost/schedule compatibility• SoS risk resolution• Component system maturity and

stability• Component system readiness

LSI I&TEffort

Associated EAMF Key Features• Customer communications and

feedback• IV&V• Transformation assessment and

impact review• In-service metrics and feedback

COSOSIMO: I&T Sub-ModelCOSOSIMO: I&T Sub-Model

29

Conclusions and Impact on COSOSIMO Evolution

Conclusions and Impact on COSOSIMO Evolution

30

Summary• What is “special” about SoSE in the enterprise

environment? – SoSE is SE+ with the focus being on “enterprise transformation”– SoSE is SE tightly integrated with the acquisition process– SoSE requires flexibility, flexibility, and more flexibility in both

engineering and acquisition as the environment changes during the transformation process

– SoSE requires more of a “governance support” mechanism than long term detailed planning and structured oversight

– Key features to being successful• Need a Strategic Plan/Enterprise Architecture for on-going SoS

evolution and transformation that includes technical, cost, and schedule aspects

• Planning and honesty about variance in actual progress are critical—need to constantly adjust to reality

• A flexible, evolvable architecture is required for on-going success• Understanding current business processes and re-engineering

those processes to take advantage of SoS capabilities is key

31

Summary (continued)

• How does EAMF compare to SoSE observations? – Addresses many of the differences and challenges identified in

recent conferences and workshops:• Added “ilities” such as flexibility, adaptability, composability• Early tradeoffs and tradeoffs above the component systems level• Managerial independence of component systems• Intense concept phase analysis followed by continuous anticipation• Business model and incentives to encourage working together at

the SoS level• Commonality of data, architecture, and business strategies at the

SoS level• Requiring accountability at the enterprise level• Evolution management• Technology maturity

32

Summary (continued)

• Can “close enough” SoSE effort estimates be obtained from current SE cost models such as COSYSMO?– Probably not– Current SE cost models do not account for levels of complexities

seen in many SoSs, the need for significant business process re-engineering, and the coordination of multiple component system “owners”

– Still need data from SoSE programs to determine the extent and impact of these differences

• What does this mean for COSOSIMO?– There is an identified need for COSOSIMO—those that are in

the midst of SoSE programs do not have the estimation support they need from existing cost models

– The path taken will be determined by the number and types of differences from the current SE cost model, COSYSMO

33

Carlock, P.G., and R.E. Fenton, "System of Systems (SoS) Enterprise Systems for Information-Intensive Organizations," Systems Engineering, Vol. 4, No. 4, pp. 242-261, 2001

DiMario, Mike (2006); “System of Systems Characteristics and Interoperability in Joint Command Control”, Proceedings of the 2nd Annual System of Systems Engineering Conference

Electronic Industries Alliance (1999); EIA Standard 632: Processes for Engineering a SystemFinley, James (2006); “Keynote Address”, Proceedings of the 2nd Annual System of Systems Engineering

Conference Garber, Vitalij (2006); “Keynote Presentation”, Proceedings of the 2nd Annual System of Systems

Engineering ConferenceINCOSE (2006); Systems Engineering Handbook, Version 3, INCOSE-TP-2003-002-03Krygiel, A. (1999); Behind the Wizard’s Curtain; CCRP Publication Series, July, 1999, p. 33Maier, M. (1998); “Architecting Principles for Systems-of-Systems”; Systems Engineering, Vol. 1, No. 4 (pp

267-284)Meilich, Abe (2006); “System of Systems Engineering (SoSE) and Architecture Challenges in a Net Centric

Environment”, Proceedings of the 2nd Annual System of Systems Engineering ConferencePair, Major General Carlos (2006); “Keynote Presentation”, Proceedings of the 2nd Annual System of

Systems Engineering Conference Proceedings of AFOSR SoSE Workshop, Sponsored by Purdue University, 17-18 May 2006Proceedings of Society for Design and Process Science 9th World Conference on Integrated Design and

Process Technology, San Diego, CA, 25-30 June 2006Siel, Carl (2006); “Keynote Presentation”, Proceedings of the 2nd Annual System of Systems Engineering

ConferenceUnited States Air Force Scientific Advisory Board (2005); Report on System-of-Systems Engineering for Air

Force Capability Development; Public Release SAB-TR-05-04

SoSE ReferencesSoSE References