M&S Based System Development and Testing in a Joint Net-Centric
Environment
Bernard P. Zeigler, Ph.D.,Co-Director,
Arizona Center for Integrative Modeling and Simulationwww.acims.arizona.edu
andJoint Interoperability Test Command
Fort Huachuca, AZ 85613-7051
There is an acute need for a new Net-centric testing paradigm at the enterprise level where joint and coalition operations are conducted (Editorial, ITEA Jnl, Sept, 2005)
Net-Centric T&E places an increased emphasis on standards conformance self-testing over the Global Information Grid Service Oriented Architecture (GIG-SOA ) and emphasizes services rather than systems
Simulation-based acquisition requires the use of M&S in all phases of the system development life-cycle
ConceptRefinement
TechnologyDevelopment
SystemDevelopment
& Demonstration
Production & Deployment
Operations & Support
Modelingand
Simulation
especially starting early in system development when defects can be caught and corrected with low cost
Systems of
Systems
Simulation-based development implies the need for simulation-based testing
-- an infrastructure to test a system that is first formulated as an abstract model is offered by distributed simulation
-- both the System under Test (SUT) and the test device are coupled by exchanging data packets on a network
Network
Test Device
send/receive messages
System Under Test (SUT)
send/receivemessages
Raises the question: How is the Test Device developed in an authoritative manner?
Specified as abstract
model, e.g. UML
Connecting middle ware, e.g. HLA
Problem: Testing Polar Opposite Requirements
To deal with the increasing complexity and advanced decision capabilities of C4ISR systems
=>testing methodology has to become more rigorous, in-depth and thorough
To keep up with the rapid change and short development life cycles expected from the system builders
=> tests have to be ready to conduct in time scales compatible with the agile development strategies of new systems.
Solution: employ formal M&S
• to increase capabilities for simulation-based testing and •as a basis to increase the automation of testing processes.
Testing of interface standards is a focus area for automated simulation-based testing.
Link-16 is required in all Joint and multi-national operations.
The Joint Interoperability Test Command (JITC) has developed an automated test generation methodology as its core technology for testing conformance of systems to Link-16 This methodology is fundamentally enabled by the DEVS formalized modeling and simulation approach
AWACS
TheaterWarning
ABL
DSP/SBIRS
F-15
JLENS
THAAD
PATRIOT
MEADS
ATACMSAVENGER
TEL
AEGIS (CEP)
SIS(MSCS)SIS(MSCS) Link-16specification
The JITC employed M&S-based testing for the initial major milestone evaluation of the Integrated Architecture Behavior Model (IABM) developed by the Joint Single Integrated Air Picture (SIAP) System Engineering Organization (JSSEO) in 2005.
Recent Successful Application
The test exercise produced significant results that uncovered flaws in the model design and added acknowledged value to the model development.
The ACTGen Development Team, NGIT & ACIMS was selected as the winner in the Cross-Function category for the 2004/2005 M&S Awards presented by the National Training Systems Association (NTSA).
Types of Distributed Simulation Testing
Testing Description Example
standards conformance
test whether system conforms to standardsupports interoperability
Tactical Data LinkStandards, Link-16, VMF, USMTF,
interoperability test whether systems can interoperate (at the syntactic, higher, levels)
Joint Translator Forwarder (JXF) – air to/from land exchange of tactical data
mission/capabilities test whether system of systems have capabilities required for mission
Joint Close Air Support (JCAS)
Levels of Distributed Simulation Testing
Level/Variation Description Example
One-on-OneStimulus/Response- managed time- real time
Test Driver sends message stimuli to which SUT responds
IABM SIAP
One-on-Many Federation-------------------------- managed time- real time
Test Driver sends message stimuli and controls supporting federates including sensors
IABM Peer to Peer Architecture
Multiplatform(Many-to-Manyinteraction)-------------------------- managed time- real time
Active mode (test driver controls scenario interaction)
Passive Mode (test manager opportunistically observes interaction
JIT 06-03 live C2 systems from USAF, USMC, Navy in remote locations
Joint Combined Hardware-in-the-Loop (HWIL) Event (JCHE) Phase 3 CNI pilot event “Runs for Record” (~Aug 06) (CNI = Navy Common Network Interface)
JXF tests
Test Driver controls the scenario
Multiplatform Distributed Simulation – controlled testing
Platform(System,
Component)
Platform(System,
Component)
Platform(System,
Component)
Test Driver
Distributed Observers look for opportunities to test
Multiplatform Distributed Simulation - uncontrolled testing
Platform(System,
Component)
Platform(System,
Component)
Platform(System,
Component)
Observer Observer Observer
Test Coordinator
Joint Test Enterprise Requirements
• Must respond to requirements for increasing complexity, evolving from standards conformance, to interoperability, to joint operations capabilities testing
• Top down test development methodology – aim for maximum reuse of test assets by supporting configuration of assets to meet objectives of each new exercise
• Requires support of appropriate concepts and levels of abstraction – e.g., Objectives, capabilities, functionalities, rule sets, test model federates
• Requires development of software at levels of abstraction with appropriate flexibility to interface with configurable middleware and hardware to meet conditions of tests, e.G., High speed/accuracy implies real-time operating system replaces default windows OS
• Develop new assets when no configuration of existing assets can satisfy objectives; then develop with reusability and fit within methodology in mind
Joint Test Enterprise Methodology
• Determine objectives up front and drive test configuration process
• Determine capabilities to be tested• Develop scenarios/mission threads to involve SUT use of
identified capabilities• Identify lower level functions supporting capabilities• Decompose functions into relevant sub sets of rules• Develop testable sequences of rules from relevant rule
sets• Apply automated development of test sequences as test
model federates
Semi-automated test suite design based on Experimental Frames
Standard ProvidesBehaviorRequirements at one or morelevels of SystemSpecification
Reference Master Model
Formalization by mapping into DEVS representationsCorresponding levels ofSystem Specification
Real-timeimplement-ation and execution
Simulation based testing in net-centric environment
Integrated Development and Testing Process
Summary
• Interoperability is mandated throughout the system development process to achieve mission effectiveness
• The emerging challenges of testing in a joint environment require a new paradigm
• A test methodology at the enterprise level that scales to net-centric services on the GIG is needed
• Automation of the generation of test environments is needed – illustrated for tactical data link standards conformance testing