Structure Control in Agent-based Simulation

Structure Control in Agent-based Simulation

Bernard P. Zeigler, Ph.D.,Co-Director,

Arizona Center for Integrative Modeling and Simulationwww.acims.arizona.edu

andJoint Interoperability Test Command

Fort Huachuca, AZ 85613-7051

Outline

• Agent and multi-agent based simulation

• DEVS modeling and simulation

• DEVS support of agents

• Structure change control

• Application to distributed opportunistic testing of complex defense collaborative agent systems

• Some issues and implications

References, Available fromwww.acims.arizona.edu

1. Theory of Modeling and Simulation, 2nd Edition, Academic Press, Bernard P. Zeigler, Herbert Praehofer , Tag Gon Kim ,2000

2. Nutaro, J., Hammonds, P., "Combining the Model/View/Control Design Pattern with the DEVS Formalism to Achieve Rigor and Reusability in Distributed Simulation",

3. Zeigler, B. P., Fulton, D., Nutaro, J., Hammonds, P., "M&S Enabled Testing of Distributed Systems: Beyond Interoperability to Combat Effectiveness Assessment", 9th Annual Modeling and Simulation Workshop, Dec. 8-11, 2003, ITEA White Sands Chapter

4. Zeigler, B.P., Fulton, D., Hammonds, P., Nutoro., J., "Framework for M&S-Based System Development and Testing in Net-centric Environment", in ITEA Journal, Nov, 2005

5. Using Discrete Event Modeling and Simulation to Automate Testing In a Net-Centric Environment, Bernard P. Zeigler, Eddie Mak, Phillip Hammonds, Dale Fulton, Dasia Benson,Kimberly Nunn,

Agent-Based Simulation• some of the simulated entities are agents• explicitly represents specific behaviors of specific individuals• contrast with traditional macro-level aggregated representations• extends object-oriented simulation • facilitates simulation of group behavior in highly dynamic situations• allows study of "emergent behavior" • well-suited to populations of heterogeneous individuals

– vehicles (and pedestrians) in traffic situations – actors in financial markets – consumer behavior – humans and machines in battle fields – people in crowds – animals and/or plants in eco-systems – artificial creatures in computer games

Multi-agent Systems

• A dynamic system might be described as a multi-agent system

• E.g. in a bio cell, agents are used as a metaphor to describe and understand the dynamics within the cell

• enzymes, DNA, and mRNA and repressors interact as autonomous reactive entities

• Suited for parallel and/or distributed simulation

manipulationskills mobility

skills

domainknowledge

perceptionabilities

domains …

navigationskills

beliefmanagementcapability

intentionmanagementcapability

goalmanagementcapability

agentmodel

languageskills

communicationcapabilities

decisionmakingabilities

Spectrum of Agent Properties

structurechangeability

Layered Architecture

Model Specification

Simulation Services

Network /Middleware

How is simulation software different from other software?

• It represents the behavior of dynamic systems whose states are functionally dependent on time

• Properly controlling the flow of time is critical• Simulation software may combine:

– continuous (time-driven) and discrete (event-driven) processes

– actual operating hardware and software representations

– wall clock and {faster/slower} than real time advance

• DEVS = Discrete Event System Specification

• Provides formal M&S framework: specification,simulation

• Derived from Mathematical dynamical system theory

• Supports hierarchical, modular composition

• Object oriented implementation

• Supports discrete and continuous paradigms

• Exploits efficient parallel and distributed simulation techniques

DEVS Background

DEVS Hierarchical Modular Model Framework

Atomic: lowest level model, contains structural dynamics -- model level modularity

Atomic

AtomicAtomic

Atomic

+ coupling

Atomic

Atomic

Atomic

Coupled: composed of one or more atomic and/or coupled models

hierarchical construction

Atomic Models

OrdinaryDifferentialEquationModels

Spiking NeuronModels

Coupled Models

Petri NetModels

Cellular Automata

n-Dim Cell Space

PartialDifferentialEquations

Self Organized Criticality

Models

Processing/Queuing/

Coordinating

ProcessingNetworks

Networks,Collaborations Physical

Space

Some Types of Models Represented in DEVS

can becomponents in a coupled model

MultiAgent

Systems

Discrete Time/

StateChartModels

QuantizedIntegrator

Models

Spiking Neuron

Networks

Stochastic

Models

ReactiveAgent

Models

Fuzzy Logic

Models

JAMES (Java-Based Agent Modeling Environment for Simulation)

• DEVS-based framework facilitates experiments with agents under temporal and resource constraints

• supports – endomorphy, i.e., models which contain internal models about

themselves and their environment

– variable structure models, i.e. models whose description entails the possibility to change their own structure and behavior

– parallel distributed execution

DEVS/RAPs

KIB (Knowledge Interchange Broker) handles

synchronization, concurrency, and timing of interchanged messages

RAP (Reactive Action Package) •defines a tree of possible ways a task may be carried out with associated contingencies•elementary constructs are query and action (command) events•events are asynchronous messages generated internally or externally•RAPs compose hierarchically to provide highly flexible reactive decision making

Testing of interface standards is a focus area for automated simulation-based testing.

Link-16 is required in all Joint and multi-national operations.

The Joint Interoperability Test Command (JITC) has developed an automated test generation (ATC-Gen) methodology as its core technology for testing conformance of systems to Link-16 This methodology is fundamentally enabled by the DEVS formalized modeling and simulation approach

Selected as the winner in the Cross-Function category for the 2004/2005 Department of Defense M&S Awards

AWACS

TheaterWarning

ABL

DSP/SBIRS

F-15

JLENS

THAAD

PATRIOT

MEADS

ATACMSAVENGER

TEL

AEGIS (CEP)

SIS(MSCS)SIS(MSCS) Link-16specification

ATC-Gen Goals and ApproachGoals: • To increase the productivity and effectiveness of standards conformance testing (SCT) at Joint Interoperability Test Command (JITC)• To apply systems theory, modeling and simulation concepts, and current software technology to (semi-)automate portions of conformance testing

Objective: Automate Testing

Capture Specification as If-Then Rules in XML

Analyze Rules to Extract I/O Behavior

Synthesize DEVS Test Models

Test Driver Executes Models to Induce

Testable Behavior in System Under Test (SUT)

Network

DEVS Simulator

Test Driver

HLA

SUT

HLA

Interact With SUT Over Middleware

Discrete Event Nature of Link-16 Specification

Constraints(Exception)Rules

Stop

Modify C2Record for TN

1 2 3

RuleProcessing

Stop, Do Nothing,Alerts, Or jump to other

Transaction

TrackDisplay

Operatordecisions

Validity checking

TransmitMsg

Other ConsequentProcessing

Jumps (stimuli) to other

Transactions of specification

Transaction Level - example P.1.2 = Drop Track Transmit

Preparation Processing

Timeouts

PeriodicMsg

Input to

systemDEVS

Output from

system

t1

t2 t

3t4

Level

3 CoupledSystem

2 I/O System

1 I/O Function

0 I/O Frame

System Theory Provides Levels of Structure/Behavior

ATC Gen Process Overview• Rule Capture in XML

– Analyst interprets the requirements text to extract state variables and rules, where rules are written in the form:

IfIf P is true now ConditionCondition

ThenThen do action A later ConsequenceConsequence

UnlessUnless Q occurs in the interim ExceptionException

• Dependency Analysis & Test Generation– DependencyDependency Analyzer (DA)Analyzer (DA) determines the relationship between rules by

identifying shared state variables

– Test Model GeneratorTest Model Generator converts Analyst defined test sequences to executable simulation models

• Test Driver– Test DriverTest Driver interacts with and connects to SUT via HLA or Simple J

interfaces to perform conformance testing

– Validated against legacy test tools

Test Driver for Controlled Testing

Middleware

Coupled Test ModelCoupled Test Model

Jx1,data1Jx2,data2Jx3,data3

Jx4,data4 Jx1,data1Jx2,data2Jx3,data3

Jx4,data4 Jx1,data1Jx2,data2Jx3,data3

Jx4,data4

Component Test Model

1


2


3

SUT

Test Model Generation for Controlled Testing

Mirroring (flipping) the transactions of a SUT model (system model behavior selected as a test case) allows automated creation of a test model

holdSend(Jx1,data1,t1) holdSend (Jx2,data2,t2)

holdSend (Jx3,data3,t3) waitReceive(Jx4,data4)

receiveAndProcess(Jx1,data1) receiveAndProcess(Jx2,data2)

receiveAndProcess(Jx3,data3) transmit(Jx4,data4)


Jx4,data4

t1 t2 t3 t4time

Test ModelTest Model

SUT Model

Distributed Observers look for opportunities to test

Multiplatform Distributed Simulation - Opportunistic testing

Platform(System,

Component)

Platform(System,

Component)

Platform(System,

Component)

Observer Observer Observer

Test Coordinator

Test Manager for Opportunistic Testing• Replace Test Models by Test Detectors• Deploy Test Detectors in parallel, fed by the Observer• Test Detector activates a test when its conditions are met• Test results are sent to a Collector for further processing

Jx1,data1Jx2,data2Jx3,data3Jx4,data4

Test Detector 1

Test Detector 2

Test Detector 3

SUO ObserverOtherFederates

ResultsCollector

Test Manager

The Test Detector watches for the arrival of the given subsequence of messages to the SUO and then watches for the corresponding system output

• Define a new primitive, processDetect, that replaces holdSend• Test Detector

– Tries to match the initial subsequence of messages received by the SUO– When the initial subsequence is successfully matched, it enables waitReceive (or

waitNotReceive) to complete the test

processDetect(Jx1,data1,t1) processDetect(Jx2,data2,t2)

processDetect(Jx3,data3,t3) waitReceive(Jx4,data4)

receiveAndProcess(Jx1,data1) receiveAndProcess(Jx2,data2)

receiveAndProcess(Jx3,data3) transmit(Jx4,data4)


Jx4,data4

t1 t2 t3 t4time

Test Test DetectorDetector

SUO

Test Detector Generation for Opportunistic Testing

Problem with Fixed Set of Test Detectors

• after a test detector has been started up, a message may arrive that requires it to be re-initialized

• Parallel search and processing required by fixed presence of multiple test detectors under the test manager may limit the processing and/or number of monitor points

• does not allow for changing from one test focus to another in real-time, e.g. going from format testing to correlation testing once format the first has been satisfied

Solution

• on-demand inclusion of test detector instances• remove detector when known to be “finished”• employ DEVS variable structure capabilities• requires intelligence to decide inclusion and removal

Dynamic Inclusion/Removal of Test Detectors

message arrives

add induced test detectors into test set

test detector subcomponent removes its enclosing test detector when test case result is known (either pass or fail)

Test Manager Active Test Suite

Test Control

removeAncestorBrotherOf(“TestControl");

addModel(‘test detector”);addCoupling2(" Test Manager ",“Jmessage",“test detector", “Jmessage");

Example: Joint Close Air Support (JCAS) Scenario

Natural Language Specification

JTAC works with ODA!JTAC is supported by a Predator!JTAC requests ImmediateCAS to AWACS !AWACS passes requestImmediateCAS to CAOC! CAOC assigns USMCAircraft to JTAC!CAOC sends readyOrder to USMCAircraft !USMCAircraft sends sitBriefRequest to AWACS !AWACS sends sitBrief to USMCAircraft !USMCAircraft sends requestForTAC to JTAC !JTAC sends TACCommand to USMCAircraft !USMCAircraft sends deconflictRequest to UAV!USMCAircraft gets targetLocation from UAV!!

AWACS Opportunistic Testing in JCAS

Test Control

CAS Model with AWACSobservation

Initially empty Test Suite

AWACS Opportunistic Testing in JCAS (cont’d)

Test Control adds appropriate Test Detector and connects it in to interface,

Test Control observes CAS request message to AWACS


Test Control passes on start signal and request message

First stage detector verifies request message receipt and prepares to start up second stage


second stage waits for expected response from AWACS to request

First stage detector removes self from test suite


Second stage observes correct AWACS response and removes itself and starts up second part


At some later time, second part of Test Detector observes situation brief request message to AWACS First stage removes itself and starts up second stage


Second stage observes situation brief output from AWACS thus passing test, It removes itself and enclosing Test Detector

Structure Change Agent Architectures

• Structure change:– Agents can add or remove other agents – Agents add or remove coupling between pairs of agents

• Scope of effect:– anywhere in the hierarchical structure– within the children of parent or any ancestor– within their peer group

• Scope of control– any agent can induce structure change– only specialized agents can induce structure change

• Implementation issues:– within same processor– in distributed simulation– in real time

Global Structural Change Examples

Application Domain Global Change Structure change agent

Geographic Global Warming Humans

Social New law Legislature, Supreme Court

Governance Change in party in powerChange in regime

IncumbentCoup leader

Economy Globalization Globalization

Business Change in business platform, e.g. www

Technology innovator

Summary• Structure control is the ability of agents to induce structural change in

themselves or others with the effects of enabling different behaviors under different circumstances.

• It has been an under-considered aspect of intelligent/adaptive properties and the collective behaviors of such agents have yet to be explored.

• Structure change is expressable in modeling and simulation environments based on Discrete Event Systems Specification (DEVS).

• It supports opportunistic testing of complex defense collaborative agent systems.

• Implications for modeling local and global structure transitions in a variety of disciplinary guises were suggested

• which agent capabilities are included or emphasized should depend on questions asked

• environment must be sufficiently rich to challenge selected agent capabilities

agentembedded in environment

agent to agent

interaction

environment

agent-environmentinteraction


skills

domainknowledge

perceptionabilities

domains …

navigationskills




agentmodel

languageskills



interactions


skills

domainknowledge

perceptionabilities

domains …

navigationskills




languageskills



down selection

SIAP agent SACHEM agent

integration/organization

manipulationskills

mobilityskills

domainknowledge

perceptionabilities

domains …

navigationskills




languageskills



Accounting for crashes

• vehicle/car-following conditions for crashes

• weather conditions

• driver perception/mental state

On foot evacuations

• information needed

• daytime location of poplulation

• children in school

• pets

• stay or leave

Documents

Structure Control in Agent-based Simulation