TOPIC 12: LEVEL 3

David L. Hall

TOPIC OBJECTIVES

Introduce the JDL Level 3 Process Describe modeling, prediction and analysis

techniques for Level 3 Indentify limitations and issues for level 3

processing

COMMENTS ON THIS LECTURE

The lectures on level-2 and level-3 could easily be merged (although it would be a very long lecture), since the methods described for level-3 are a continuation of the automated reasoning methods introduced in the level-2

The examples presented have a DoD/military flavor since an enormous amount of research has been funded in this area – with extensive developments

These examples and methods are easily extendible to non-military applications such as environmental monitoring, public health, disaster relief and other areas

LEVEL 3 PROCESSING (CONSEQUENCE

REFINEMENT)

SITUATION AND THREAT ASSESSMENT FUNCTIONS

a prioriIntelligence Preparation

Technical Doctrinal

Databases

Threat (RISK)

Analysis

Situation Abstractions

Situation Assessment

N

2

1

. . .

Command and

Control Decision-Making

Level 1 Data Fusion

Products

• Construct Representa- tions of Data

• Interpret and Express the Environment

– Objects– Groups– Events– Activities

• Predict Future Courses of Action• Three Perspectives

– White– Blue– Red

Level 2,3 Data Fusion

Products

• Multiple Possible Explanations of Situation and Threat

• Planning• Expected Outcomes of Blue COAs

LEVEL THREE PROCESSING:THREAT REFINEMENT

LEVEL THREE PROCESSING

THREAT REFINEMENTTHREAT REFINEMENT

ESTIMATE/AGGREGATE

FORCE CAPABILITIES (RED/BLUE)

PREDICT ENEMY INTENT

IDENTIFYTHREAT

OPPORTUNITIES

ESTIMATE IMPLICATIONS

• Force vulnerabilities• Timing of critical events• Threat system priorities• Friendly system opportunities

MULTI-PERSPECTIVE ASSESSMENT

• Offensive/Defensive

COMMANDER’S DECISION-

MAKING METHODOLOGY

SENIOR COMMANDER’S

CONCEPT

OWN TROOP’S MISSION

INTERMEDITE MISSIONS

Mission Tree(Objectives

to be hit)

Particular Missions

Analysis of Situation

Concept of Combat Operations

Tactical Missions of Sub-units of Troop Branches

Troop Coordination Procedure

Measures for Political Work Combat Operations Support, and Organization

of Command and Control

OPTIONSELEMENTDECISION

Analysis of Mission

Selection and Formulation of Best Decision Option

ENEMY

OWN TROOPS

ADJACENT UNITS

TERRAIN

HYDROMETEOROLOGICAL CONDITIONS, TIME OF YEAR

RADIATION SITUATION

ECONOMIC CONDITION OF COMBAT OPERATIONS

AREA & SOCIOPOLITCAL MAKE-UP OF POPULATION

SITUATION ELEMENTS

NOTIONAL ENEMY COURSE OF ACTION DISPLAY*

Mountain 1

Road 2

Bridge 2

River 2

Bridge 1

Road 1

Large Barrier

Marsh 1

Forest 1

Lake 1

River 1

Extended Barrier

* Antony, R., Principles of Data Fusion Automation, Artech House, Inc., Norwood, MA, 1995, p. 92.

Richard Antony discusses the “Intelligent Preparation of the Battlefield” concept and related situation and decision-support displays; such concepts are often used in a wide variety of areas including business continuity planning, preparation for disasters & disaster relief; and many large-scale operations

INTELLIGENCE PREPARATION OF THE BATTLEFIELD PROCESS

INTELLIGENCEDATA

IDENTIFIEDENEMY

HOLDINGS

ENEMYDOCTRINE

WEATHERANALYSIS

PERCEIVEDENEMY

SITUATION

TERRAIN ANALYSIS

LIKELYENEMYACTION

AREASOF

INTEREST

COMMANDERDECISIONS

COLLECTIONMANAGEMENT

THREAT EVALUATION THREAT INTEGRATION

Expected Enemy

Reactions to Decisions

ASSESSING THE THREAT/CONSEQUENCES

• WHITE VIEW OF THE BATTLEFIELD ENVIRONMENT• RED VIEW OF THE ENEMY WAR PLAN• BLUE VIEW OF THE FRIENDLY FORCE MISSIONS

GENERAL NOTION OF THE THREAT MODEL:GENERAL NOTION OF THE THREAT MODEL:

WHITE VIEW BLUE VIEW RED VIEW

• Effects of the EnvironmentEffects of the Environment– Weather– Terrain– Political treaties– Communication nets

• Operational ArtOperational Art– Objectives– Offensive– Economy of force– Maneuver– Unity of command– Security– Surprise– Simplicity

• Enemy Battle PlansEnemy Battle Plans– Why– When– Where– Force structure– Objectives– Time table– Options– Tactics– Doctrine

Note: the concept of “shifting” perspectives is a valuable tool in addressing nearly any situation and it’s consequences; what am I planning to do or want to happen (blue view); how might others react to my plans and activities (red view), and how the environment affect both me and others (white view)?

KNOWLEDGE REPRESENTATION

Physical and mathematical models Equations Neural Nets

Language constructs Ontology/Taxonomy Logical constructs (e.g. predicate logic) Examples, Stories and Cases

Analogical models Graphs Trees Special notations (chemical symbols, musical

notes) Diagrams Cognitive Maps Etc

MAJOR REASONING APPROACHES

Knowledge representation Rules Frames Scripts Semantic nets Parametric Templates Analogical methods

Uncertainty representation Confidence factors Probability Dempster-Shafer evidential intervals Fuzzy membership functions Etc.

Reasoning methods & architectures

Implicit methods Neural nets Cluster algorithms

Pattern templates Templating methods Case-based reasoning

Process reasoning Script interpreters Plan-based reasoning

Deductive methods Decision-trees Bayesian belief nets D-S belief nets

Hybrid architectures Agent-based methods Blackboard systems Hybrid symbolic/numerical systems

PATTERN TEMPLATES

Logical Templating Methods Case-Based Reasoning

LOGICAL TEMPLATES

Based on a concept similar to grading tests or papers using a template (score sheet) to quickly determine the number of correct answers

Logical templates can be created including parametric relations, causal factors, sub-entities, etc. to characterize a complex entity, activity or event

Logical template methods are an extension of decision-trees and pattern recognition

Test paper Name ______1) 2)

Answer Sheet

1)2)

Student answers are matched against correct answer sheet to see how many the student got correct

INTELLIGENCE PREPARATION OF THE

BATTLEFIELD TEMPLATESTemplate Description Purpose When Prepared

Doctrinal Enemy doctrinal deployment forvarious types of operations withoutconstraints imposed by weatherand terrain. Composition,formations, frontages, depths,equipment numbers and ratios,and high value targets (HVT) aretypes of information displayed.

Provides the basis for integratingenemy doctrine with terrain andweather data.

Threat Evaluation

Situation Depicts how the enemy mightdeploy and operate within theconstraints imposed by theweather and terrain.

Used to identify critical enemyactivities and locations. Provides abasis for situation and targetdevelopment and HVT analysis.

Threat Integration

Event Depicts locations where criticalevents and activities are expectedto occur and where critical targetswill appear.

Used to predict time-related eventswithin critical areas. Provides abasis for collection operations,predicting enemy intentions, andlocating and tracking HVT.

Threat

DecisionSupport

Depicts decision points and targetareas on interest keyed tosignificant events and activities.The intelligence estimate is ingraphic form.

Used to provide a guide as to whentactical decisions are requiredrelative to battlefield events.

Threat Integration

TEMPLATE PROCESSING

FLOW

B

START

STOP

Receive Triggering Information

Receive Candidate Template

Receive Related Events from Database

Perform Logic Checks

Ending Processes

Make Identification Declaration

Compute MOC

A

A

Make Ambiguity Declaration

YES

NO

YES NO

YES

NO

YES

PassNecessary

Test

PassSufficiency

Test

MOC > TR

MOC > TA

BMore

Templates?

NO

NO

YES

Notes:Notes:

TA = Acceptance Threshold

TR = Rejection Threshold

MOC = Measure of Correlation

SPEEDING TICKET TEMPLATE EXAMPLE

Threat elements Moving violations

Speeding ticket DUI Reckless driving Failure to stop at stop sign

or stoplight Non-moving

violations Illegal parking Failure to have vehicle

inspected Other

Speeding Ticket Threat Template

White Conditions• Speed limit• Visibility• Posted speed limit• Location wrt

known speed traps

• etc

Blue Conditions• Own car speed• Condition of driver• Appearance of driver• Gender of driver• Color of vehicle• etc

Red Conditions• RWR indicator• Visible enemy• COMINT externals• COMINT internals• etc.

Logical relations• If RWR and own car speed >(1.2* speed limit) threat• Etc

WHAT IS CASE-BASED REASONING?

Cases are descriptions of situations and the actions taken to respond to them

Case-based reasoning is an approach to building knowledge systems that: Bases reasoning on retrieval of cases that are similar to the

current situation Supports learning from experience

Reference: J. Dannenhoffer, Case-Based Reasoning, presented to the AIAAA AI Technical Committee, January 1992.

THE CASE-BASED REASONING PROCESS

Reference: J. Dannenhoffer, Case-Based Reasoning, presented to the AIAAA AI Technical Committee, January 1992.

ACCEPT NEW CASE

RETRIEVE RELEVANT CASES

SELECT MOST RELEVANT CASE(S)

CONSTRUCT SOLUTION OR INTERPRETATION OF NEW CASE

VALIDATE SOLUTION/INTERPRETATION

UPDATE MEMORY WITH NEW CASE

PROCESS REASONING

Script Interpreters Rule-bases systems Expert Systems

Plan-based Reasoning

GENERAL CHARACTERISTICS OF RULE-BASED SYSTEMS (PROCESS

REASONING)Application Domain:Application Domain:

Approach:Approach:

Development:Development:

Evaluation:Evaluation:

• Specific fairly narrow real-world problems (poor/missing data)

• Heuristic, rule-based search strategies in general plus facts and computation methods• Knowledge engineering/knowledge representation• Control: data-driven or goal directed• Software: LISP, PROLOG or other script-like

language

• Development support system• Incremental, evolutionary development process

• No absolutes -- experts are evaluators

In the early heyday of AI research (1980s & early 1990s) these types of reasoning were termed “expert” systems

BASIC STRUCTURE OF AN EXPERT SYSTEM

USERUSER

MAN-MACHINE INTERFACE

CONTROL STRUCTURE(RULE INTERPRETER/INTERFACE

ENGINE)

KNOWLEDGE BASE

• Heurtistics• Facts• Algorithms

GLOBAL DATABASE(Dynamic System

Status)

SystemInput

CONCEPTUAL INFERENCE CYCLE

KnowledgeBase (KB)

DynamicData

Search KB

AnyRules

?

Select Rule

Done ?

Quit

Quit

YESYES

NONO

NONO

Fire/Execute Rule

• Update Dynamic Data• Execute Sub-Routine• Request Input Data• Etc.

KNOWLEDGE ENGINEERING

MILITARY/DOMAIN EXPERTMILITARY/DOMAIN EXPERT

• Military Organization/Protocols• Rules of Engagement• Military Doctrine• Military Equipment Characteristics

– Weapons– Electronics

• Communications

KNOWLEDGE BASEKNOWLEDGE BASE

• Scenario• Rule Base• Tree Constructs• Database Design• Facts/Algorithms

SOFTWARE ENGINEERSOFTWARE ENGINEER

• Development Support System• Soft Programming• Software Architecture• Computer Environment• Numerical Techniques

PLANNING/GOAL DECOMPOSITION

Planning provides another effective way to represent knowledge including timelines, roles and responsibilities, hierarchies of plans, causality, etc.

Planning analogies have been used effectively for automated reasoning including course of action analysis tools, impact analysis, decision trees, hypothesis evaluation, gaming methods and more recently team-based intelligent agents

Plan A Plan B

GOAL DECOMPOSITION

ACCOMPLISH MISSION

ATTACK TARGET SURVIVE THREAT

Detect Candidate

Target

Evaluate Target

Determine Attack Tactic

Identify Threat

Monitor Threat

Determine Threat Tactic

Specify Target

Target of Opportunity

Acquire Target

Select Weapon

Select Attack Profile

Revise Plan

Estimate Range

Bearing

Infer Status

Intention

Avoid Threat

Suppress Threat

Revise Plan

CONCEPT OF GOAL/PLAN HIERARCHY

Known Target Locations

Defend againstTarget

MILITARY GOALS

Reconnaissance CoordinationBlockage

Feint

Surveillance StrikeTankOperations

Damage Assessment

CounterMeasures

SINGLE-AGENT PLAN LIBRARY

MULTI-TARGET MISSION TEMPLATE LIBRARY

Monitor Mission Defend Mission

• • •

• • •

• • •

Destroy Target

Attack Mission

DEDUCTIVE METHODS

Decision Trees Bayesian Belief Nets

Belief networks Bayesian networks Causality nets, etc.

Dempster-Shafer Belief Nets

BAYESIAN BELIEF NETS Representation of relationships or causality via Bayesian probability

(publicized by Judea Pearl in 1988) Knowledge contained in

Directional acyclic graph Nodes represent variables Links express (parent/child) relationships (e.g. causal

relationships) Each node has a conditional probability relation specified

Knowledge propagation via Bayesian chain rule Network as a whole represents the joint probability distribution Note: Bayesian Networks also called Markov Chain

See for example the tutorial at: http://www.cs.ubc.ca/~murphyk/Bayes/bayes.html

BAYESIAN BELIEF NETS

A

B C

D

E

Example of a directed acyclic graph

BAYES NET FOR TARGET IDENTIFICATION

Target, No Target

Land Sea Air

Target Type(Tgt1, Tgt2, …, Non-Tgt)

Target Activity(launch, hide, reload, move)

Target Dimension

CommEquipment

RadarType

CommActivity

Radar Activity

Length(IMINT)

Width(IMINT)

Frequency(COMINT)

Duration(COMINT)

PRI(ELINT)

Frequency(ELINT)

• Evidence can be injected into any node in the form of a likelihood function This increase propagates to the parent nodes and the children nodes Propagation continues until all nodes have been updated• The sum of probabilities in a set of children equals that of the parent

Example provided by KC Chang via M. Liggins

HYBRID METHODS

Blackboard Systems Agent-based Architectures Hybrid Symbolic/Numeric Systems

EXAMPLE: BLACKBOARD ARCHITECTURE CONCEPT

S

HA

RE

D M

EM

OR

YS

HA

RE

D M

EM

OR

Y

PROBLEM DOMAINPARTITION N

PARTITION 3

PARTITION 2

PARTITION 1

KA

KA

KA

KA

KB

KB

KB

KB

CONTROL STRUCTURE CONTROL STRUCTURE

EXTERNAL INTERFACEEXTERNAL INTERFACEHUMAN COMPUTER INTERFACEHUMAN COMPUTER INTERFACE

KA = KNOWLEDGE AGENTKB = KNOWLEDGE BASE

SUMMARY OF AGENT ATTRIBUTES

Bradshaw (1997) lists the following as possible agent attributes:•Reactivity. the ability to selectively sense and act•Situatedness. being in continuous interaction with a dynamic environment, able to perceive features of the environment important to them, and effect changes to the environment.•Autonomy. goal-directedness, proactive and self-starting behavior•Temporal continuity. persistence of identity and state over long periods of time•Inferential capability. can act on abstract task specifications using prior knowledge of general goals and preferred methods to achieve flexibility, goes beyond the information given, and may have explicit models of self, user situation, and/or other agents•Adaptivity. being able to learn and improve with experience•Mobility. being able to migrate in a self-directed way from one host platform to another across a network

SUMMARY OF AGENT ATTRIBUTES (CONT.)

• Social ability - the ability to interact with other agents(and possibly humans) via some kind of agent-communication language, and perhaps cooperate with others.

• Knowledge-level'' communication ability. the ability to communicate with persons and other agents with language more resembling human-like “speech acts” than typical symbol-level program-to-program protocols

• Collaborative behavior. can work in concert with other agents to achieve a common goal

Wooldridge and Jennings [4] add the following as possible agent attributes:

• veracity - an agent will not knowingly communicate false information

• benevolence - agents do not have conflicting goals, and that every agent will therefore always try to do what is asked of it, and

• rationality - an agent will act in order to achieve its goals, and will not act in such a way as to prevent its goals being achieved — at least insofar as its beliefs permit

INTELLIGENT AGENT AUTOMATED REASONING

En

vironm

ent

Sensors

Effectors

Agents perception of world

What world will be like if I do actions

A, B, F ….

Internal Model of World State

How the world is changing

How actions change world

Action to performGoalsE

nviron

men

t

Sensors

Effectors

Agents perception of world

What world will be like if I do actions

A, B, F ….

Internal Model of World State

How the world is changing

How actions change world

Action to performGoals

Agent Characteristics

•Wish agent to be pro-active

•Agent maintains a list of one or more goals

•A goal is a description of a desirable situation (state of the world)

•Actions are chosen so as to achieve the goals

•Deliberative – agent needs to reason about the actions to take to achieve goals

•Goal achievement may involve long sequences of actions – may involve extensive search and planning

Goal-based reactive agents can be developed to emulate human-like behavior for information search and understanding

• Multi-agent logic language for encoding teamwork (MALLET)

Act onInfo Needs

IdentifyInfo Needs

(DIARG)

InformationNeeds

Responsibilities(Petri Nets)

TeamKnowledge(MALLET)

ResponsibilitySelection

Belief

DomainKnowledge

(JARE)

BeliefUpdate

Information

See research by Dr. John Yen at http://agentlab.psu.edu/

Example of Team-Based Intelligent Agents to Support Data Fusion

Information Fusion 2+

Information Fusion 1

Information Fusion 1

Information Fusion 1

Team DecisionContext

Computational SMMContext

Shared MentalModel

Information Fusion 2+

Information Fusion 1

Information Fusion 1

Information Fusion 1

Team DecisionContext

Computational SMMContext

Shared MentalModel

TOPIC 12 ASSIGNMENTS

Preview the on-line topic 12 materials Read Wark and Lambert chapter 11 referenced

above Read chapter 2 in Mlidinow (2008)

DATA FUSION TIP OF THE WEEK

Level 3 processing is ultimately about consequence prediction – assisting a user/analyst in determining how the current situation may evolve (i.e., alternate hypothetical futures), how these alternative futures may affect the current situation, how to identify potential decisions and how to evaluate the consequences of alternative decisions. We need to seek a balance between providing insight for the analyst/decision-maker without inducing “analysis paralysis”.