Uncertainty Aware Semantics for Information Fusion

Kathryn Laskey

George Mason University C4I and Cyber Center and SEOR Department

Fusion 2017 Plenary SessionXi’an, China, July 2017FUSIO

N 2017

1

Once upon a time…• Fusion systems were stovepipes

– Used by a single organization for a single purpose

– Built on idiosyncratic database schema and input-output formats

– Labor-intensive manual transformation of outputs for use by another stovepipe

• Focus was only on objects and their physical properties– Reports closely tied to physical features of objects– Conclusions concerned object states & trajectories– Higher level fusion mostly left to humans

• Expressivity was very limited– Handled only standardized messages, special-case scenarios,

specific sensor types

– Semantics were implicit and hardwiredFUSION 20

17

2

What the Future Demands• Reason about complex situations

– Many objects and actors interacting in space and time• Accept heterogeneous reports

– At multiple levels of JDL hierarchy– From large numbers of diverse, geographically dispersed sensors– Differing semantics– Both hard (physical domain) and soft

(informational domain)– Hypotheses of interest only indirectly tied to reports

• Manage pervasive uncertainty• Interoperate with a wide variety of systems using

diverse standards, protocols and processes– Share information– Coordinate actions across space and time– Operate across security levels, organizational and national boundaries

• Respond rapidly to changes in problem context and environment

FUSION 20

17

3

Canonical Level 1 Fusion Problem• World contains a physical object having:

– Type– Physical attributes– Kinematic state

• Sensors provide noisy observations of state and/or attributes

• Objective is to infer one or more of: – Past, present & future

trajectory of state – Type of object– Attributes of object FUSIO

N 2017

4

Canonical HLIF Problem• World contains an evolving situation:

– Has a type– Contains entities

• with attributes and states• having relationships to each other

– Has a state which includes states of component entities

• Sensors provide noisy observations – Directly or indirectly related to attributes and states of entities

• Objective is to infer: – Type of situation– Attributes and states of entities– Past, present & future trajectory of stateFUSIO

N 2017

5

Relational Representation and HLF• Formal semantic models (ontologies) are based on relational

representation:– Entities of different types…– with attributes and behaviors…– related to each other

• Some key aspects to represent:– Space and time– Human, social, cultural and

behavioral (HSCB) factors– Observables and their relationship

to hypotheses of interest

• These elements may all be uncertain

FUSION 20

17

6

Uncertainty and Semantics• Traditional semantic models cannot

represent uncertainty• Traditional uncertainty models cannot

represent domain semantics• Statistical relational models extend relational

representations to include uncertainty• A probabilistic ontology uses a statistical

relational model to express uncertainty within a semantically rich representationFUSIO

N 2017

7

Bayesian Network

… and some domain semanticsFUSIO

N 2017

8

Another Bayesian Network

Repeated structure: 2 objects, 2 timesteps

FUSION 20

17

1 region1 vehicle1 timestep

1 region2 vehicles2 timesteps

1 region2 vehicles1 timestep

2 regions3 vehicles2 timesteps

FUSION 20

17

10

A Multi-Entity Bayesian Network Model FUSIO

N 2017

11

Some Challenges• Combinatorics and tractability• Efficient models for interesting special problems• Models and reasoning at multiple levels of

resolution• Modeling soft factors

– Lack of universally accepted theory and models– Ground truth often nonexistent or infeasible to obtain– Sample sizes often too small for reliable statistical

estimation• Combining models and algorithms based on

very different semantics and assumptionsFUSION 20

17

12

In Conclusion• Probabilistic ontologies provide sufficiently rich

representation to support automation across JDL levels– Represent domain semantics– Solid theoretical foundation– Learn from experience– Support for uncertainty management– Fuse hard and soft information– Efficient and scalable inference

• Many important research challenges remain

FUSION 20

17