Insight - Home | The National Academies of Sciences ...sites.nationalacademies.org/cs/groups/pgasite/documents/webpage/... · Insight National Academies Board on Global Science and

Insight

National Academies Board on Global Science and Technology

Committee on Integrating Humans, Machines and Networks:

A Global Review of Data-to-Decision Technologies

Washington, D.C.

February 26, 2013

Ben Cutler

Program Manager

Distribution Statement A Approved for Public Release, Distribution Unlimited

Distribution Statement A Approved for Public Release, Distribution Unlimited 2

• Combine data across multiple sources

• Manage efficient use of sensors and platforms

• Identify threats using behavioral discovery and prediction algorithms

• Collaborate

Insight: one unified global ISR picture

An adaptable, integrated human-machine ISR exploitation system

Doppler

GISR_2011-04-22_1280x1024_1000Kbps_C001_withDistro_70.wmv


3

• Data from diverse sources

• Ambiguity/uncertainty of information

• Who? What? Where? When? Why?

• Pedigree, provenance

• Report of an observation (delivery of electrons) may be received long after the event

• Sequence neutrality – obtain the same results regardless of the order in which data is integrated

• Tactical relevance requires real-time insights

• Real-time analysis

• Real-time data sharing

• Data analytics at massive scale

• High computation requirement per unit data

• Fleeting existence of a critical piece of information buried in a sea of data

• “Bad guys hiding in the noise floor”

Challenges

4

• Threat networks include the following elements

• Movement

• Command and Control

• Intelligence (includes collection and deception)

• Logistics

• Fires (IEDs, artillery, air defenses, aircraft)

• Protection (physical defenses, air defenses, electronic systems)

• Identification by

• Signatures

• Physical characteristics indicate this aircraft is a fighter jet

• Activity Based Intelligence

• Pattern of life (e.g., activities, associations) indicate a group of people appears to be an insurgent cell

Finding threat networks


Vast sea of data

Find the threat network (all names fictional)

5 Distribution Statement A Approved for Public Release, Distribution Unlimited

6

• Correlate unusual activity with historical information and intelligence estimates

• Determine past and current locations of people of interest, vehicles, and other artifacts, and estimate patterns of life

• Detect deviations from normalcy that warrant further analysis

• Set up tripwires or watch boxes that alert when an event partially or fully matches a pattern of interest based on some combination of space, time, and network relationships

• Alert the analyst to any meetings in the south end of town

• Task assets to fulfill information requirements

• And alert the analyst when an information requirement (information that is needed for understanding or for making timely decisions) is fulfilled

• Situational analysis: what has happened, what is happening, what is likely to happen?

Some system requirements for finding insurgent networks



Data to information: VIRAT interpretation of video

Activity types Digging Carrying Walking

Video data collected from Creech AFB, March 2009

8

Data to information: wide area motion imagery example

• 25 cm GSD

• Six cameras with ortho-rectified (stitched and geo-

registered) imagery

• NITF file format

with encoded sensor metadata


AFRL RYA LAIR Dataset, 21 October 2009 WAMI Data

9

Tracking result

Wide area

motion imagery example

Tracking result


10

Relevant information content (ground truth)

~6,500 tracks in 7 minutes


11

• Raw sensor data

• ~200 megapixels/second

• Pixel format and interpretation is sensor-specific

• Generally, we just want information about the entities in the video

• Where are the cars and people?

• Features (e.g., large, red sedan)

• Activities – what are they doing? (e.g., car making a u-turn)

• Information: “tracks”, one per active entity

• In this example: 200 megapixels/second => 15 tracks/second

• Tracks distill data into computable information that Insight can process

• Track format accepted by Insight is standard across motion imagery sensors

• Model-based presentation allows downstream algorithms to be sensor-agnostic

• Track information may include feature or activity information

• Very large wide area motion imagery sensors

• 10s of gigapixels/second

• 10+ kilotracks/second

Data to information: wide area motion imagery


• Real-time information integration and automated exploitation

• Multi-echelon, integrated, theater through tactical unit level support

• Sensor ISR: space, air, and ground

• Other information sources: soldier information, reports, CI/HUMINT, SIGACTs, IPB,

demographics, friendly locations (BFT), OSINT, law enforcement, commercial sources, etc.

Diverse information sources

12

• Automated resource

management for

dynamic tasking and

cross-cueing based on

intelligence

requirements

• Direct support

systems

• Organic systems

• General support

systems


13

Operational concept


Insight World Model

14

Object types provided by Intel Object types created by reasoning components


15

Data conditioning

• Physical sensors have unique

reference frames – space/time bias

• Observer location, perspective

• Observed entity locations

• Incomplete data

• Obscuration

• Omission (e.g., HUMINT uncertainty)

• Duplicate data

• Multiple tracks or detections for a single entity

• Conflicting elements of a textual report

• Conditioning

• Provides a common reference frame

• Estimates incomplete or conflicting data

4

Performance example - Blue Devil HDBlue Devil video frame overlay onto PeARL foundational image map

Using raw sensor metadata

Example - Blue Devil HDBlue Devil video frame overlay onto PeARL foundational reference map

Unregistered - using raw sensor metadata

PeARLReference

Blue Devil Unregistered

5

Blue Devil Registered

PeARLReference

Example - Blue Devil HDBlue Devil video frame overlay onto PeARL foundational reference map

Registered - using CMREG corrected metadata

Blue Devil HD video

frame overlay on

PeARL foundational

reference map

(using raw sensor metatdata)

(using corrected metadata)

unregistered error

registered

Simulated platform

trajectories (with biases)

-250 -200 -150 -100 -50 0 50 100 150 200 250-200

-150

-100

-50

0

50

100

150

200

X (km)

Y (

km

)

lon

git

ud

e

latitude -250 -200 -150 -100 -50 0 50 100 150 200 250

-200

-150

-100

-50

0

50

100

150

200

X (km)

Y (

km

)

latitude

lon

git

ud

e

Simulated platform

trajectories (with biases mitigated)

simulated platform Distribution Statement A Approved for Public Release, Distribution Unlimited

16

An example with biases 1 vehicle, 3 sensor tracks with biases

Tracks of the same vehicle

produced by 3 geographically dispersed ground-based GMTI radars

1. Beacon 2. Harry 3. Quarry


An example with biases mitigated Track fusion after data conditioning

4 radars track different segments of a road network

Fusion provides a clear picture of vehicle activity on the road network

17

Tracks from 4 geographically dispersed ground-based GMTI radars

1. Beacon

2. Harry 3. Quarry 4. Brigade

Fused tracks


18

Dynamically-generated traffic patterns provide context for comparing against real-time intelligence

• Traffic normalcy modeling

• Track starts, stops, and velocity distributions

• Vehicle/people track counts and

activity

• Compare to real-time fused tracks to detect anomalies

• Unusually fast vehicles or isolated track start/stops

• Unusual vehicle/people activity

• Correlate with other information sources (e.g., historical

information/SIGACTs, other sensor observations, etc.)

Traffic Pattern Analysis

All fused tracks Dynamically-generated traffic patterns

Heavy traffic Moderate traffic Light traffic True detections of isolated track starts/stops

False alarms of isolated track starts/stops

Detections of an isolated track start/stop

True detections of track starts/stops


• EDL is the entry point for source information (e.g., HUMINT, SIGACTs, network information)

• EDL produces enriched graph fragments

• Brings in context information to enable correlation of source data

• Discovers roles and links

• Processes data (tracks, locations,

associations) and metadata (probabilities, pedigree)

• Enriched graph fragments are used to build up the dynamic graph

• Insight and 3rd party applications can process enriched data

• Analysts are able to navigate

source evidence in a broader context

EDL enriches source information to build up linked evidence

19

Element Discovery & Labeling (EDL)

Meetings Known associations

Tracks Communications

Enrich

ed a

bst

ract

ion levels


Reasoning engines enable top-down exploration and bottom-up alerts to activities of interest

20

Analyst-vetted cases

New anomaly

Analyst-vetted cases

Resolved anomaly

Bottom-up inference management

Top-down hypothesis management

Patterns Pattern matches Hypotheses

• Increases analyst ability to

manage hypotheses derived

from uncertain data

• Continuously monitors evidence

streams to detect analyst-

defined patterns of interest

• Detects emerging threats

buried in multiple, complex,

high-volume data streams

• Explains patterns discovered in

the data

• Prioritizes and explains

anomalies using analyst-vetted

cases

• Focuses analyst attention on

anomalies of

importance/relevance

Evidence stream


Analyst workstation at FT-1

21 Distribution Statement A Approved for Public Release, Distribution Unlimited


FT-1: geospatial, network, and evidence manager views

Efficient Overview, Alerting, and Navigation

23

• Alert overview provides a summary of open information requests, their status, and related

information products • Incoming alerts allow rapid drill down to more detailed information views


Lightweight evidence management tools

24

• The analyst can rapidly organize, mark up, and revise hypotheses and connections

between entities and activities

• Touch-point for seeding and reviewing results of automated reasoning capabilities


25

Insight field test data volumes (GB)

132.7 TB database available at https://www.vdl.afrl.af.mil/ as the Global ISR project site Distribution Statement A

Approved for Public Release, Distribution Unlimited

https://www.vdl.afrl.af.mil/



26

Community use of Insight field test data

Unique, high-fidelity truthed data sets that have been made available to ISR researchers across

the Defense and Intelligence communities – 475 researchers as of January 2013

Industry 339

FFRDCs & National Labs (MIT/LL, LLNL, SDL)

18

IC (NGA) 3

JIEDDO 1

DARPA 13

Air Force (AFIT, AFLCMC, AFRL, ASC, ESC)

86 Navy (ONR, SPAWAR, NSWC) 8

Army (ARL, CERDEC I2WD, CERDEC NVESD)

7

Breakout of Global ISR Project Site Users


27

• Data from diverse sources

• Ambiguity/uncertainty of information

• Who? What? Where? When? Why?

• Pedigree, provenance

• Report of an observation (delivery of electrons) may be received long after the event

• Sequence neutrality – obtain the same results regardless of the order in which data is integrated

• Tactical relevance requires real-time insights

• Real-time analysis

• Real-time data sharing

• Data analytics at massive scale

• High computation requirement per unit data

• Fleeting existence of a critical piece of information buried in a sea of data

• “Bad guys hiding in the noise floor”

Summary


www.darpa.mil


[email protected]

Documents

Insight - Home | The National Academies of Sciences ...sites.nationalacademies.org/cs/groups/pgasite/documents/webpage/... · Insight National Academies Board on Global Science and