Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
1
Objectives Develop new methods to fuse hard sensor data
Develop understanding of hybrid human
cognition and automated processing for
improved situational awareness
Establish test and evaluation approach and
associated data sets
Create an integration environment for team
collaboration, T&E, and transition
Scientific/Technical Approach Establish a framework for human-centric fusion
Develop a T&E approach progressing from
synthetic data to human in the loop experiments
Create an architecture and infrastructure for
algorithm integration and transition
Design and implement algorithms for fusion of
physical sensor data including new sensor types
Accomplishments Evolved SYNCOIN: new meta-data/sensor data
Generated TML for 4th year demonstrations
Performed new hard sensor data processing
Developed data visualization toolkit
Conducted concept/cognitive analysis
Developed automated inference tools including
Complex Event Processing & Intelligent Agents
Demonstrated utilization of SYNCOIN data in
AXISPro
ta
PSU Overview & SYNCOIN David Hall, Jake Graham, Jeff Rimland, Rick Tutwiler
and Guoray Cai
PSU/UB Sensemaking Architecture
Hard sensor fusion
Visual Analytics Workbench
Cognitive Task Analysis
Complex Event Integration
3
Text Entities
Activities
Locations
Attributes
Human
Observers
Physical
Sensors
Computer
Aided Human
Encoding (HE)
Defined fuzzy
relational operations
Fused Hard
Data (Entities)
Co
mp
ute
r a
ssis
ted
Hu
ma
n m
eta
-da
ta g
en
era
tio
n
RDF-like
triples
Auto Correlation
(ID/loc)
Me
ta-D
ata
Ba
se
Visual Analytics Workbench Hard Sensor Fusion
Analyst input
PIRs
RFIs
I&W
Working
Hypotheses
etc
1. 1.
2.
3.
4.
PSU Processing Flow and
Functions
Overview of PSU 4rd Year
Accomplishments
1. Synthetic hard/soft data set Continued evolution of the SYNCOIN data set
New physical sensor data
New soft and hard sensor data links and meta-data
Conducted human in the loop cognitive task analyses
2. Hard sensor data fusion New algorithms for fusion of hard sensor data
Range imaging tracking, (Interacting Multiple Mode (IMM) Kalman Filters) Tracking
VNIR Color Particle Filter Tracking
VNIR Image fusion and Multi-Model Object characterization
Range/Depth Automated Segmentation Algorithm
3. New automated inference tools Intelligent agents for improved focus of analyst attention
Complex event processing
4. Visualization toolkit Implemented web-based interactive visual analysis (IVA) toolkit
Created relational database to link SYNCOIN geo, temporal and human network data
5. Integration & network based processing Robust cyber-infrastructure for distributed H/S processing
StreamBase CEP, Advanced message queuing protocol (AMQP), RabbitMQ, Open Geospatial Consortium standards for
TML and Event Pattern Markup Language, AchemyAPL, RDF/OWL
Demonstrated SYNCOIN data in AXISPro
Students supported/Degrees Awarded: - 6 graduates/undergraduate students: J. Rimland, D. Chen, R. Grace, A. Godbole, M. Lesniewski, A. Sridara
- 3 Undergraduate students: J. Shields, B. Ripka and G. Traylor
- 4 faculty: D. Hall, J. Graham, M. McNeese, R. Tutwiler, and G. Cai
- Degrees awarded: (MS, PhD): A. Godbole (M.S.) and J. Rimland (PhD)
Publications: - Refereed conference papers 10
- Book and book chapters - 3
- Technical reports 1
- Presentations - 10
Technology Transitions:
- Interactions with industry Distributed SYNCOIN to 11 organizations and individuals
Raytheon Corporation
QuinetiQ (UK)
Aptima
VIStology INC
Saffron Technology
Arctan
Overwatch
- Interactions with other government agencies Network Science Collaborative Technology Alliance (U. of Illinois)
Institute for Defense Analysis
Evaluation of Techniques for Uncertainty Representation Working Group (ISIF)
Project Statistics and Summary
5
SYNCOIN
Conceptual Framework Synthetic soft-sensor database consistent with dominant security environment of Iraq 2010
COIN Inspired story line
Focus on people, events, locations and movements in and around Bagdad
Soft data augmented with physical sensor data collected at Pleasant Gap Fire Safety Facility
Multiple threads and storylines
Ground-truth specification of people, events, activities, locations, timeline
Approach Interpretation of ethno-religious groups, culture and political landscape & interactions with allied forces during OIF
Military aspects of COIN domain (IED events, support networks and motivations)
Realistic without revealing tactical or operational tradecraft
Data augmented with meta-data obtained using students acting in analyst teams
Example Sensor Exercise
Scenario/Theme: IED Attack / Coordinated Sniper Fire
Participants: 21 (PSU+TSU)/ 3 vehicles
Event Days: 3-5
Sensors: 9+ Cameras, 1 Flash LIDAR, 2+ KINECT
Mobile Devices: GeoSuite Mobile App on Android
Event/Activity Synchronization: auditory/visual cues
Each Data Collect requires several weeks of planning/coordination;
3-5 days of data collection; followed by multiple weeks of
processing equating to over .5 TB of data
Cognitive Task Analyses and Meta-
Data Generation
SYNCOIN scene setter
Background assumptions
Command-level tasking
SYNCOIN message set
Derived PIRs and anticipated I&W for Collection
Sequential Processing and Analysis of SYNCOIN messages
Interim Assessment
Tentative Working Hypotheses
Predicted Threat Activity Reports
Evaluation of Observed (Actual) Indicators
Revision and update of Predicted Threat Activity Reports
Meta-data for Enhancement of SYNCOIN Data Set
Student Teams Acting in Analysis Role
9
Hard/Soft Fusion Demonstration
Peter Willet, University of Connecticut
Gavin Powell, ADS Innovation Works, UK, government technical area lead for TA 6 - Distributed Coalition Information Processing
for Decision-Making
David Nicholson, BAE Systems, London, UK
David Dearing, Stottler Henke Associates
David Braines, Hursley Emerging Technology Services
Erick Blasch, Air Force Research Laboratory Sensors Directorate (AFRL/SNAA)
Marco Pravia, BAE Systems
Kamal Premaratne, University of Miami
James Law, SPAWARSYSCEN U. S. Navy Space and Naval Warfare Systems Center
Chase Cotton, Network Science Collaborative Technology Alliance Program (CTA), U. S. Army Research Laboratory
ETURWG Evaluation of Techniques for Uncertainty Representation Working Group, International Society of Information Fusion
(ISIF)
International Technology Alliance
Brian Simpson, Raytheon Corporation
Simon Maskell, QinetiQ, UK
Charlotte Shabarkh, Aptima, Woburn, MA
Brian Ulicny, VIStology, INC, Framingham, MA
Dr. Joan Carter, Institute for Defense Analysis, Alexandria, VA
Network Science Collaborative Technology Alliance, University of Illinois, Champaign, IL
Jim Fleming, Saffron Technology, Cary, NC
Charles Morefield, Arctan, Arlington, VA
Rick Beckett, Overwatch, Textron, Philadelphia, PA
Distribution of SYNCOIN Data
11
Proposed Plans for Year 5
Hard sensor data fusion Complete automated classification of human forms from 2-D and 3-D depth map fused data
Design & implement a multi- -scene characterization)
Final data collect using KINECT merged color tracker and depth map tracker
Automated sense-making algorithms Apply and extend Complex Event Processing/Multi-Agent Systems to SYNCOIN data
Develop coherence network algorithm for SYNCOIN
Link Complex Event Processing/Multi-Agent Systems and Coherence Network processing
Visual analytics Extend current toolkit (e.g., text analysis, recommender support, etc.)
Enable collaborative analysis with multiple distributed analysts
Integrate visual interaction (human-supported cognition) with computer automated processing
Extend cognitive task analysis to improve understanding of analytical reasoning processes
Field a beta version of the toolkit and conduct human in the loop experiments