Networked ATR Systems Design Considerations: …jao/Talks/InvitedTalks/060503talk.pdf · Networked ATR Systems Design Considerations: System Performance and Resource Constraints Joseph

Networked ATR Systems Design Considerations: System Performance

and Resource Constraints

Joseph A. O’Sullivan Electronic Systems & Signals Research

LaboratoryDept. Electrical & Systems Engineering

Washington [email protected]

http://essrl.wustl.edu/~jao

Presented June 5, 2003 at ONR Command, Control and Combat Systems Gathering.

Michael D. DeVoreDepartment of Systems and Information

EngineeringUniversity of Virginia

[email protected]://people.virginia.edu/~md9c

Alan Van NevelNAVAIR

China Lake, CA

J. A. O’Sullivan, M. D. DeVore, A. Van Nevel ONR Meeting 6/5/2003Networked ATR Systems Design

2

CollaboratorsCollaborators

Donald L. SnyderDaniel R. FuhrmannRobert PlessNatalia A. Schmid (to WVA)Brandon Westover • MD/PhD in Neuroscience• Charlie Anderson• David Van Essen

Joseph A. O’Sullivan, WUSTLMichael D. DeVore, UVA

Supported by ONRFaculty

Others

StudentsLee MontagninoAndrew Li


3

OutlineOutline• Motivation

- Navy Problems- Dynamic Network Implementations for ATR

• Rate-Recognition Theory- Coding Analogy- Theorems

• Implementation Considerations- Successive Refinement- Communication-Computation

• Conclusions


4





• Conclusions

G ENERARE U UTF RUM

CNO SSG XX FORCENetVision

Enhanced Capabilities•Shared battlefield awareness•Speed of decision making•Effects-based operations•Dispersed force protection•Mission flexibility•Global reach back•Sustainability

Networks •Fully-netted force•Processing and collaboration•Real-time shared knowledge

Multi-Tiered Sensor Field•Scalable - space to sea bed•National/Theater/Organic assets•The Warrior

““The architecture for a The architecture for a fully netted maritime forcefully netted maritime force””

Theater Ballistic

Missile Defense

Theater Air Defense

Command & Control

Naval Fires

Undersea Warfare

Sustainment


6

ATR System ImplementationATR System Implementation

System model accounts for:• Recognition Algorithm• Number of bits to

represent an image• Network throughput• Processor cycles per

instruction• Size of processor local

memory• etc.

Goal: Near optimal use of resources in a dynamic environment

• System operates within constraints- Accuracy, bandwidth, computational capability, throughput

• Demands and resources may change during an engagement- Damage, jamming, new capabilities, preemption, budgeting, etc.

Sensor Data Link

Weapon Link

Shooter Link

•Recce Imagery (SAR, IR, Visible)•Intelligent Bandwidth Compression

Ground Recce/Intel Station Strike Planning System

•Wide Area Cueing (ATC)

•Select Target

Targeting Info Link•Link Target data to TOC(type, location, motion,…)

•Strike Planning•Weapon Selection

•ATR•Reference Library, Aimpoint

Terminal ATR

•Strike plan•Target location•ATR parameters

Network Centric Warfare:The Sensor to Shooter Problem


8

Navy Need/ImpactNavy Need/Impact• Need

– Robust networked processing for automatic target recognition (ATR)

– Ability to leverage real-time sharing of data as envisioned in FORCEnet

» Rapidly evolving system architecture» Evolving time constraints» Evolving set of targets

• Impact– Successively refinable ATR: near optimal performance

subject to dynamic time constraints– ATR system performance subject to dynamic resource

constraints dynamically reconfigurable ATR– Development of system design guidelines based on known

processor/timeline/bandwidth limitations– Extract information content from imagery, maximizing

bandwidth usage


9

ATR Performance and ComplexityATR Performance and ComplexityComparison in terms of:• Performance achievable at a given complexity• Complexity required to achieve a given performanceInformation Theory Basis: Rate-Distortion Theory

Rate-Recognition Theory

• Theoretical curves• Operational curves• Actual performance

Model errorsVarying clutter, backgroundNonideal implementations


10


- Navy Problems- Network Implementations for ATR


• Implementation Considerations

- Successive Refinement- Communication-Computation

• Conclusions


11

Object Recognition AbstractionObject Recognition Abstraction

• Objects as Random Vectors • Recognition as M-ary Hypothesis Testing• Analogy to Random Coding Problems

p(xn) p(xn) p(xn) p(yn |xn)Selectone

Model Memory

…Xn(1) Xn(M)Xn(2) Yn

Xn(h)

h


12

Models for Objects: Physical SignaturesModels for Objects: Physical Signatures• Signatures as realizations of random processes• Joint models for multiple measurements

– Model for underlying physical signature– Models for measurements used to derive templates p(Xn)

M independent templates– Models for measurements of candidate

signatures p(Yn)– Joint distribution given identical

physical signature p(Xn,Yn) with marginal distributions p(Xn) and p(Yn)

– This talk: pairwise i.i.d.• Measure of Size n

– Example: increasing image resolution


13

Example: Radar ATR Example: Radar ATR • Publicly available SAR images from MSTAR program;

over 15,000 images in data set• Model measurements as realizations of stochastic

processes• Estimate model parameters for training

2S12S1 T62T62 BTR 60BTR 60 D7D7 ZIL 131ZIL 131 ZSU 23/4ZSU 23/4


14

Preliminary ResultsPreliminary Results• M-ary hypothesis testing

– results rely on two information rate functions– Chernoff bounds– Rate for minimum probability of error is Chernoff information

(two rate functions cross at zero threshold)

• Capacity results: guaranteed error rate, exponential growth in number of hypotheses– Signatures analogous to random codewords for channel

coding– Recognition problem analogous to communication with

random codebook– Channel reliability function gives growth rate


15

Model for Generic Object Model for Generic Object Recognition SystemRecognition System

p(xn) p(xn) p(xn)

φ

p(yn |xn)Selectone…

ff ff ff…

Xn(1) Xn(M)Xn(2) Yn

Xn(h)

hµm(1) m(2) m(M)


16

Generic ModelGeneric Model

• Objects or Patterns Learned Over Time– Long-term memory– Finite (compressed) representation

• Observation Depends on Object– Randomly selected object– Random observation model– Visual image, sensed pattern, biometrics

• Finite (compressed) Representation of Data


17

Rate Region for Pattern RecognitionRate Region for Pattern Recognition• Let Xn(j) be i.i.d. with i.i.d. elements;M= 2nRc

• Fix compression functions f and φ• Compression ranges 2nRx and 2nRy

• Finite (compressed) representation of data• A rate triple (Rc,Rx,Ry) is achievable if there is

a sequence of such codes whose probability of error goes to zero as n goes to infinity.

• Define mutual information between two random variables U and V to be

( )∑∑=u v

vpupvupvupVUI )()(),(ln),();(


18

Rate Region for Pattern Rate Region for Pattern Recognition: TheoremRecognition: Theorem

• Suppose there exist finite sets U and V and distributions p(u|x) and p(v|y). Then all rate triples satisfying

are achievable• Conversely, if a rate triple is achievable,

then there exist finite sets U and V and distributions p(u|x) and p(v|y) satisfying

);(),;(),;( VUIRYVIRXUIR cyx <>>

);(),;(),;( VUIRYVIRXUIR cyx ≤≥≥


19


p(xn) p(xn) p(xn)

φ


ff ff ff…

Xn(1) Xn(M)Xn(2) Yn

Xn(h)

hµm(1) m(2) m(M)

Un(1) Un(2) … Un(M) Vn


20

Comments on Comments on RateRate––Recognition TheoremRecognition Theorem

• Single-letter characterization• Not surprising … (plausible rate bounds)

• Surprising … (distributed information theory)

• Memory: databases, brains• Image representation: digital, neural firing

patterns• Given representations, find limits• Given recognition rate, optimize representations

);(),;(),;( VUIRYVIRXUIR cyx ≤≥≥


21





• Conclusions


22

ModelModel--Based RecognitionBased Recognition

= argmax p(r | a, θ)a,θ

aθ

^^

Wherer is an observation vectora is a target classθ is a target pose

For fixed a, function p constitutes a target model- Generally estimated from training data- Often of a complexity-restricted class- May be a standard form parameterized by a function

Maximum-likelihood from statistical models

Target Target ClassifierClassifier

ââ=T72=T72


23

Likelihood ApproximationsLikelihood Approximations• Choose a sequence of approximations p1, p2, …

with pn+1 a better approximation than pn

Pr[error | pn+1] ≤ Pr[error | pn]

• Let C(pn) be a measure of average resource consumption or complexity when approximation pn is employed

• Choose sequence so that better approximations involve higher complexity

C(pn+1) ≥ C(pn)• Standard Approach: Static implementation by

selecting n to satisfy constraints on Pr[error | pn] and C(pn)


24

Example: Radar ATR Example: Radar ATR Theory and PerformanceTheory and Performance

• Publicly available SAR images from MSTAR program; over 15,000 images in data set

• Compare performance and database complexity for a variety of data models

2S12S1 T62T62 BTR 60BTR 60 D7D7 ZIL 131ZIL 131 ZSU 23/4ZSU 23/4


25

Recognition Performance Recognition Performance

•Model correctness: validity of statistical likelihood•Model segmentation: parts of data modeled

•Approximation complexity: number of poses

BMP2 Variance Image at 6 Sizes

Fine model of a T72 tank (1/7 relative scale),72 templates totaling 1.1M floats

Coarse model of aT62 tank, 1 template with 16K floats


26

ModelModel--Based RecognitionBased Recognition

FunctionFunctionEstimationEstimation

LL((rr||aa,,θθ)) InferenceInference

Scene and SensorScene and SensorPhysicsPhysics

Training DataTraining Data

ProcessingProcessing

ââ=T72=T720 50 100 150 200 250 300 350 400

7.4

7.6

7.8

8

8.2

8.4

8.6

8.8x 104

Azimuth (degrees)

Raw HRR Raw HRR DataData

SAR ImageSAR Image LogLog--likelihoodlikelihoodfunctionfunction


27

Example ResultsExample Results

•240 variations per model families•10 model families (6 reported)


28

PerformancePerformance--Complexity LegendComplexity Legend

Forty combinations of number of piecewise constant intervals and training window width


29

Example: Approximating LikelihoodsExample: Approximating Likelihoods

••••••

• Model the SAR image of a bulldozer as a function of azimuthri ~ CN(0, σi

2(a,θ) )

• Likelihood function depends on parameter function σi2

• Sequence of piecewise constant approximations

Increasing AngularResolution

↓

Increasing azimuth →

σ 2(D7,θ)


30

Dynamic Dynamic ReconfigurabilityReconfigurability• Seek algorithms that dynamically adjust to fit

requirements- can’t necessarily determine n ahead of time

• Let ∆C(pn+1) be the additional resources consumed using pn+1 assuming problem with pn already solved

• Good designs characterized by

C(pn+1) ≈ C(pn) + ∆C(pn+1)

• Produce a sequence of answers (a1,θ1), (a2,θ2), … with increasing accuracy and resource consumption

C'(pn+1) = C(p1) + ∆C(p2) + … + ∆C(pn+1)- stop when resource allocation exhausted


31

Example: Delta Cost FunctionsExample: Delta Cost Functions• Let cost be average number of bits read from

database• Divide azimuth into Nd non-overlapping

intervals of width d

˜ σ d,i2 θk , a( )=

1d

σi2 θ, a( )dθ

2πkNd

− d2

2πkNd

+ d2

∫Approximations d and d/2

are hierarchically related: ( ) ( ) ( )[ ] ,~,~,~

122,2

2,2

12, 22

aaa kikikid dd ++= θσθσθσ


32

Sequence SelectionSequence Selection• Selection of sequence pn drastically affects the

parametric curve Pr[error | pn] vs. C'(pn)• Good designs decrease error rapidly at start of

sequence- useful results even if search is terminated early- can make use of additional resources if available

• Example: Error probability vs. database communication- Design #1: “Leaf Search”Refine sequential 1.4º intervals

- Design #2: “Breadth First”Divide the most likely interval


33

Example: Search AlgorithmExample: Search Algorithm

Error rate vs. bits transmitted from database to processor

• Classification depends on extent of search

• Eventually, search covers all possibilities

• Breadth-first search quickly finds good solutions (a, θ)

• Small overhead present with ordered searches


34

Other Consumption MeasuresOther Consumption Measures• Network bandwidth is one of many types of

resources• Other average rates of resource consumption:

– Elapsed time per classification– CPU cycles per classification– Database (magnetic) storage per model class– Power dissipation

• Changing resource consumption rates due to:– Variation in application requirements– Reallocation of resources to higher priority tasks– Damaged or offline computation elements– Disrupted communication paths– Power considerations


35

ExampleExample

• Pr[error] as function of:- Power consumption (W)- Processing time (s)

• Constant processor clock frequency (500 MHz)

• Variable network utilization

- From 100Mb/s to 1Gb/s

How does resource availability affect classifier accuracy?


36

ExampleExampleHow can a lack of one resource by accommodated?

Network utilization, processing time, and

power consumption to achieve Pr[error] = 3%

Variable processor clock frequency from 500MHz to

700MHz


37

Networked ATR SystemNetworked ATR System

• Local capabilities plus interaction with remote components

• Communicate observation as well as classification- Human in the loop- Remote contribution to classification when available

• Sequence of partial classifications (an,θn)


38

ConclusionsConclusions

•• New Theorems in RateNew Theorems in Rate--Recognition Recognition TheoryTheory

—— Mutual InformationMutual Information——BoundsBounds—— Joint Compression and Channel CodingJoint Compression and Channel Coding

•• Constrained Implementations of ATR Constrained Implementations of ATR ForceNetForceNet

—— Dynamic resource constraintsDynamic resource constraints—— System PerformanceSystem Performance——System ResourceSystem Resource


39

Project Objectives1.1.5.1 Data Compression

1.1.5.1.3 ATR issues from compression

1.3.2 Automated Resource Allocation1.3.2.1 Resource allocation not performed with respect to operational conditions or network status

4.1 Information Integration Fusion4.1.5 Mathematical issues processing large number of sensors and propagating errors

Networked ATR Systems Design Considerations: System Performance and Resource Constraints

Project Description & Technical Approach

The work will develop system implementation guidelines and ATR algorithms which can operate near optimally under dynamic constraints. The system will adapt to evolving system architectures, time constraints, processor availability and communication bandwidth. A fundamental theoretical goal is the extension of rate-distortion theory for communications to a rate-recognition theory for ATR. A component of this approach will use successively refinable models to allow for efficient and flexible processing.

Project Payoff/Impact on Naval Needs

• Provide flexibility and optimal computing architecture within the Expeditionary Sensor Grid and ForceNet• Enable collaborative and distributed ATR processing in a dynamic environment• Quantifiable ATR performance metrics and probabilities• Robust ATR capabilities

Project Start/Milestones/Funding


40


41

ConclusionsConclusionsNavy/Marine Corps ImpactNavy/Marine Corps Impact

•• Successively Successively refinablerefinable ATR: near optimal performance ATR: near optimal performance

subject to dynamic time constraints subject to dynamic time constraints

•• ATR System performance subject to dynamic resource ATR System performance subject to dynamic resource constraints constraints dynamically reconfigurable ATRdynamically reconfigurable ATR

•• Development of system design guidelines based on Development of system design guidelines based on known processor/timeline/bandwidth limitationsknown processor/timeline/bandwidth limitations

•• Extract information content from imagery, Extract information content from imagery, maximize bandwidth usagemaximize bandwidth usage


42

ATR Problem ScopeATR Problem Scope

• Multiple imaging sensors: LADAR, IR, EO, Multispectral, Hyperspectral, SAR

• Network of sensors, processors, databases • Constraints imposed by system architecture, time constraints, and

target model set• Successive refinability to dynamically tradeoff accuracy and

resources (time, bandwidth, computations, etc.)

Model Database

Scene(source)

ImagingSensor Platform

ATR ProcessorInferred Target

and Pose

ImagingSensor Platform Interconnection

Networks

ImagingSensor Platform


43

Point of ViewPoint of View

Navy Applications TheoryMotivation, Implementation Constraints

Theory: abstraction, idealized models and constraintsSystem design guidelines, target

computations (compression, models, templates)

Simulations, performance bounds


44


p(xn) p(xn) p(xn)

φ


ff ff ff…

Xn(1) Xn(M)Xn(2) Yn

Xn(h)

hµm(1) m(2) m(M)

Un(1) Un(2) … Un(M) Vn


45

Traditional Approaches to ATRTraditional Approaches to ATR• Platform-centric, stovepiped• Subsystems (sensor, communications, data processing)

have independent design and operation• Difficult to prove overall system is operating optimally• Often brittle point solutions

Fixed• System architecture

Sensors, computational hardware, communications,geometry, signal processing blocks (compression, etc.)

• Time constraints• Target model set


46

Single ATR SystemSingle ATR System

Model Database

Scene SensorPlatform

InterconnectionNetwork

ATRProcessor Inferred Target

and Pose

ATR processor solves a sequence of inference problemsMay sport multiple CPUsProblem computations may overlap

Sensor platform collects scene information for processorMay incorporate multiple measurements

Model database represents known objectsMay support multiple resolutions

Network connects sensor, processor, and database


47

Differential EntropyDifferential Entropy

Plot of average differential entropy of r given a=a and θ in [Θ,Θ+∆Θ] as a function of ∆

Networked ATR Systems Design Networked ATR Systems Design Considerations: System Performance Considerations: System Performance

and Resource Constraintsand Resource ConstraintsJ. A. OJ. A. O’’Sullivan, Washington USullivan, Washington U

M. D. M. D. DeVoreDeVore, U. Virginia, U. VirginiaA. Van A. Van NevelNevel, NAVAIR, NAVAIR


49

• Derive statistical models from data, physics, constraintspR|Θ,a(r|θ,a) - conditional data modelpΘ|a(θ|a) – prior probability on orientationp(a) – prior probability on target class

• Recognition: select most likely target• Evaluate performance over test set• Repeat for different models and complexity constraints

Target Target ClassifierClassifier

ââ=T72=T72Given a SAR image r, determine a corresponding target class â∈A and pose θ

Target Recognition ProblemTarget Recognition ProblemStatistical Likelihood ApproachStatistical Likelihood Approach


50

Recognition Performance Recognition Performance

•Model correctness: validity of statistical likelihood•Model segmentation: parts of data modeled

•Approximation complexity: number of poses

BMP2 Variance Image at 6 Sizes

Fine model of a T72 tank (1/7 relative scale),72 templates totaling 1.1M floats

Coarse model of aT62 tank, 1 template with 16K floats


51

Example ResultsExample Results

•240 variations per model family•10 model families (6 reported)

Documents

Networked ATR Systems Design Considerations: …jao/Talks/InvitedTalks/060503talk.pdf · Networked ATR Systems Design Considerations: System Performance and Resource Constraints Joseph