Upload
cathleen-collins
View
230
Download
3
Embed Size (px)
Citation preview
Data Fusion- A review
Data Fusion- A reviewLayout
1-Benefits of multisensor devices2-Typical sensors used in data fusion3-Sensor performance4-Data fusion models5-Decision fusion in parallel sensor suite6-Comparison of mathematical tools in data fusion
Data Fusion- A review
Benefits of Multiple sensor devices:
-Reduction in measurement time-A downtime reduction and an increase in reliability-Redundant and complementary information-A higher signal-to-noise ratio-A reduction in measured uncertainty-A more complete picture of the environment
Data Fusion- A review
Sensor Output format Applications
Optical sensor Image Mobile robot guidance
Radar Pulse signal Target detection and target tracking
Infrared sensor Image Object identification
Satellite Image Surveillance and pattern recognition
Ultrasonic sensor Pulse signal Mobile robot guidance
NDT sensor Voltage Materials examination
Sonar Pulse echo Obstacle detection
Laser Image Pattern recognition
X-ray Image Medical
Survey of typical sensors used in Data fusion
Data Fusion- A review
Data Fusion- A reviewSensor performance
POD : Probability Of Detection
PFC: Probability of False Call
ROC: Receiver Operating Characteristic(POD versus PFC)
Sensor performance can be statistically represented using:
Major advantage of ROC curves compared to POD curves: In ROC curves false calls are taken into accountBut…In practice, ROC curves are difficult to realise.
Data Fusion- A reviewSensor performance
The performance and potential of each used sensor needs to be established in order to assign weight of evidence,for example, in sensor data fusion.
The most common sources of uncertainty-little or no knowledge about measurement-incomplete measurement (when data are approximated rather than waiting for complete data which may be time consuming and costly)-limitations of the system
Data Fusion- A reviewSensor performance
Common types of errors
-ambiguous
-incomplete
-incorrect
-measurement
-random
-systematic
-reasoning
- practicality (environment)-Human error-Equipment malfunction-False negative-False positive
-Incorrect output-Unreliable-No output
-Calibration error-Precision-Accuracy
-Inductive error-Deductive error
Data Fusion- A reviewData Fusion Models
-Multisensor data integration and fusion center-Three-level fusion paradigm-Centralized signal detection system-Distributed (decentralized) signal detection system
[X.E. Gros, NDT Data Fusion, 1997]
Data Fusion- A reviewData Fusion Models
Data Processing
Assignment of Bayes or Dempster-
Shafer Rule of
Combination
Sensor Data
Selection
Y1
Y2
Yn
Z1
Z2
Zn
OptimumEstimationDecisionLevelFused
Data
Integration Fusion Integration
X1
X2
Xn
Raw Data
Sensor 1
Sensor 1
Sensor 1
ProcessedData
Multisensor data integration and fusion center
Measurements from n sensors are integrated, data is then processed withevidental reasoning, probabilistic and belief theories, the results are classifiedand selected before a decision on the optimum fused information is made.
Data Fusion- A reviewData Fusion Models
Three-level fusion paradigm
Level of evidence
Level of Dynamics
Signal Level
Database
Sensors
DataFusion
DecisionFusion
Features fusion
Data Fusion- A reviewData Fusion Models
Distributed (decentralized) signal detection system
Fusion Center
GlobalDecision
Level
Measurement
Measurement
Measurement
Sensor 1
Sensor 2
Sensor N
Feature Extraction
Level
Local Decision
Local Decision
Local Decision
Fuse identity declarations using Bayesian theory, the Dempster-Shafer paradigm or Thomopoulos generalized evidence processing (GEP).The output from each sensor is a decision which forms the inputs to a fusion center where association is performed.
Data Fusion- A reviewData Fusion Models
Centralized signal detection system
Fusion Center
DecisionLevel
Measurement
Measurement
Measurement
Sensor 1
Sensor 2
Sensor N
More suitable for fusion of raw data but the association phase can be difficult .
Data Fusion- A reviewData Fusion Models
Four major sensor network types
-Serial-Parallel-Parallel-Serial-Serial-Parallel
Sensor 1
Sensor 2
Sensor n
Data Fusion- A review
Data Fusion- A review
Sensor 1
Sensor 2
Sensor j
Sensor t
Local
processor 1
Local
processor 2
Local
processor j
Local
processor t
Data set Z1
Data set Z2
Data set Zj
Data set Zt
Decision
Fusion
Processor
OutputO1
OutputO2
OutputOj
OutputOt
Consistentdecision acrossthe suite?
Yes
No
A recursive processing structure for enhanced performance with aparallel sensor suite. B.V. Dasarathy, 1991, IEEE
Decision fusion in parallel sensor suite
Data Fusion- A review
Sensor output can be regarded as a decision arrayof n decisions. The efficiency of each sensor, , is the probabilityof correctness of the decision Dj from sensor j, a measure of the effectiveness of a sensor.
Cj and Wj : the belief that the decisions from sensorj are correct and wrong (based on the Dempster-ShaferTheory)Uj: the ignorance (uncertainty) of a measurand
j
1
1 ( )
j j
j j j
C W
U C W
Data Fusion- A review
ck , wk , uk : incremental probabilities of the joint correct, incorrect decisions and nondecisions respectively
1 1 1
2 2 2 1
1
1 1
1
1
k k k k
k k
i i ki i
c w u
c w u u
c w u u
c w u
1k k kC W u
1
k
k ii
C c
1
k
k ii
W w
Decision fusion in parallel sensor suite
( 1)1 1
1
( 1)1 1
1
ki
ki
ki
ki
C c u
W w u
( 1) 11
1 1
1
1
kki
i
uu
u
Data Fusion- A reviewDecision fusion in parallel sensor suite
1 1max max max max
1 1 1 1
1( ) ( )k k k k
c wC W C W
c w c w
1 11 1 1
1 1
(1 ) (1 )0 1
(1 ) (1 )
k k
k k
u uC c W w u
u u
Data Fusion- A reviewDecision fusion in parallel sensor suite
Data Fusion- A review
Second sensor
Decision
First sensor Decision
Target Nontarget Undecided
Target Target Undecided Undecided
Nontarget Undecided Nontarget Undecided
Undecided Undecided Undecided Undecided
A simple decision fusion rules matrix
Decision fusion in parallel sensor suite
Data Fusion- A review
1 1 2
1 1 2
1 1 2 1 2
1
(1 )(1 ) 1
( ) 2 1
: '
c
w
u
sensor s correct decision rate
1 1 2
1 1 2
min( , )
min((1 ), (1 ))
c
w
sensor1
sensor2
Local processor 1
Local processor 2
Decision Fusion
processor
Decision fusion in parallel sensor suite
Data Fusion- A review
Perfect Sensor Case: ηi = 1 ( i = 1,2 )
1
1
1
max
max
1
0
1
1
0
k
k
c
w
u
C
W
η : the efficiency of the imperfect sensor
The fused decision approaches the correct decision asymptotically even though one of the sensors may remain imperfect and the user does not know which one it is.
Decision fusion in parallel sensor suite
Data Fusion- A review
Bad Sensor Case: ηi = 0 ( i = 1,2 )
1
1
1
max
max
0
1
0
1
k
c
w
u
C
W
Fusion leads to complete failure of the system.Therefore no totally faulty sensor can beallowed to operate indefinitely in a two-sensor fusion system of this type.
Decision fusion in parallel sensor suite
Data Fusion- A review
Equally Imperfect Sensor Case: ηi = η ( i = 1,2 )2
1
21
1
2 2
max2 2max
1
(1 ) 1
2 (1 ) 1
1 2
1 2 2 1 2 2k
c
w
u
C W
)1(2ln
/)12)(1(ln
k
Minimum number of recursions needed for the fused decision to bebetter than the decision derived by the individual sensor:
Decision fusion in parallel sensor suite
Data Fusion- A review
Fused correct (c1), incorrect (w1) and non-decision (u1) rates vs. sensorefficiency (η)
Initial (c1,w1) and final (Ck |max , Wk |max )fused decision rates vs. sensor efficiency
u1|max=0.5
Decision fusion in parallel sensor suite
c,wu
c1,w1,Ck |max,Wk |max
η η
Data Fusion- A review
Fused correct decision rate (Ck ) vs. sensor efficiency ( η) at different numbers of recursions (k)
Fused correct decision rate (Ck ) vs. recursion number (k) at different sensor efficiencies (η)
Decision fusion in parallel sensor suite
Ck
η=0.1 η=0.2η=0.3η=0.4η=0.5η=0.6η=0.7η=0.8η=0.9
k
Ck
η
Data Fusion- A review
General Case
1 2 1 2
21
1
21
2
2max
max( , ) min( , ) 0 1
1
(1 )(1 ) 1
[(1 ) 2 ] 1
2 ( 1) 1k
c
w
u
C
Asymptotic fused correct decisionrate( Ck|max ) vs. sensor efficiency( η ) at different sensor performanceratios (α )
Decision fusion in parallel sensor suite
η
Ck |max
η1 and η2 are related by α
Data Fusion- A review
1 1 1 1 11 1
1 (1 ) 1 1 ( )t t
j jj j
c w u c w
Decision fusion in parallel sensor suite
Asymptotic fused decision efficiency ( Ck | max )vs. number of sensors ( t) for different sensorefficiencies (η )
t
Ck |max
η=0.1 η=0.2η=0.3η=0.4η=0.5η=0.6η=0.7η=0.8η=0.9
Data Fusion- A review
Equally Imperfect Sensor Case: ηi = η ( i = 1,2,…,t )
tt
t
ktt
t
k
tt
t
t
QP
r
q
p
)1(
)1(
)1(
1)1(1
1)1(
1
maxmax
1
1
1
tt
t
k)1(1ln
}/)1{(1)1(ln )1(
Minimum number of recursions needed for the fused decision to be better than the decision derived by the individual sensor:
Decision fusion in parallel sensor suite
Data Fusion- A reviewDecision fusion in parallel sensor suite
Data Fusion- A review
11
1t
jj
c
11 1
1
11
t
jj
tw
m
1 1 11 1u c w
Decision fusion in parallel sensor suite
m: the number of hypothesis
Data Fusion- A reviewDecision fusion in parallel sensor suite
The minimum number of sensors for the correct fused decision rate to exceed the incorrect fused decision rate:
1/1ln
1lnmin m
mt
The asymptotic values of the fused decision rates:
,
, 1k Max t
f mC
f m
1
, 1
t
k Max tW
f m
11, tmmf
Data Fusion- A review
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
w1 :majority c1:majority
w1 :consensus
c1 consensus
Initial fused decision rates vs. sensor efficiency with three sensors(comparison of the consensus and majority based fusion methods)
c1, w1
η
Binary Decision making
Decision fusion in parallel sensor suite
Data Fusion- A review
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
w1 :majority c1 majority
w1 :consensus c1 consensus
Initial fused decision rates vs. sensor efficiency with three sensors(comparison of the consensus and majority based fusion methods)
c1, w1
η
Multihypothesis Decision making (m=3)
Decision fusion in parallel sensor suite
Data Fusion- A reviewDecision fusion in parallel sensor suite
Data Fusion- A reviewFusion Methodology
The most common data fusion and integration methods
Fusion method ApplicationsPixel level fusion Image processing, image segmentation
Bayesian theory Decision making between multiple hypotheses
Demspter-Shafer theory of evidence Decision making, Beliefs intervals
Neural Network Signal interpretation
Neyman-Pearson criteria Decision making
Fuzzy Logic Handle vagueness
Knowledge based system Pattern recognition
Markov random field Image processing
Data Fusion- A reviewFusion Methodology
Classical Inference
The most common inference approaches based on an observed sample of data for acceptance or rejection of a hypothesis:
-Maximum a posteriori-Likelihood ratio criterion-Neyman-Pearson test-Bayes criteria
Data Fusion- A reviewFusion Methodology
Classical Inference
Maximum a posteriori
Compares two probabilities assigned to two hypothesis and favors either one or the other depending only on their chance of occurrence.
)|()|( 10 yHpyHp
y is an observation from a sensor and Hi a hypothesis i
Data Fusion- A reviewFusion Methodology
Classical Inference
Likelihood ratio criterion
A test to decide between hypothesis H0 or its alternative Hi . If Λ(u)>t , H0 is true otherwise, H1 is true..
tHup
Hupu i
n
ii
)|(
)|()(
10
0
1
H0 and H1 are hypothesis 0 and 1, n the number of sensors, ui random observed sample data and t, the threshold (significance level) determined from experiment.
Likelihood ratio =(level of sufficiency)
Λ(u) : the degree to which the observation of evidence u influences the Prior probability of H
Data Fusion- A reviewFusion Methodology
Classical Inference
Neyman-Pearson hypothesis test
A general theory used to make a decision between two hypothesis. Hypothesis H0 is rejected if the following equation is verified:
tHu
Hu
)|(
)|(
1
0
The threshold t is chosen depending on the risk the user is prepared to take to accept or reject H. the smaller the value of t, the lower the risk.
Data Fusion- A reviewFusion Methodology
Classical Inference
Bayes criteria
A cost function based on false alarm and probability of detection is used to select between two hypotheses H0 and H1. P0 and P1 are a priori probabilities which govern the decision output.
The cost function C for each decision outcome:-C00 : the cost function assigned to the decision 0 when the true outcome is 0 P(H0|H0) :the probability associated with this decision- C01 : the cost function assigned to the decision 0 when the true outcome is 1 P(H0|H1) :the probability associated with this decision- C10 : the cost function assigned to the decision 1 when the true outcome is 0 P(H1|H0) :the probability associated with this decision- C11 : the cost function assigned to the decision 1 when the true outcome is 1 P(H1|H1) :the probability associated with this decision
Data Fusion- A reviewFusion Methodology
Classical Inference
Bayes criteria
The expected values of the cost as the risk R is defined as :
00 0 0 0 01 0 0 1 10 1 1 0 11 1 1 1( | ) ( | ) ( | ) ( | )R C P P H H C P P H H C PP H H C PP H H
The decision intervals are defined as:
00 1 01 11
1 0 10 00
( | ) ( )( )
( | ) ( )( )
Hp y H p H C C
p y H p H C C
Where the right hand side is the threshold of the test and should be such that the cost is as small as possible.
Data Fusion- A reviewFusion Methodology
Bayesian Inference
Used to estimate the degree of certainty of multiple sensors providing information about a measurand.
Uses a priori probability of a hypothesis to produce an a posteriori Probability of this hypothesis.
Suppose there are n mutually exclusive and exhaustive hypotheses H0…Hn that an event E will occur.The conditional probability p(E|Hi) states the probability of an event E that Hi is true and is given by:
( | ) ( )( | )
( )i i
i
p E H p Hp H E
p E
p(Hi) : a priori probability of the hypothesis Hi
p(Hi|E): a posteriori probability of having E given that Hi is true
Data Fusion- A reviewFusion Methodology
Bayesian Inference
If multiple sensors are used…
0 10 1
( | ) ( | )... ( | ) ( )( | ... )
( )n
nj
j
p H E p H E p H E p Ep E H H H
p H
Data Fusion- A review
Bayesian Fusion
iii APABP
APABPBAP
)()|(
)()|()|(
•Target location and tracking •Search for formation of targets in a region•…
Fusion Methodology
Data Fusion- A review
Example: Two sensor data fusion
x: to be identified (e.g. aircraft)
2,1101 iyYY iii
Latest data set Old data set Current measurement
ionnormalisatYxPYxP
YYxPYxPYxPYYxP
)|()|(
)|()|()|()|(
20
10
20
10
21
112
11
1
Fusion Methodology
Data Fusion- A reviewFusion Methodology
Bayesian Inference
Some limitations:
-no representation of ignorance is possible-prior probability may be difficult to define-result depends on choice of prior probability-it assumes coherent sources of information-adequate for human assessment (more difficult for machine-driven decision making)-complex with large number of hypotheses-poor performance with non-informative prior probability (relies on experimental data only)
Data Fusion- A review
Dempster-Shafer Evidental reasoning
Fusion Methodology
Often described as an extension of the probability theory or a Generalization of the Bayesian inference method.
Frame of discernment Θ={X0, X1 , …Xn}Mass probability (basic probability assignment (bpa)) : m(X)
0 ( ) 1
( ) 1
( ) 0
i
X
m X
m X
m
Data Fusion- A review
Dempster-Shafer Evidental reasoning
Fusion Methodology
: 2 [0,1]
( ) ( )Y X
Bel
Bel X m Y for each X
Bel(X) : the degree of support
Properties of the belief function:
( ) 1
( ) 0
0 ( ) 1
( ) ( )
( ) ( ) 1
Bel
Bel X if X
Bel X if X and X
Bel X m X for each X containing only one element
Bel X Bel X
Data Fusion- A review
Dempster-Shafer Evidential reasoning
Fusion Methodology
Dempster rule of combination:
1 2
1 2 3 1 1 2 2( ) ( ) ( ) ( )X X Z
m m Z m Z K m X m X
Data Fusion- A review
…
Dempster-Shafer Evidental reasoning
Geometrical representation of Dempster rule of combination
m1,2 (Xi,Xj)
m2(Xn)
m2(Xj)
m2(X1)
1
0
0 1
m1(X1) m1(Xi) m1(Xn)
Fusion Methodology
Data Fusion- A reviewFusion Methodology
Dempster-Shafer Evidental reasoning
0 Belief Disbelief 1
Plausibility
Incertitude
[Bel(X),Pls(X)] Decision
[0,1] Total ignorance, no belief in support of X
[1,1] Proposition X is completely true
[0,0] Proposition X is completely false
[0.4,1] Partial belief, tends to support X
[0,0.7] Partial disbelief, tends to refute X
[0.3,0.5] Both support and refute X
Data Fusion- A review
Dempster-Shafer FusionGives a rule for calculating the confidence measure of each state,based on data from both new and old evidence.Assigns its masses to all of the subsets of the entities thatcomprise a system
Fusion Methodology
•Mobile robot map building (e.g. occupancy grid)
)()()()(1
)()()()()()()(
OmEmEmOm
OmUmUmOmOmOmOm
osos
ososos
{occupied, empty, unknown} :{O,E,U}m:confidence in each elementms:confidence from sensorsmo:confidence from old existing evidence
Data Fusion- A reviewFusion Methodology
Dempster-Shafer Evidental reasoning
Some features:
-An overestimation of the final assessment can occur
-Small changes in input can cause important changes in output
-High efficiency with bodies of evidence in pseudo-agreement
-Lower efficiency with bodies of evidence in conflict
Data Fusion- A review
Bayesian Fusion vs. Dempster-Shafer Fusion
Bayes: •Works with probabilities, numbers that reflect how often an event will occur•Less calculations.
Dempster-Shafer:•Considers a space of elements that each reflect not what Nature chooses, but rather the state of our knowledge after making a measurement.•Calculations tend to be longer.•Allows more explicitly for an undecided state of our knowledge.(in military it is sometimes far safer to be undecided than to decide wrongly)•Sometimes fails to give an acceptable solution.
Fusion Methodology
Data Fusion- A reviewFusion Methodology
Fuzzy Logic Inference Technique
Is very flexible and there is no universal rule of formalism which can be associated with it.
Fuzzy logic evaluates qualitatively a signal from a sensor and fuzzy sets associate a grade (numerical value) to each element.
Element Associated value Associated reliability
Signal high [1.0,0.7] Certain
Signal medium [0.7,0.3] Uncertain
Signal low [0.3,0.0] Incorrect
Typical associated values for different elements in fuzzy logic
Data Fusion- A reviewFusion Methodology
Fuzzy Logic Inference Technique
A multilevel system to handle vagueness:
-sensor level-data fusion level-reasoning level
Produce information
Integrate information
Generates a decision making use of artificialintelligent systems
Fuzzy logic methods can be very useful to represent uncertainty from multiple sensors and to handle vagueness.
Data Fusion- A reviewFusion Methodology
Fuzzy Logic Inference Technique
Combining information from multiple images to improve classification accuracy of a scene where images are processed at the pixel level using segmentation algorithm .
Can be performed for… image processing and image smoothing image segmentation to combine information perceived by visual sensors.
Fusion center
Data Fusion- A reviewFusion Methodology
Artificial Intelligence
AI techniques developed for data association make use of expertsystems and neural networks
Artificial Neural networks (NNs) are software simulated processing units or nodes, which are trained in order to solve problems.
NNs can be very useful to solve problems in applications where it is difficult to specify an algorithm.They are composed of interconnected nodes that act as independent processing units
Data Fusion- A reviewFusion Methodology
Artificial Intelligence
0
n
i ii
y f w x
node
Input xi
Weights wi
A two-layer neural network, Perceptron
Output signal
Data Fusion- A reviewFusion Methodology
Artificial Intelligence
Some NN applications in data fusion
-For sensor data fusion for detection and correct classification of space object maneuvers observed by radar of different frequencies and resolution.
-Used in decision systems for target tracking, object detection, recognition and classification in defence applications
-In image processing operations such as filtering and segmentation
-To select matching pixel based fusion from sensors for robotics application. To perform pixel-to-pixel image association for object identification
-Applied to non-destructive examination for eddy current signal classification and automatic tube inspection,defect characterisation, classification of weld defects and signal interpretation.
Data Fusion- A reviewA Review on Decision Fusion Strategies
Acknowledgements:
• This powerpoint presentation was prepared by Miss Mahdavi and Miss Bahari former M.Sc. Students at School of ECE , University of Tehran in Dec. 2005 where here is highly appreciated.