H.B. Mitchell

Multi-Sensor Data Fusion An Introduction

With 81 Figures and 59 Tables

Springer

Contents

Part I Basics

1 Introduction 3 1.1 Definition 3 1.2 Synergy 4 1.3 Multi-Sensor Data Fusion Strategies 5

1.3.1 Fusion Type 5 1.3.2 Sensor Configuration 6 1.3.3 Input/Output Characteristics 6

1.4 Formal Framework 7 1.4.1 Multi-Sensor Integration 9

1.5 Catastrophic Fusion 10 1.6 Organization 12 1.7 Further Reading 13

2 Sensors .' 15 2.1 Introduction 15 2.2 Smart Sensor 16 2.3 Logical Sensors 17 2.4 Interface File System (IFS) 17

2.4.1 Interface Types 18 2.4.2 Timing 19

2.5 Sensor Observation 20 2.5.1 Sensor Uncertainty Ay 21

2.6 Sensor Characteristics 23 2.7 Sensor-Sensor Properties 23 2.8 Sensor Model 24 2.9 Further Reading 28

X Contents

3 Architecture 29 3.1 Introduction 29 3.2 Fusion Node 31

3.2.1 Properties 32 3.3 Simple Fusion Networks 33

3.3.1 Single Fusion Cell 33 3.3.2 Parallel Network 33 3.3.3 Serial Network 35 3.3.4 Iterative Network 36

3.4 Network Topology 37 3.4.1 Centralized 38 3.4.2 Decentralized 39 3.4.3 Hierarchical 42

3.5 Software 44 3.6 Further Reading 44

Part II Representation

4 Common Representational Format 47 4.1 Introduction 47 4.2 Spatial-Temporal Transformation 50 4.3 Geographical Information System 51

4.3.1 Spatial Covariance Function 54 4.4 Common Representational Format 55 4.5 Subspace Methods 58

4.5.1 Principal Component Analysis 59 4.5.2 Linear Discriminant Analysis 60

4.6 Multiple Training Sets 64 4.7 Software 67 4.8 Further Reading 67

5 Spatial Alignment 69 5.1 Introduction 69 5.2 Image Registration 69

5.2.1 Mutual Information 70 5.3 Resample/Interpolation 74 5.4 Pairwise Transformation T 76 5.5 Image Fusion 77 5.6 Mosaic Image 80 5.7 Software 81 5.8 Fürther Reading 82

Contents XI

6 Temporal Alignment 83 6.1 Introduction 83 6.2 Dynamic Time Warping 85 6.3 Dynamic Programming 86

6.3.1 Derivative Dynamic Time Warping 88 6.3.2 Continuous Dynamic Time Warping 89

6.4 Video Compression 91 6.5 Software 94 6.6 Further Reading 94

7 Sensor Value Normalization 97 7.1 Introduction 97

7.1.1 Sensor Value Normalization 99 7.2 Binarization 99 7.3 Parametric Normalization Functions 103 7.4 Fuzzy Normalization Functions 104 7.5 Ranking 104 7.6 Conversion to Probabilities 107

7.6.1 Platt Calibration 107 7.6.2 Binning 108 7.6.3 Kernels 108 7.6.4 Isotonic Regression 109 7.6.5 Multi-Class Probability Estimates 110

7.7 Software 111 7.8 Further Reading 111

Part III Data Fusion

8 Bayesian Inference 115 8.1 Introduction 115 8.2 Bayesian Analysis 115 8.3 Probability Model 117 8.4 A Posteriori Distribution 118

8.4.1 Standard Probability Distribution Functions 119 8.4.2 Conjugate Priors 119 8.4.3 Non-Informative Priors 122 8.4.4 Missing Data 123

8.5 Model Selection 126 8.5.1 Laplace Approximation 127 8.5.2 Bayesian Model Averaging 129

8.6 Computation 130 8.6.1 Markov Chain Monte Carlo 130

XII Contents

8.7 Software 131 8.8 Further Reading 131

9 Parameter Estimation 133 9.1 Introduction 133 9.2 Parameter Estimation 133 9.3 Bayesian Curve Fitting 137 9.4 Maximum Likelihood 140 9.5 Least Squares 142 9.6 Linear Gaussian Model 143

9.6.1 Line Fitting 145 9.6.2 Change Point Detection 147 9.6.3 Probabilistic Subspace 149

9.7 Generalized Millman Formula 151 9.8 Software 153 9.9 Further Reading 153

10 Robust Statistics 155 10.1 Introduction 155 10.2 Outliers 157 10.3 Robust Parameter Estimation 158

10.3.1 Student-i Function 159 10.3.2 "Good-and-Bad" Likelihood Function 161 10.3.3 Gaussian Plus Constant 164 10.3.4 Uncertain Error Bars 164

10.4 Classical Robust Estimators 167 10.4.1 Least Median of Squares 167

10.5 Robust Subspace Techniques 168 10.6 Robust Statistics in Computer Vision 168 10.7 Software 170 10.8 Further Reading 171

11 Sequential Bayesian Inference 173 11.1 Introduction 173 11.2 Recursive Filter 175 11.3 Kaiman Filter 178

11.3.1 Parameter Estimation 181 11.3.2 Data Association 182 11.3.3 Model Inaccuracies 186 11.3.4 Multi-Target Tracking 188

11.4 Extensions of the Kaiman Filter 188 11.4.1 Robust Kaiman Filter 189 11.4.2 Extended Kaiman Filter 190

Contents XIII

11.4.3 Unscented Kaiman Filter 191 11.4.4 Switching Kaiman Filter 192 11.4.5 Kriged Kaiman Filter 194

11.5 Particle Filter 195 11.6 Multi-Sensor Multi-Temporal Data Fusion 195

11.6.1 Measurement Fusion 195 11.6.2 Track-to-Track Fusion 197

11.7 Software 200 11.8 Further Reading 200

12 Bayesian Decision Theory 201 12.1 Introduction 201 12.2 Pattern Recognition 201 12.3 Naive Bayes' Classifier 205

12.3.1 Representation 205 12.3.2 Performance 206 12.3.3 Likelihood 207

12.4 Modifications 210 12.4.1 Feature Space 210 12.4.2 Model Assumptions 214 12.4.3 Learning Methods 215 12.4.4 Multiple Classifiers 216

12.5 Error Estimation 217 12.6 Pairwise Classifiers 218 12.7 Software 219 12.8 Further Reading 219

13 Ensemble Learning 221 13.1 Introduction 221 13.2 Bayesian Framework 221 13.3 Empirical Framework 224 13.4 Diversity Techniques 225 13.5 Diversity Measures 227

13.5.1 Ensemble Selection 230 13.6 Classifier Types 230 13.7 Combination Strategies 231

13.7.1 Simple Combiners 231 13.7.2 Meta-Learners 236

13.8 Boosting 238 13.9 Recommendations 240

13.10 Software 240 13.11 Further Reading 240

XIV Contents

Part IV Sensor Management

14 Sensor Management 243 14.1 Introduction 243 14.2 Hierarchical Classification 244

14.2.1 Sensor Control 244 14.2.2 Sensor Scheduling 245 14.2.3 Resource Planning 245

14.3 Sensor Management Techniques 246 14.3.1 Information-Theoretic Criteria 246 14.3.2 Bayesian Decision-Making 247

14.4 Further Reading 248 14.5 Postscript 248

Part V Appendices

Software Sources 251

Background Material 253 B.l Probability Theory 253 B.2 Linear Algebra 254 B.3 Square Matrices 255

References 257

Index 277