Upload
carmel-sanders
View
227
Download
2
Embed Size (px)
Citation preview
Motion Tracking
Image Processing and Computer Vision: 8 2
Introduction Finding how objects have moved in
an image sequence Movement in space Movement in image plane
Camera options Static camera, moving objects Moving camera, moving objects
Image Processing and Computer Vision: 8 3
Contents Acquiring targets
Image differencing Moving edge detector
Following targets Matching Minimum path curvature Model based methods Kalman filtering Condensation Hidden Markov Model
Image Processing and Computer Vision: 8 4
Camera calibration revisited Image to camera co-ordinate
transformation Intrinsic parameters
Camera to world co-ordinate transformation Extrinsic parameters
Image Processing and Computer Vision: 8 5
Camera to Image Co-ordinates Distortionless Camera
If no distortions uniform sampling
Co-ordinates linearly related offset and scale
s yo yyimy
sxoxximx
Image Processing and Computer Vision: 8 6
Camera to Image Co-ordinates Distorting Camera
Periphery is distorted
k2 = 0 is good enough
yxr
rkrkydy
rkrkxdx
222
42
211
42
211
Image Processing and Computer Vision: 8 7
Pinhole Camera
Image
ObjectOpticalcentre
Image and centre, object and centre are similar triangles.
f Z
Z
Yfy
Z
Xfx
Image Processing and Computer Vision: 8 8
World frame
Camera frame
translate androtate
Camera and World Co-ordinates
Image Processing and Computer Vision: 8 9
System architecture
Acquiretarget
Followtarget
Start End
Image Processing and Computer Vision: 8 10
Target acquisition Finding a target to follow
Differencing Moving edge detector
Image Processing and Computer Vision: 8 11
Change and Moving Object Detection Simplest method of detecting
change Compute differences between
Live and background images Adjacent images in a sequence
Image Processing and Computer Vision: 8 12
Image Differencing Differences due to
Moving object overlying static background
Moving object overlying another moving object
Moving object overlying same moving object
Random fluctuation of image data
Image Processing and Computer Vision: 8 13
Image Processing and Computer Vision: 8 14
Demo Inter frame differencing Difference from a background
pFinder
Image Processing and Computer Vision: 8 15
Background image Detecting true differences required
an accurate background Lighting changes? Camera movement?
Image Processing and Computer Vision: 8 16
Background image updates Periodically modify whole
background Will include changes in new
background Systematically incorporate non-
changed portions of image into background BBelse
LBBthenbackgroundLif
titi
tititi ti
1,,
,1,, ,
Image Processing and Computer Vision: 8 17
Critique Can identify changes in the image
data But what do the changes mean?
Need a second layer of processing To recognize changes Optical flow sidesteps this problem...
Image Processing and Computer Vision: 8 18
Target following Observing the positions of an
object or objects in a time sequence of images.
Object matching Minimum path curvature Model based methods
Image Processing and Computer Vision: 8 19
Matching Locate objects in each image Match objects between images Use methods of previous lectures
Image Processing and Computer Vision: 8 20
Minimum path curvature Observations of two objects in three
images
Image Processing and Computer Vision: 8 21
Image Processing and Computer Vision: 8 22
Image Processing and Computer Vision: 8 23
Image Processing and Computer Vision: 8 24
Image Processing and Computer Vision: 8 25
Which is “best” solution? One with overall straightest paths
For each solutionFor each path
Compute total curvature
Sum
Image Processing and Computer Vision: 8 26
Model based tracking Mathematical model of objects’
motions: position, velocity (speed, direction),
acceleration Can predict objects’ positions
Image Processing and Computer Vision: 8 27
System overview
MotionModel
VerifyLocation
PredictLocation
Update?
Image Processing and Computer Vision: 8 28
Newton’s laws
s = position u = velocity a = acceleration
all vector quantities measured in image co-ordinates
tautsts 2
0
21
Simple Motion Model
Image Processing and Computer Vision: 8 29
Prediction Can predict position at time t
knowing Position Velocity Acceleration
At t=0
Image Processing and Computer Vision: 8 30
Uncertainty If some error in a - a or u - u Then error in predicted position -
s
tautsts 20 2
1
Image Processing and Computer Vision: 8 31
Verification Is the object at the predicted
location? Matching
How to decide if object is found Search area
Where to look for object
Image Processing and Computer Vision: 8 32
Object Matching Compare
A small bitmap derived from the object vs.
Small regions of the image Matching?
Measure differences
Image Processing and Computer Vision: 8 33
Search Area: Why? and Where? Uncertainty in knowledge of model
parameters Limited accuracy of measurement Values might change between
measurements Define an area in which object
could be Centred on predicted location, s s
Image Processing and Computer Vision: 8 34
Update the Model? Is the object at the predicted
location? Yes
No change to model No
Model needs updating Kalman filter is a solution
Image Processing and Computer Vision: 8 35
Kalman filter
Mathematically rigorous methods of using uncertain measurements to update a model
Image Processing and Computer Vision: 8 36
Kalman filter notation Relates
Measurements y[k] e.g. positions
System state x[k] Position, velocity of object, etc
Observation matrix H[k] Relates system state to measurements
Evolution matrix A[k] Relates state of system between epochs
Measurement noise n[k] Process noise v[k]
Image Processing and Computer Vision: 8 37
Mathematically
k
ky
ky
kx
kx
ky
kx
kkkk
n
nxHy
0
0
1
0
0
0
0
1
How do observations relate to model?
Image Processing and Computer Vision: 8 38
Prediction of System State Relates system states at epochs k
and k+1
k
kky
kky
kkx
kkx
T
T
kky
kky
kkx
kkx
kkkkkk
v
vxAx
|
|
|
|
1000
100
0010
001
|1ˆ|1ˆ
|1ˆ|1ˆ
||1ˆ
Image Processing and Computer Vision: 8 39
Prediction of Observation From predicted system state,
estimate what observation should occur:
k
kky
kky
kkx
kkx
kky
kkx
kkkkkk
n
nxHy
|1ˆ|1ˆ
|1ˆ|1ˆ
0
0
1
0
0
0
0
1
|1ˆ
|1ˆ
|1ˆ|1ˆ
Image Processing and Computer Vision: 8 40
Updating the Model Predict/estimate a
measurement Make a measurement Predict state of model How does the new
measurement contribute to updating the model?
kk |1ˆ y
1ky kk |1ˆ x
Image Processing and Computer Vision: 8 41
Updating the Model
G is Kalman Gain Derived from A, H, v, n.
kkkkk
kkkkkkk
|1ˆ1|1
|11|1ˆ1|1ˆ
yyy
yGxx
covariancesystem
|||1
|1
|
C
GHCCC
nHHCHCG
kkkkkk
TkkTkk
Image Processing and Computer Vision: 8 42
Example Tracking two corners of a minimum
bounding box Matching using colour Image differencing to locate target
Image Processing and Computer Vision: 8 43
Condensation Tracking So far considered single motions What if movements change?
Bouncing ball Human movements
Use multiple models plus a model selection mechanism
Image Processing and Computer Vision: 8 44
Selection and Tracking
Occur simultaneously Maintain
A distribution of likely object positions plus weights
Predict Select N samples, predict locations
Verify Match predicted locations against image Update distributions
Image Processing and Computer Vision: 8 45
Tracking Using Hidden Markov Models Hidden Markov model describes
States occupied by a system Possible transitions between states Probabilities of transitions How to recognise a transition
Image Processing and Computer Vision: 8 46
Optic Flow Assume each pixel moves but does
not change intensity Pixel at location (x,y) in image 1 is
pixel at (x+x,y+y) in image 2. Optic flow associates displacement
vector with each pixel
Image Processing and Computer Vision: 8 47
t
Iu
y
Iv
x
I
t
I
t
y
y
I
t
x
x
I
tt
Iy
y
Ix
x
I
Ott
Iy
y
Ix
x
ItyxI
ttyyxxItyxI
x
0
0
0
,,
,,,,
2
Image Processing and Computer Vision: 8 48
Aperture Problem
I/ x horizontal difference I/ y vertical difference I/ t difference between imagesOne equation, two unknowns cannot solve equationCould solve for movement
perpendicular to gradient
Image Processing and Computer Vision: 8 49
Solution
Impose smoothness constraint:
Minimise total of sum of squares of velocity component gradients
image y
v
x
vy
u
x
u 2222
Image Processing and Computer Vision: 8 50
EED
EvEuEPD
PEvv
D
PEuu
yx
taverageyaveragex
yaverage
xaverage
222
is a constantIterate over a pair of frames, orover a sequence
Image Processing and Computer Vision: 8 51
Critique Assumptions
Linear variation of intensities Velocity changes smoothly
These are invalid Especially at object boundaries
Image Processing and Computer Vision: 8 52
Area Based Methods Match small regions in image 1
with small regions in image 2 Assume objects move but do not
deform Same formulation as for optical flow,
different areas involved….
tyxIttyyxxI ,,,,
Image Processing and Computer Vision: 8 53
Matching
Compute (x, y) by finding (u,v) that minimises
Then (u,v) = (x, y)
wi
wi
wj
wjtjyixIttjvyiuxIvuD ,,,,2
,
Image Processing and Computer Vision: 8 54
Summary Target acquisition
Image differencing Background model
Target following Matching Minimum path curvature Model based methods Optic flow
Image Processing and Computer Vision: 8 55
This “telephone” has too many shortcomings to be seriously considered as a means of communication. The device is of no value to us.Western Union internal memo, 1876