Upload
nelle-bullock
View
47
Download
4
Tags:
Embed Size (px)
DESCRIPTION
Motion estimation from image and inertial measurements. Dennis Strelow and Sanjiv Singh. On the web. Related materials: these slides related papers movies VRML models at: http://www.cs.cmu.edu/~dstrelow/northrop. Introduction (1). micro air vehicle (MAV) navigation. - PowerPoint PPT Presentation
Citation preview
1
Motion estimation from image and inertial measurements
Dennis Strelow and Sanjiv Singh
2
On the web
Related materials:
these slides
related papers
movies
VRML models
at:
http://www.cs.cmu.edu/~dstrelow/northrop
3
Introduction (1)
micro air vehicle (MAV) navigation
AeroVironment Black Widow AeroVironment Microbat
4
Introduction (2)
mars rover navigation
Mars Exploration Rovers (MER) Hyperion
5
Introduction (3)
robotic search and rescue
RhexCenter for Robot-Assisted Search and Rescue, U. of South Florida
6
Introduction (4)
NASA ISS personal satellite assistant
7
Introduction (5)
Each of these problems requires:
6 DOF motion
in unknown environments
without GPS or other absolute positioning
over the long term
…and some of the problems require:
small, light, and cheap sensors
8
Introduction (6)
Monocular, image-based motion estimation is a good candidate
In particular, simultaneous estimation of:
multiframe motion
sparse scene structure
is the most promising approach
9
Outline
Image-based motion estimation
Improving estimation
Improving feature tracking
Reacquisition
10
Outline
Image-based motion estimation
refresher
difficulties
Improving estimation
Improving feature tracking
Reacquisition
11
Image-based motion estimation: refresher (1)
A two-step process is typical…
First, sparse feature tracking:
Inputs: raw images
Outputs: projections
12
Image-based motion estimation: refresher (2)
13
Image-based motion estimation: refresher (3)
Second, estimation:
Input:
Outputs: 6 DOF camera position at the time of each
image 3D position of each tracked point
projections from tracker
14
Image-based motion estimation: refresher (4)
15
Image-based motion estimation: refresher (5)
Algorithms exist
For tracking:
Lucas-Kanade (Lucas and Kanade, 1981)
16
Image-based motion estimation: refresher (6)
For estimation:
SVD-based factorization (Tomasi and Kanade, 1992)
bundle adjustment (various, 1950’s)
Kalman filtering (Broida and Chellappa, 1990)
variable state dimension filter (McLauchlan, 1996)
17
Image-based motion estimation: difficulties (1)
So, the problem is solved?
18
Image-based motion estimation: difficulties (2)
If so, where are the automatic systems for estimating the motion of:
in unknown environments?
from images in unknown environments?
19
Image-based motion estimation: difficulties (3)
…and for automatically modeling
rooms
buildings
cities
from a handheld camera?
20
Image-based motion estimation: difficulties (4)
Estimation step can be very sensitive to:
incorrect or insufficient image feature tracking
camera modeling and calibration errors
outlier detection thresholds
sequences with degenerate camera motions
21
Image-based motion estimation: difficulties (5)
…and for recursive methods in particular:
poor prior assumptions on the motion
poor approximations in state error modeling
22
Image-based motion estimation: difficulties (6)
151 images, 23 points
23
Image-based motion estimation: difficulties (7)
24
Outline
Image-based motion estimation
Improving estimation
overview
image and inertial measurements
Improving feature tracking
Reacquisition
25
Improving estimation: overview
26
Improving estimation: overview
27
Improving estimation: image and inertial (1)
Image and inertial measurements are highly complimentary
Inertial measurements can:
resolve the ambiguities in image-only estimates
establish the global scale
28
Improving estimation: image and inertial (2)
Images measurements can:
reduce the drift in integrating inertial measurements
distinguish between rotation, gravity, acceleration, bias, noise in accelerometer readings
29
Improving estimation: image and inertial (3)
30
Improving estimation: image and inertial (4)
31
Improving estimation: image and inertial (5)
Other examples:
• global scale typically within 5%
• better convergence than image-only estimation
32
Improving estimation: image and inertial (6)
Many more details in:
Dennis Strelow and Sanjiv Singh. Motion estimation from image and inertial measurements. International Journal of Robotics Research, to appear.
33
Outline
Image-based motion estimation
Improving estimation
Improving feature tracking
Lucas-Kanade
Lucas-Kanade and real sequences
The “smalls” tracker
Reacquisition
34
Improving feature tracking: Lucas-Kanade (1)
Lucas-Kanade has been the go-to feature tracker from shape-from-motion
iteratively minimize the intensity matching error…
…with respect to the feature’s position in the new image
35
Improving feature tracking: Lucas-Kanade (2)
Additional heuristics used to apply Lucas-Kanade to shape-from-motion:
task: heuristic:
choose features to track high image texture
detect mistracking or occlusion
convergence and matching error
handle large motions image pyramid
36
Improving feature tracking: Lucas-Kanade (3)
Lucas-Kanade advantages:
fast
subpixel resolution
can handle some large motions well
uses general minimization, so easily extendible
37
Improving feature tracking: Lucas-Kanade (4)
0.1 average pixel reprojection error!
38
Improving feature tracking: Lucas-Kanade and real sequences (1)
But Lucas-Kanade performs poorly on many real sequences…
39
Improving feature tracking: Lucas-Kanade and real sequences (2)
…and image-based motion estimation can be sensitive to errors in feature tracking
40
Improving feature tracking: Lucas-Kanade and real sequences (3)
41
Improving feature tracking: Lucas-Kanade and real sequences (4)
42
Improving feature tracking: Lucas-Kanade and real sequences (5)
43
Improving feature tracking: Lucas-Kanade and real sequences (6)
Why does Lucas-Kanade perform poorly on many real sequences?
the heuristics are poor
the features are tracked independently
task: heuristic:
choose features to track high image texture
detect mistracking or occlusion
convergence and matching error
handle large motions image pyramid
44
Improving feature tracking: the “smalls” tracker (1)
smalls is a new feature tracker for shape-from-motion and similar applications
eliminates the heuristics normally used with Lucas-Kanade
enforces the rigid scene constraint
45
Improving feature tracking: the “smalls” tracker (2)
Leonard Smalls; tracker, manhunter
46
Improving feature tracking: the “smalls” tracker (3)
epipolar geometry
1-D matchingalong epipolar lines
geometric mistracking detection
feature death and birthoutput
to 6 DOFfeatur
esestimatio
n
47
Improving feature tracking: the “smalls” tracker (3)
epipolar geometry
1-D matchingalong epipolar lines
geometric mistracking detection
feature death and birthoutput
to 6 DOF
SIFT
features
estimation
features
48
Improving feature tracking: the “smalls” tracker (4)
SIFT keypoints (Lowe, IJCV 2004):
image interest points
can be extracted despite of large changes in viewpoint
to subpixel accuracy
A keypoint’s feature vectors in two images usually match
49
Improving feature tracking: the “smalls” tracker (5)
Epipolar geometry between adjacent images is determined using…
SIFT extraction and matching
two-frame bundle adjustment
RANSAC
epipolar geometrySIFTfeatur
es
50
Improving feature tracking: the “smalls” tracker (6)
Search for new feature locations constrained to epipolar lines:
1. initial position from nearby SIFT matches
2. discrete SSD search (e.g., 60 pixels)
3. 1-D Lucas-Kanade refines the match
1-D matchingalong epipolar lines
51
Improving feature tracking: the “smalls” tracker (7)
Mistracked or occluded features are detected using geometric consistency between triples of images
geometric mistracking detection
• three-frame bundle adjustment
• RANSAC
52
Improving feature tracking: the “smalls” tracker (8)
After tracking in each image:
features are pruned to maintain a minimum separation
new features are selected in those parts of the image not already covered
feature death and birthoutput
to 6 DOFfeatur
esestimatio
n
53
Improving feature tracking: the “smalls” tracker (9)
54
Improving feature tracking: the “smalls” tracker (10)
55
Improving feature tracking: the “smalls” tracker (11)
56
Improving feature tracking: the “smalls” tracker (12)
57
Outline
Image-based motion estimation
Improving image-based motion estimation
Improving feature tracking
Reacquisition
58
Reacquisition (1)
Image-based motion estimates from any system will drift:
if the features we see are always changing
given sufficient time
if we don’t recognize when we’ve revisited a location
59
Reacquisition (2)
60
Reacquisition (3)
61
Thanks!
Related materials:
these slides
related papers
movies
VRML models
at:
http://www.cs.cmu.edu/~dstrelow/northrop