View
216
Download
0
Category
Tags:
Preview:
Citation preview
Discontinuity Preserving Stereo with Small Baseline
Multi-Flash Illumination
Rogerio Feris1, Ramesh Raskar2, Longbin Chen1, Karhan Tan3 and Matthew Turk1
1University of California, Santa Barbara2Mitsubishi Electric Research Labs
3Epson Palo Alto Lab
Introduction
Correspondence Problem
Stereo Near Depth Discontinuities:
- Occlusion Problem
- Perspective Distortions
- Violation of Smoothness Constraints
Passive Versus Active Methods
Introduction
Our Approach:
Small Baseline Multi-Flash Illumination
- Simple, Inexpensive
- Compact, Self-Contained
- Discontinuity Preserving
Depth Edges with Multi-Flash
Raskar, Tan, Feris, Yu, Turk – ACM SIGGRAPH 2004
Bottom Flash Top Flash Left Flash Right Flash
Ratio images and directions of epipolar traversal
Shadow-Free
Depth Edges
Shadow-Free Depth Edges
Qualitative Depth Map
Qualitative Depth
Sign of Depth Edge
- Indicates which side is the foreground and which side is the background
Shadow Width
- Encodes object relative distances
Sign of Depth Edge
+ -+-
(+) Foreground (-) Background
Original Ratio Left Ratio Right Signed Edges
Shadow Width Bottom Flash Image Ratio Image
Plot Along Scanline
Shadow Width Bottom Flash Image Ratio Image
Shadow Width Estimation:
Meanshift Segmentation algorithm applied on the ratio image
Imaging Geometry
Object
Flash
Shadow
CameraB
z1
z2
f
21
12 )(
zz
zzfBd
Shadow Width
d
Qualitative Depth
Working on this Equation …
)log()log()1log(
)11log()1log(
1
12
1
22
1
22
zzd
z
z
fB
dz
z
z
fB
dz
Log Depth DifferenceShadow Width
Gradient-Domain
Problem!
Qualitative Depth
1) Compute Sharp Depth Gradient G = (Gh,Gv)
otherwise ),()1log(
edgedepth anot is y)(x, if 0),(
yxsdyxG
hhh
Log Depth Difference Sign of depth edge
2) Compute Q’ by integrating G (Poisson Equation)3) Qualitative depth map Q = exp(Q’)
Qualitative Depth
Useful Prior Information for Stereo !
Occlusion Map
Partial Occlusion Problem
Object
Camera
Occlusion
A B
(Seen by A but not by B)
Occlusion Bounded by Shadows
Object
CameraA B
Flash
Occlusion (Seen by A but not by B)
Occlusion Bounded by Shadows
Object
CameraA B
Flash
Lower Bound Shadow
Occlusion Bounded by Shadows
Object
CameraA B
Flash
Upper Bound Shadow
Occlusion Bounded by Shadows
Object
Camera
Occlusion
A B
Average of Upper/Lower Shadow widths
Flash
Occlusion Bounded by Shadows
Occlusion Map
Left View Right View
Discontinuity Preserving Stereo
Matching
Local Stereo
Problem: Shape and size of correlation window
- Small Window Ambiguities / Noise
- Large Window Problems at Depth Discontinuities
Depth Edge Preserving Local Stereo
Object Boundary (Depth Edge)
Correlation Window
Local Stereo
Smooth Disparity
Delimited by depth edges + Occlusions
Correlation Window
Problem: Shape and size of correlation window
- Small Window Ambiguities / Noise
- Large Window Problems at Depth Discontinuities
Depth Edge Preserving Local Stereo
Local Stereo
Left View Depth Edges + Occlusion Ground Truth
Challenging Scene:
- Ambiguous patterns, textureless regions, geometrically complex object, thin structures
Local StereoConventional 9x9
Conventional 31x31
Our Approach 31x31--- Conventional
Stereo Our Approach
Global Stereo
Global Optimization – Markov Random Field (MAP-MRF)
X = {xs} Disparity of each pixel (Hidden)
Y = {ys} Matching cost at each disparity (Observed)
X3 X1 X2 X7
X4 X6
X5 X8
y1 y2
Global Stereo
Global Optimization – Markov Random Field (MAP-MRF)
s s sNt
tsstsss xxyxYXP)(
),(),()|(
X = {xs} Disparity of each pixel (Hidden)
Y = {ys} Matching cost at each disparity (Observed)
Data Term
Smoothness Term
Inference by Belief Propagation [Jian Sun et al, 2003]
Global Stereo
Qualitative Depth Map as Evidence
- Used to set the smoothness term
- Information propagation is stopped at depth edges
- Encourage disparities for neighboring pixels according to depth difference in qualitative map
Occlusion Penalty
Global Stereo
Conventional Belief Propagation Our Approach
RMS: 0.9589 RMS: 0.4590
Conclusions
Contributions
- Stereo with small baseline illumination
- Useful Feature Maps (Qualitative Depth + Occlusion Map)
- Enhanced Local and Global Stereo Algorithms
Pros / Cons
- Robust, Simple, Inexpensive and Compact
- Limited to handle outdoor scenes and motion
Website (datasets, source code)
- http://www.cs.ucsb.edu/~rferis/multi-flash-stereo
Thank you !
Multi-Flash Stereo Webpage
http://www.cs.ucsb.edu/~rferis/multi-flash-stereo
Four Eyes Lab, UCSB
http://ilab.cs.ucsb.edu
Occlusion Bounded by Shadows
Occlusion Detection by averaging length of shadows
Images taken with light sources surrounding the other camera
Recommended