View
222
Download
4
Tags:
Embed Size (px)
Citation preview
Stanford CS223B Computer Vision, Winter 2005
Lecture 5: Stereo I
Sebastian Thrun, Stanford
Rick Szeliski, Microsoft
Hendrik Dahlkamp and Dan Morris, Stanford
StereoStereo
Sebastian Thrun Stanford University CS223B Computer Vision
Stereo Vision: Illustration
http://www.well.com/user/jimg/stereo/stereo_list.html
Sebastian Thrun Stanford University CS223B Computer Vision
Stereo Vision: Outline
Basic Equations Epipolar Geometry Image Rectification Reconstruction Correspondence Dense and Layered Stereo (Active Range Imaging Techniques)
Sebastian Thrun Stanford University CS223B Computer Vision
Pinhole Camera Model
Imageplane Focal length f
Center ofprojection
Sebastian Thrun Stanford University CS223B Computer Vision
Pinhole Camera Model
Imageplane
),,( ZYXP
),,( ZYXP
f
Oy
x
z
Z
Z
Y
Y
X
X
ZZ
YY
XX
OPPO
Sebastian Thrun Stanford University CS223B Computer Vision
Pinhole Camera Model
Imageplane
),,( ZYXP
),,( ZYXP
f
Oy
x
z
)1,,()1,,(),,(Z
Yf
Z
XfyxZYX
YyXxZ
YfY
Z
XfXfZ
Sebastian Thrun Stanford University CS223B Computer Vision
Basic Stereo Derivations
),,(1 ZYXP 1Oy
x
z
f
2Oy
x
z
B
BfxxZ ,,, offunction a as for expression Derive 21
1p
2p
Sebastian Thrun Stanford University CS223B Computer Vision
Basic Stereo Derivations
),,(1 ZYXP 1Oy
x
z
f
2Oy
x
z
B
211
11
1
12
1
11 ,
xx
BfZ
Z
Bfx
Z
BXfx
Z
Xfx
Sebastian Thrun Stanford University CS223B Computer Vision
What If…?
),,(1 ZYXP 1Oy
x
z
f
2Oy
x
z
B
1p
2p
),,(1 ZYXP 1Oy
x
z
1p
f2O
y
x
z
2p
Sebastian Thrun Stanford University CS223B Computer Vision
Epipolar Geometry
pl pr
P
Ol Or
Xl
Xr
Pl Pr
fl fr
Zl
Yl
Zr
Yr
Rrotation Tontranslati
Sebastian Thrun Stanford University CS223B Computer Vision
Epipolar Geometry
plp
r
P
Ol Orel er
Pl Pr
Epipolar Plane
Epipolar Lines
Epipoles
Sebastian Thrun Stanford University CS223B Computer Vision
Epipolar Geometry
Epipolar plane: plane going through point P and the centers of projection (COPs) of the two cameras
Epipoles: The image in one camera of the COP of the other
Epipolar Constraint: Corresponding points must lie on epipolar lines
Sebastian Thrun Stanford University CS223B Computer Vision
Essential Matrix
pl pr
P
Ol Orel er
Pl Pr
Orthogonality T, Pl, PlT: 0)( lT
l PTTP
)( TPRP lr Coordinate Transformation:
0
0
0
xy
xz
yz
TT
TT
TT
S
ll SPPT
0)( lT
rT SPPR
0lT
r RSPP
0)( lT
rT PTPRResolves to
RSE Essential Matrix 0lT
r EPP
Sebastian Thrun Stanford University CS223B Computer Vision
Essential Matrix
pl pr
P
Ol Orel er
Pl Pr
0
0
0
xy
xz
yz
TT
TT
TT
SRSE Essential Matrix
0 lTr Epp0l
Tr EPP
Projective Line: lr Epu
Sebastian Thrun Stanford University CS223B Computer Vision
Fundamental Matrix
Same as Essential Matrix in Camera Pixel Coordinates
0lTr pFp
0lTr Epp
Pixel coordinates 1 lT
r EMMF
Intrinsic parameters
Sebastian Thrun Stanford University CS223B Computer Vision
Computing F: The Eight-Point Algorithm
Input: n point correspondences ( n >= 8)– Construct homogeneous system Ax= 0 from
• x = (f11,f12, ,f13, f21,f22,f23 f31,f32, f33) : entries in F• Each correspondence give one equation• A is a nx9 matrix
– Obtain estimate F^ by SVD of A:• x (up to a scale) is column of V corresponding to the least
singular value– Enforce singularity constraint: since Rank (F) = 2
• Compute SVD of F:• Set the smallest singular value to 0: D -> D’• Correct estimate of F :
Output: the estimate of the fundamental matrix F’ Similarly we can compute E given intrinsic
parameters
0lTr pFp
TUDVA
TUDVF ˆ
TVUDF' '
Sebastian Thrun Stanford University CS223B Computer Vision
Recitification
Idea: Align Epipolar Lines with Scan Lines.
Question: What type transformation?
Sebastian Thrun Stanford University CS223B Computer Vision
Locating the Epipoles
pl pr
P
Ol Orel er
Pl Pr
Input: Fundamental Matrix F– Find the SVD of F– The epipole el is the column of V corresponding to the
null singular value (as shown above)– The epipole er is the column of U corresponding to the
null singular value (similar treatment as for el) Output: Epipole el and er
TUDVF
el lies on all the epipolar lines of the left image
0lTr pFp
0lTr eFp
0leF
Sebastian Thrun Stanford University CS223B Computer Vision
Stereo Rectification (see Trucco)
Stereo System with Parallel Optical AxesEpipoles are at infinity
Horizontal epipolar lines
pl
pr
P
Ol Or
Xl
Xr
Pl Pr
Zl
Yl
Zr
Yr
T
Sebastian Thrun Stanford University CS223B Computer Vision
pl
pr
P
Ol Or
Pl Pr
Reconstruction (3-D): Idealized
Sebastian Thrun Stanford University CS223B Computer Vision
pl
pr
P
Ol Or
Pl Pr
Reconstruction (3-D): Real
See Trucco/Verri, pages 161-171
Sebastian Thrun Stanford University CS223B Computer Vision
Summary Stereo Vision (Class 1)
Epipolar Geometry: Corresponding points lie on epipolar
line
Essential/Fundamental matrix: Defines this line
Eight-Point Algorithm: Recovers Fundamental matrix
Rectification: Epipolar lines parallel to scanlines
Reconstruction: Minimize quadratic distance