1
Basic Single-View Geometry
2
Finite cameras
Man Drawing a Lute (The Draughtsman of the Lute), woodcut 1525, Albrecht Dürer
3
The basic pinhole model Central projection of pts in space
onto a plane Center of projection in the origin Plane of projection
(principal/image plane) in z=f Principal Point in (0,0, f)
=
10100000000
1ZYX
ff
ZfYfX
ZYX
( )[ ]0IPPXx
| 1,,diag ff==
∝
=
ZfYfX
ZYfZXf
vu
11
Camera Center, Optical Center
Principal Plane
Principal Point Principal Ray,
Optical axis
4
Principal Point offset
K: camera calibration matrix Xcam: camera located at the origin of a coordinate
system with principal axis aligned with the z axis
=
++
∝
+
+
=
101000000
111
ZYX
pfpf
ZZpfYZpfX
pZYf
pZXf
vu
ZYX
y
x
y
x
y
x
[ ] [ ]0I0IKP
PXx
|100
00
|
==
=
y
x
pfpf
cam
Position of the Principal Point ( px,py,f )
5
Camera Rotation and Translation (1)
xcam
ycam
zcam
C
O
Z
X Y
R,t
( )CRt
tXRCXRX~
~~~~
−=
+=−=cam
6
Camera Rotation and Translation (2)
In a world coordinate frame:
General mapping given by a pinhole camera 9 dof (3 for K, 3 for R, and 3 for C)
( ) [ ] [ ] XCRRCRRXCXRX ~
1
~~~~camcam −=
−=⇒−=ZYX
[ ] [ ] | ~ | tRKCIKRP
PXx
=−=
=
7
Digital cameras
Intrinsic camera parameters αx=fmx, αy =fmy mx, my : no. of pixels per unit distance s : skew (usually zero) x0 =mxpx, y0 =mypy coords of the principal pt in
terms of pixel dimensions A total of 10 d.o.f.
4 intrinsic 6 extrinsic (camera position and orientation)
[ ]
==
=
1000 , ~ | 0
0
yxs
- y
x
αα
KCIKRP
PXx
s≠0
8
Decomposition of the camera matrix Problem: given the projection matrix P, find the (intrinsic
and extrinsic) camera parameters Camera center
Point C for which PC=0 It can be shown that
Position, orientation and intrinsic parameters
Decomposition of M=KR done with the “RQ-decomposition” method (orthogonal matrix times upper triangular matrix)
• No ambiguities if K is set to have positive diagonal entries
[ ]( )[ ]( )[ ]( )[ ]( )
=
=
321
421
431
432
,,det,,det,,det,,det
pppppppppppp
C
tzyx
[ ] [ ] KRMCIMCMMP === , ~ | ~ | --
9
Euclidean vs projective spaces The 3D world is Euclidean and so is the image coordinate
system Projective geometry is used for turning central projection
into a linear operator Directions are pts on π∞ Homogeneous coordinates
Why not use a projective coordinate frame at all times? Tf from the projective world coordinate frame to the camera
coordinate frame is a 4x4 homogeneous matrix Map from P3 to P2 is a 3x4 matrix of rank 3
HMP
=
010000100001
4x4 homography 3x3 homography
10
Vanishing points
11
Vanishing line
12
Vanishing lines and points
13
Computation of the projection matrix Given a number of correspondences between world pts Xi ∈P3 and image pts xi
∈P2, compute the 3x4 matrix P such that xi=PXi
DLT (Direct Linear Transformation) algorithm Since the equation xi=PXi is homogeneous, xi and PXi are not equal. They
have the same direction but they may differ in magnitude, therefore we can write
We will use this expression to produce a linear estimation algorithm for the projection matrix
0=× ii PXx
14
DLT method for the estimation of P The j-th row of P is denoted by pj T (4-vector)
since pj Txi= xiT pj, we can write
0pA
0ppp
XXXX
XX
=
=
−−
−
i
Tii
Tii
Tii
Tii
Tii
Tii
xyxz
yz
3
2
1
00
0
( )
−−
−==×⇒
= ×
iT
iT
iT
ii
ii
ii
iiii
iT
iT
iT
i
xyxz
yz
XpXpXp
PXxPXxXpXpXp
PX3
2
1
3
2
1
00
0
15
DLT method for the estimation of P Only two of the three eqs are linearly independent,
therefore it can be rewritten as
Pay attention to which row you drop! E.g. If one of the pts is at infinity (zi=0), then we should not drop the third row, otherwise we would end up with just one equation
Keeping all three rows could also lead to a better conditioned set of eqs.
( )( ) ( )12112122
00
3
2
1
×=××=
=
−−
0pA
0ppp
XXXX
i
Tii
Tii
Tii
Tii
xzyz
16
DLT method for the estimation of P
The equation is linear in the unknown p The matrix elements of Ai are quadratic in the known
coordinates of the pts Each point correspondence gives rise to two independent
eqs in the entries of P, which has 11 d.o.f. Since we need 11 equations, we will use “5 ½
correspondences” (we will only use one of the eqs. for the last correspondence)
Overdetermined solutions are often advisable (noisy data)
Degenerate configurations of 3D points could prevent a complete calibration of the system (for example when the 3D points lie on a single plane)*
( )( ) ( )12112122 ×=××= 0pAi
* For more details see the book of Hartley and Zisserman (Ch. 21) and “A Flexible New Technique for Camera Calibration”. Z. Zhang. Technical Report MSRTR-98-71, Microsoft Research, December 1998. Available together with the software at http://research.microsoft.com/˜zhang/Calib/
17
Calibration through geometric error minimization
It is also possible to estimate the P elements by minimizing the geometric error between real image (xi) points and those computed by applying the projection matrix (P) to the 3D known points (Xi).
An iterative technique (i.e. Levenberg-Marquardt) is required for solving the problem. The DLT solution can be used as a starting point for the minimization.
( )
( )( ) ( ) ( )222
2
ˆ/ˆ/ˆ/ˆ/ˆ,
,,
,minˆ
iiiiiiiiii
iiii
iii
zyzyzxzxd
zyx
d
−+−=
=
= ∑
xx
x
PXxpP
18
Camera calibration Calibration:
An image pt x back-projects to a ray defined by x and the camera center
Calibration relates x to the direction of the ray In the Euclidean coords of the camera:
• Points on the ray
• Projection of points on the ray
( ) )~(~~1~ CdCCdX −+=−+= λλλ
[ ] Kdd
0IKx =
=
1|
λ
The camera calibration matrix K is the (affine) tf btw x and the ray’s direction d=K-1x measured in camera’s Euclidean coordinates
Note: generally d=K-1x is not a unit vector!
19
Instrinsic Camera calibration Angle between two rays with directions d1 and d2,
corresponding to the image pts x1 and x2, respectively, is given by
( ) ( )( ) ( ) ( ) ( )
( )( ) ( ) 2
112
11
21
1
21
21
11
11
21
11
2211
21cos
xKKxxKKxxKKx
xKxKxKxK
xKxKdddd
dd
−−−−
−−
−−−−
−−
=
==
TTTT
TT
TT
T
TT
T
θ
If the camera is calibrated (K known), then the angle btw rays can be measured from their corresponding image pts
20
The image of the absolute conic The calibration matrix K is related to the image of the
absolute conic ω The image of pts on π∞ can be easily derived
[ ][ ] KRd
dCIKRPXx
CIKRP
πd
X=
−==⇒
−=
∈
=
∞
∞∞
0~|
~|
0
The mapping btw π∞ and an image is given by the planar homography x=Hd, with H=KR
Note: this map does not depend on the position of the camera (C) but only on orientation and intrinsics!
21
The image of the absolute conic Since the absolute conic Ω∞ is on π∞, we can compute its
image through the homography H=KR ( ) ( ) ( ) ( )
( ) ( ) ( )( )( ) 1111
1111
11
1
−−−−−−
−−−−−−
−−−−∞
−−
===
===
====Ω=⇒
⇓
TTT
TTTT
TT
T
KKKKKRRK
KRRKKRKR
KRKRKRIKRωICCHHC
Hxx
The Image of the Absolute Conic (IAC) is the conic ω=(KKT)-1
22
Properties of the IAC The IAC depends only on the intrinsic camera
parameters The angle between two rays is
The Dual of the IAC (DIAC) is a line conic with matrix ( ) ( )( ) TT KKKKωωωω ==∝=
−−−−1111* det
23
Properties of the IAC
Once ω is identified in an image, then K is also identified A symmetric matrix ω*=KKT can be uniquely decomposed
into a product of an upper triangular matrix and its transposed (Cholesky factorization)
A plane π intersects π∞ in a line, which in turn intersects Ω∞ in two pts which are the circular pts of π
The imaged circular pts lie on ω at the pts of intersection btw the vanishing line of π and ω
These two properties can be used for calibration purposes
24
A simple calibration method The image of three squares (on non-
parallel planes) provides enough constraints to compute K The correspondence between the
4 corners of each square and their image allows us to compute the homography H between the plane of the square and the image plane
The same homography can be used to determine the image of the circular pts on π as H(1, ±i,0)T
Repeating that for each one of the three squares we obtain three distinct pairs of images of circular pts, which are enough to characterize the IAC ω (5 pts are enough to determine a conic)
K can be computed from ω∗=KKT through Cholesky factorization
Object coordinates on each object plane
0,0
0,A
A,0
A,A
25
A calibration method Compute the homography H between the plane of the square and the
image plane
Detemine the image of the circular pts as H(1, ±i,0)T Z
X
Y Z=0
H
21
0
1~ hhHm ii ±=
±=∞
26
A calibration method The image of circular points lies on the image of the
absolute conic:
Requiring that both real and imaginary parts be zero yields:
Two constraints for every image of the model plane
three images are required
( ) ( ) ( ) ( ) 0211
212121 =±±=±± −− hhKKhhhhωhh iiii TTT
21
211
121
211
1
11
221
1
0
0
hKKhhKKhhKKhhKKh
hKKhhKKh−−−−−−−−
−−−−
=⇒=−
=TTTTTTTT
TTTT
27
Lens Distortion (1)
Ideal projection
Observed position
dr dt
Xr
Yr
dr: radial distortion dt: tangential distortion
Pu Pd
28
Lens Distortion (2)
a
b
Xr
Axis of a maximum tangential distortion
Axis of a minimum tangential distortion
Radial Distortion Tangential Distortion
a) Barrel Distortion (typical in wide wide-angle photos) b) Pincushion distortion (typical in tele-photos)
Tangential distortions are typical due to misalignments among optical elements of the lens. Normally are negligeble.
29
Lens Distortion (3)
tele wideangle
30
Lens Distortion (4)
ZX
fxu =
ZY
fyu =
xdu dxx += ydu dyy +=
( )( )
++=
++=4
22
1
42
21
rkrkydrkrkxd
dy
dx 22dd yxr +=
Radial Component Description Camera Coordinate System
0
0
yyyxxx
u
u
−=−=
31
Lens Distortion (5) An image before and after radial distortion compensation
32
Calibration with planar target Using only a set of coplanar 3D reference points it is possible to calibrate a
camera system when: the position of the principal point (x0,y0) is known; the skew factor (s) is 0.
The considered target plane must be non parallel to the camera image plane.
Camera
Y
Z
Zw
c
Yw
c
c
wX
X
Zw
Yw
wX
Z c
Yc
cX
=R +T
+
=
z
y
x
w
w
w
c
c
c
TTT
ZYX
rrrrrrrrr
ZYX
987
654
321
33
The TSAI’s algorithm (1) Algorithm
Extrinsic Parameters estimated exploiting the Radial Alignment Constraint (R.A.C.)
Intrinsic Parameters estimated through nonlinear minimization
Key features + Efficient estimation + Fast and robust calculations - Based on a simplified camera model
(the P.P. is assumed as known, only the first term of the radial distortion is considered)
[ ] [ ][ ]
( ) ( )( ) ( )
( ) ( )( ) ( )xWWWdyWWWd
xWWWuyWWWu
dcdcccdd
ucucccuu
Tccc
Tddd
Tuuu
TZrYrXryTZrYrXrxTZrYrXryTZrYrXrx
yXxYYXyxyXxYYXyx
RAC
ZYXP
fyxpfyxp
+++=+++
+++=+++=−=×=−=×
⇒
=
==
321654
321654
0,,0,,
,,
,, ;,,
O
x
y
P
Q
p
q
R.A.C. : PQ // pq Image coord. system with origin in the P.P.
34
The TSAI’s algorithm (2)
( ) ( )
dyWdyWdyxdyWdyWd
ydWdWdxdWdWd
xWWdyWWd
xTrYxTrXxTTyTrYyTrXy
TxYrxXrxTyYryXry
TYrXryTYrXrx
=−−++
=−−−++⇓
++=++
−−−−− 15
14
112
11
5421
2154
0
If we assume Zw=0 for all the targets points and we consider distorted image coordinates, we have:
The projection equation assumes the form:
(1)
( ) ( )( ) ( ) uWWuzyyWW
uWWuzxxWW
zWW
yWW
y
u
zWW
xWW
x
u
yYrXryTTYrXrxYrXrxTTYrXr
TYrXrTYrXry
TYrXrTYrXrx
8754
8721
87
54
87
21 ;
+=−+++=−++
⇓
++
++=
++++
=
αα
αα
(2)
35
Tsai`s Two-Stage Camera Calibration Stage 1: Compute 3D orientation and x and y positions:
Compute (Xd, Yd) from calibration points in frame memory. Compute r1/Ty, r2/Ty, Tx/Ty, r4/Ty, and r5/Ty from the eq. (1) of the
previous slide (one eq. for each reference point). Compute r1 to r9, Tx, and Ty using orthogonality constraints on the R
matrix.
Stage 2: Compute αx, αy, k1, and Tz: Compute approximate f and Tz by ignoring lens distortion
(k1=0→xu=xd; yu=yd) by using eq. (2) of the previous slide (two eq.s for each reference point)..
Compute exact f, Tz, k1 by nonlinear optimization using the above initial values.
For details see: "A versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses", Roger Y. Tsai, IEEE Journal of Robotics and Automation, Vol. RA-3, No. 4, August 1987, pages 323-344.
36
Tsai’s Calibration improvement The quality of the calibration can be improved though the use of a full non-linear optimization based on the minimization of the global error between the known 2D positions of the target points and those estimated through the camera model
Different views of a 2D calibration pattern can be considered (in this case also the relative positions-orientations of the calibration plate must be estimated)
This way all the camera parameters (including the position of the principal point) may be considered.
For details see: "Flexible Camera Calibration By Viewing a Plane From Unknown orientations”. Z. Zhang. International Conference on Computer Vision (ICCV'99), Corfu, Greece, pages 666-673, September 1999
37
Two-view geometry
38
Point correspondence
39
Epipolar geometry The epipolar geometry btw two views is the geometry of
the intersection of the image planes with the pencil of planes having the baseline as axis
Any plane containing the baseline is an epipolar plane Epipolar lines: intersection of the epipolar plane with the image planes Epipoles: intersection of the baseline with the image planes
40
Epipolar geometry Converging cameras
Motion parallel to the image plane
41
Examples
42
Examples
43
The fundamental matrix Geometric derivation
A pt x in one image is transfered through the plane p to a pt x’ in the second image
π
x
e
x'
e' c c '
X
Hπ
xHx π='
44
The fundamental matrix Geometric derivation
The epipolar line through x’ is obtained joining x’ with the epipole e’.
π
x
e
x'
e' c
X
l'
c’
45
The fundamental matrix The fundamental matrix F may be written as
where Hπ is the transfer mapping from one image to the other through any plane π
Since has rank 2 and Hπ has rank 3, F is a matrix of rank 2 It is a mapping from the 2-D projective plane of the first
image to the pencil of epipolar lines through the epipole e’, therefore it must be rank-2
In order to compute F we need at least 7 correspondences between pts in the two images F has 7 d.o.f.:
• A 3x3 homogeneous matrix has 8 d.o.f • The constraint det F=0 reduces it to 7
46
Correspondence condition For any pair of corresponding pts in the two images we
have
In fact, if the two pts correspond, then x’ lies on the
epipolar line l’=Fx corresponding to the pt x If image pts satisfy the correspondence condition, then the
rays defined by these pts are coplanar
47
Properties of F Epipolar lines:
Epipole:
F has 7 dof: 3x3 homogeneous matrix:
8 dof det F=0: -1 dof
''' xFlxFxlx T=⇒=
( ) ( ) 0 , ' contains ' , ==≠∀⇒=≠∀ xFeFxeexeFxlex TT ''
0' , 0 == FeFe T
( ) ( ) ( ) 0' , '' contains ' , '' ===≠∀⇒=≠∀ xFexFexFeexexFlex TTTTTT
48
Computation of the fundamental matrix Basic equations Need at least 7 pts to determine F from this equation
Given a pair of pts (x,y,1)T and (x’,y’,1)T the eq. Becomes
From a set of n pt matches we have a set of linear eqs.
' , 0' xxFxx ↔∀=T
0Af =
=
33
11
''''''
11'11
'11
'1
'11
'11
'1
1
1
f
f
yxyyyxyxxyxy
yxyyyxyxyxxx
nnnnnnnnnnnn
( ) 01,,,',',',',','
0'''''' 333231232221131211
=⇓
=++++++++
fyxyyyxyxyxxx
fyfxffyyfyxfyfxyfxxfx
49
From the camera projection matrices to F (1)
( ) ( )( ) IPPPPPPPPPPP
xXPxPX
==⇒==
==
−+−+ 11 of inverse pseudo
''TTTT
( ) ( ) ( ) [ ] ( )[ ] ( )+
+++
=
==∧=∧=
PPeFFxxPPexPPexPPCPl
'''''''''
x
x
P+x lies on r, in fact it is projected by the first camera onto x P P+x=x
c
π
x e
x' e'
X l'
c’
r r’
c
50
From the camera projection matrices to F (2)
[ ] [ ]
[ ] [ ] 1
1
''''
;1
;
;|'' ;|
−+
−+
==
=
=
==
RKKtKPPCPF
0C
0K
P
tRKP0IKP
xx
T
For example we have:
π
x e
x' e'
X l'
c’
r r’
c
51
More on camera matrices and F If H is a 4x4 matrix representing a projective
transformation of P3, then the fundamental matrices corresponding to the pairs of camera matrices (P;P’) and (PH;P’H) are the same
A pair of projection matrices uniquely determine the fundamental matrix
[ ] [ ]
[ ] [ ] [ ]
1|'''
|' , |
MmFMxmMx0
mM0x
PCPl
mMP0IP
×× =⇒=∧
=
∧=
⇓==
Converse not true!
52
Canonical cameras given F (1) The fundamental matrix determines a pair of camera
matrices at best up to a projective tf Derivation of a specific formula for a pair of cameras
with canonical form given F based on two results:
0' have we, and setting 0' symmetric-skew '===
∀=⇔
FxxXP'x'PXxXFPXPXFPP
T
TTT
Result 1: A non-zero matrix F is the fundamental matrix corresponding to a pair of camera matrices P and P' iff P'TFP is skew-symmetric
53
Canonical cameras given F (2) Result 2: Let F be a fundamental matrix and S any
skew-symmetric matrix. Define
then F is the fundamental matrix of the pair (P,P’)
Proof is based on Result 1:
which is skew-symmetric, as
[ ] [ ] 3 rank with (epipole), , , '''|'| P0FeeSFP0IP TT ===
[ ] [ ]
=
==
00'|'|' T
TT
T
TTTT
00FSF
Fe0FSF
0IFeSFFPP
( ) ( ) 0FSSFSFFFSFFSFFSF =+=+=+ TTTTTTTTTT
54
Canonical cameras given F (3) Result 3: The camera matrices corresponding to a
fundamental matrix F can be chosen as This result follows from choosing and leads to the following
[ ]×= 'eS
Result 4: The most general formula for a pair of canonic camera matrices corresponding to a fundamental matrix F is
where v is any 3-vector and λ a non-zero scalar
55
F estimation with 7 correspondences We must solve the system Af=0 where, in this case A is
a 7×9 matrix It has generally rank 7, therefore the solution of the
system is a 2-dimensional space with form αF1+(1-α)F2. F1 and F2 are obtained as the matrices corresponding to the generators f1 and f2 of the right-null space of A (for example by using Z=null(A) in Matlab)
The condition det(F)=0 becomes det(αF1+(1-α)F2)=0. It is a cubic equation in α. One or three real solutions are possible. Therefore one or three different F matrices are the results of this estimation process
56
Computation of the fundamental matrix 8-pt method
The equation Af=0 is a homogeneous one, therefore f is determined only up to a scale factor
Enforcement of the singularity constraint F must be rank-2 and this is not guaranteed by the above procedure Replace F with the rank-2 matrix F’ that minimizes the Froboenius norm
||F’-F||frob
( )
( ) T
TT
srdiag
tsrtsrdiag
VUF
VUUDVF
0,,'
,,,
=
↓
≥≥==
[ ]AA
AAA
of conjugatecomplex theof transpose*
,
*2
,
T
ji
Tjifrob
tracea∑ ==
57
The normalized 8-point algorithm (1) 8-point algorithm performs badly in presence of noise Normalization of data
8-pt algorithm is sensitive to origin of coordinates and scale
Data must be translated and scaled to “canonical” coordinate frame
Normalizing transformation is applied to both images Translated so that centroid is in the origin Scaled so that RMS distance of points from origin is √2
58
The normalized 8-point algorithm (2) Normalization: Transform the image coordinates: Solution: Compute F from the matches
Singularity constraint: Find singular that lies the closest to
Denormalization:
ii
ii
xTxTxx
′=′=
'ˆˆ
ii xx ˆˆ ′↔
0ˆˆˆ =′iTi xFx
TFTFTxFTxxFx 'ˆ'0'ˆ''0ˆ'ˆ'ˆ Ti
TTii
Ti =⇒=⇒=
F′ˆ F
59
Automatic computation of F (1) 1. Interest pts: compute interest pts in each image 2. Putative correspondences: compute a set of pt matches based
on proximity and luminance profile similarity 3. RANSAC robust estimation: repeat for N samples
a. Select a random sample of 7 correspondences and compute F using the 7-pt method
b. Compute the distance d⊥ of each putative match (given x, the distance between x’ and the epipolar line l’=Fx).
c. Compute the No. of inliers consistent with F as the no. of correspondences for which d⊥<t pixels
d. If there are 3 real solutions for F, the No. of inliers is computed for each solution and the solution with most inliers retained
⇒ Choose F with the largest no. of inliers
4. Non-linear estimation (Levemberg-Marquardt) 5. Guided matching search for further interest pts using
F The last two steps are iterated until until the number of
correspondencs is stable
60
Automatic computation of F (2) Why use the 7-points algorithm instead of the 8-points algorithm? Given w the probability that the match between a couple of points is correct, the probability p that, in N extractions of groups of s couples of points, there is at least one set of couples all correctly matched is: p=1-(1-ws)N
For p=0.99, w=0.5, s=7 N>586 For p=0.99, w=0.5, s=8 N>1176 Therefore the use in the proposed procedure of the 7-point alg. for F estimation allows us to significantly reduce the number of trials
61
Automatic computation of F (3) What is d⊥? Given x and x’, d⊥ is the minimum Euclidian distance between x’ and the epipolar line calculated from the available F (l’=Fx).
x’ l’ d⊥
[ ][ ]
22
;,,'
;,,'
ba
cwyb
wxa
d
cba
wyxT
T
+
++=
=
=
⊥
l
x
62
Automatic computation of F (4) What is the step #4 of the proposed procedure? Given a first estimation of the fundamental matrix Chose camera matrices as From the correspondences xi↔xi’ and determine in order to minimize Iteratively minimize Over There are 12+3N variables.
F
[ ] [ ][ ]0'
;'|' ' ;|=
==
FeeFeP0IP
Tx
F iX
( ) ( )22 'ˆ,'ˆ, iiii dd xxxx +
( ) ( )∑ +i
iiii dd 22 'ˆ,'ˆ, xxxx
N1i ˆ,ˆ=iXF
12 parameters is an over-parametrization of F: . Also minimal parameterization is possible. For example: Singularity when epipole of 1° imm is. ∞
+++
=βαβαβα
eedcdcbaba
F1
63
Motion vectors of the corners
Inliers Outliers
Example
64
RANSAC results
65
The essential matrix Specialization of F when using normalized
coords:
Adopting normalized coords, the fundamental matrix becomes
Properties: 5 dof: 3 (R) + 3 (t) – 1 (scale) 2 extra constraints w.r.t. F
[ ] [ ]
====
→
== −−
tRPXPPXKxKx
tRKPPXx K
|ˆˆˆ
|
11
1-
[ ] [ ]×× == tRRRtE T
66
Computation of the essential matrix The essential matrix can be computed from point correspondences
(expressed in normalized coordinates):
Or directly from: 0ˆ'ˆ =xEx T
FKKE T'=
A 3x3 matrix is an essential matrix iff two of its singular values are equal and the third one is zero
Note: From E it is possible to retrieve and up to a scale factor and a four-fold ambiguity. From F, P and P’ are retrieved up to a projective transformation.
P 'P
67
Retrieving the camera matrices from E Assume that E=Udiag(1,1,0)VT (SVD)
A reconstructed point X will be in front of both cameras in only one of these four solutions
For a given essential matrix E=Udiag(1,1,0)VT, and first camera matrix P=[I|0], there are four possible choices for the second camera matrix P’:
where
[ ][ ][ ][ ]3
3
3
3
'
'
'
'
uVUWPuVUWPuUWVP
uUWVP
−=
=
−=
=
TT
TT
T
T
−=
100001010
W
68
Triangulation Triangulation:
Knowing x and x’ Knowing P and P ’
Compute X such that X
x
e
x'
e' c c '
;PXx = ;'' XPx =
69
Triangulation in presence of noise In the presence of noise, back-projected lines do not
meet
x
c c '
'x
X
x
'x
70
Linear triangulation methods We need the equations x=PX and x'=P'X to be satisfied as
best as possible For P is true that:
Pp
0XpXp
XpXpXpXp
0Xppp
PXxPXx
of rows theare where
1
31
31
32
3
2
1
iT
TT
TT
TT
T
T
T
xyxy
yx
=
−−−
⇓
=
∧
=∧⇒∝
Only two rows are linear independent
71
Linear triangulation methods Write such conditions in the form of linear equations in X
Solve for X (right null space of A) It is possible to generalize for a multiple-view case Notice that we are not minimizing any geometrically-
meaningful quantity – not optimal
72
Minimizing geometric error Find X that minimizes difference between projected and measured
points
xx e
x'
e' c c '
X
'xd d'
Minimization of the Cost Function: 22 )'ˆ,'(')ˆ,()( xxxxX ddC +=Can be done by an iterative minimization technique (for example Levemberg-Marquardt). A close form solution is also possible (see the Hartley-Zisserman book), it requires to find the real roots of a 6-degree polynomial.
73
Point at minimum distance from 2 lines
xx
c c'
'x
X
'x
d
d’
[ ] [ ]
−
−+
−
−
=
−
−+
−
−
=
−
−+
−
−=
=
−+
−=∧+∧=
'ˆ''ˆˆˆ
'ˆ'''ˆˆˆ
'ˆ''ˆ'ˆˆ
'ˆ'ˆˆ''ˆD2222
2
cXcXcXcX
cXcXcXcX
cXcXcXcX
cXacXaXcaXca
TT
TT
TT
TT
xx
WW
AAAA
AAAA
22
2 ˆ'''''ˆD
;ˆ''''' ;ˆ
XcxcxcXc
cxcx
XcxcxcdXc
cxcxd
∧+∧=
∧=∧=
( ) ( ) viewsmoreor three toextendedeasely becan n formulatio The
'''ˆ
0'ˆ'ˆ2
point 3D searched theis 0Dfor which X The
1
2
2
ccX
cXcXD
opt
WWWW
WW
++=
=
−+
−=∇
=∇
−