Lecture16

CSE486, Penn StateRobert Collins

Lecture 16:

Planar Homographies

CSE486, Penn StateRobert Collins Motivation:

Points on Planar Surface

x

y


Review : Forward Projection

UVW

XYZ

xy

uv

WorldCoords

CameraCoords

FilmCoords

PixelCoords

Mext Mproj Maff

MintUVW

XYZ

uv

MUVW

uv

34

24

14

33

23

13

31

22

12

31

21

11

m

m

m

m

m

m

m

m

m

m

m

m


World to Camera Transformation

XX

YY

ZZ

PC

UU

VV

WWPW

Translate by - C (align origins)

CRotate toalign axes

R

PC = R ( PW - C ) = R PW + T


Perspective Matrix Equation(Camera Coordinates)

PM p ⋅= int

10

0

0

1

0

0

0

0

0

0

'

'

'

=

Z

Y

X

f

f

z

y

x

C

Z

Yfy

Z

Xfx

=

=


Film to Pixel Coords

10

0

0

1

0

0

0

0

0

0

'

'

'

=

Z

Y

X

f

f

z

y

x

100

a13a12a11

a23a22a21

w’

v’

u’

2D affine transformation from filmcoords (x,y) to pixel coordinates (u,v):

u = Mint PC = Maff Mproj PC

MprojMaff


Projection of Points on Planar Surface

x

y

Point on plane

Rotation +Translation

Perspectiveprojection

Film coordinates


Projection of Planar Points


Projection of Planar Points (cont)

Homography H(planar projectivetransformation)


Projection of Planar Points (cont)

Homography H(planar projectivetransformation)

Punchline: For planar surfaces, 3D to 2D perspectiveprojection reduces to a 2D to 2D transformation.

Punchline2: This transformation is INVERTIBLE!


Special Case : Frontal Plane

What if the planar surface is perpendicular tothe optic axis (Z axis of camera coord system)?

Then world rotation matrix simplies:


Frontal Plane

So the homography for a frontal plane simplies:

Similarity Transformation!


Convert to Pixel Coords

u

v

100

a13a12a11a23a22a21

pixelsInternal cameraparameters


Planar Projection Diagram

Here’s wheretransformationgroups get useful!


General Planar Projection

H1

A1

H



H1

A1

H-1


Frontal Plane Projection

S1

A1

A


Frontal Plane Projection

S1

A1

A-1



H2H1

H


Summary: Planar Projection

x

y

Point on planeRotation + Translation

Perspectiveprojection

Pixel coordsu

v

Internalparams

Homography

CSE486, Penn StateRobert Collins Applying Homographies to Remove

Perspective Distortion

from Hartley & Zisserman

4 point correspondences suffice forthe planar building facade

CSE486, Penn StateRobert Collins Homographies for

Bird’s-eye Views



Homographies for Mosaicing



Two Practical Issues

How to estimate the homography given four or more point correspondences (will derive L.S. solution now)

How to (un)warp image pixel values to produce a new picture (last class)


Estimating a Homography

Matrix Form:

Equations:


Degrees of Freedom?

There are 9 numbers h11,…,h33 , so are there 9 DOF?

No. Note that we can multiply all hij by nonzero kwithout changing the equations:


Enforcing 8 DOFOne approach: Set h33 = 1.

Second approach: Impose unit vector constraint

Subject to theconstraint:


L.S. using Algebraic Distance

Setting h33 = 1

Multiplying through by denominator

Rearrange


Algebraic Distance, h33=1 (cont)

Point 1

additionalpoints

2N x 8 8 x 1 2N x 1

Point 4

Point 3

Point 2


Algebraic Distance, h33=1 (cont)

A h = b2Nx8 8x1 2Nx1Linear

equations

Solve: AT A h = AT b 2Nx8 8x1 2Nx18x2N 8x2N

(AT A) h = (AT b)8x18x8 8x1

h = (AT A) (AT b)-1

Matlab: h = A \ b


Caution: Numeric Conditioning

R.Hartley: “In Defense of the Eight Point Algorithm”

Observation: Linear estimation of projective transformation parameters from point correspondences often suffer from poor“conditioning” of the matrices involves. This means the solutionis sensitive to noise in the points (even if there are no outliers).

To get better answers, precondition the matrices by performinga normalization of each point set by:• translating center of mass to the origin • scaling so that average distance of points from origin is sqrt(2).• do this normalization to each point set independently


Hartley’s PreConditioning

H?PointSet1 PointSet2

Scale so averagepoint dist is sqrt(2)S1 S2

Translatecenter of massto origin

T1 T2

Estimate Hnorm

H = T2-1 S2

-1 Hnorm S1 T1


A More General Approach

What might be wrong with setting h33 = 1?

If h33 actually = 0, we can’t get the right answer.


Algebraic Distance, ||h||=1

||h|| = 1

Multiplying through by denominator

Rearrange

= 0

= 0


Algebraic Distance, ||h||=1 (cont)

additionalpoints

2N x 9 9 x 1 2N x 1

4 POINTS


Algebraic Distance, ||h||=1 (cont)

A h = 02Nx9 9x1 2Nx1Homogeneous

equations

Solve:

SVD of ATA = U D UT

Let h be the column of U (unit eigenvector) associated with the smallest eigenvalue in D. (if only 4 points, that eigenvalue will be 0)

AT A h = AT 02Nx9 9x1 2Nx19x2N 9x2N

(AT A) h = 09x19x9 9x1

Documents

Lecture16