EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix

EE 7700

Geometric Transformations

Bahadir K. Gunturk 2

Geometric Transformation

translation

scale

rotation & scale

2

1

43

21

b

b

y

x

aa

aa

y

x

Rigid flow

11001

'

'

243

121

y

x

baa

baa

y

x

cossin

sincos

s

s

0

0

Rotation matrix

Scale matrix


Affine Flow

2

1

43

21

b

b

y

x

aa

aa

yy

xx

v

u


Perspective Flow

1 2 3

7 8

'1

a x a y ax

a x a y

4 5 6

7 8

'1

a x a y ay

a x a y

'

'

u x x

v y y




EE 7730

RANSAC: RANdom SAmple Consensus


Outliers

Consider the least squares fit for optical flow:

1 2 1

3 4 2

( , )

( , )

u x y a x a y b

v x y a x a y b

1 1 1 1

1 1 1 2

1

3

4

2

1 0 0 0

0 0 0 1

1 0 0 0

0 0 0 1n n n

n nn

u x y a

v x y a

b

a

u ax y

bx yv

If some of the values are wrong, it will degrade the estimation.


Outliers

It is best not to include outliers in the estimation.

Outliers

Best fit is degraded due to the outliers

Line Fitting Problem: Given (x1,y1), …, (xN,yN); find the line y=ax+b

1 1 1

1N N

y xa

by x


Robust Estimation

Two step process: Classify data points as outliers or inliers Use inliers only to fit a model


RANSAC

Repeat for k times: Randomly choose n points (the smallest number of points

required) from the data. Estimate the parameters using these points. For each data point other than these n points:

Check if the data point is within a threshold, t, distance of current model; if it is, the data point is consistent with current model.

The total number of data points that are consistent is model’s support.

If the support is larger than a predetermined number, d, then there is a good fit. Re-estimate the parameters using these inliers.

Choose the best fit with the smallest fitting error.


RANSAC

Two samplesand their supports

for line-fitting


Example

Find the perspective parameters

from Hartley & Zisserman

1 2 3

7 8

'1

a x a y ax

a x a y

4 5 6

7 8

'1

a x a y ay

a x a y


Example

Apply corner detectors to both images



Example

Find the best match within a search window.



Example

Initial match results

188 matched features in left image pointing to locations of corresponding right image features



Example

Inliers and outliers after RANSAC

89 outliers 99 inliersfrom Hartley & Zisserman


Panoramic Image Reconstruction

Application: Mosaic construction

Find featuresMatch featuresFit parametric model

EE7730

Stereo Vision


Pinhole Camera


Review: Perspective Projection

' ' 'x y f

x y z


Stereoscene pointscene point

optical centeroptical center

image planeimage plane

p p’

p p’


Stereo Constraints

X1

Y1

Z1

O1

Image plane

Focal plane

M

p p’

Y2

X2

Z2O2

Epipolar Line

Epipole


From Geometry to Algebra

O O’

P

pp’

All vectors shown lie on the same plane.


From Geometry to Algebra

O O’

P

pp’

( , ,1)[ ( ')] 0 with

' ( ', ',1)

T

T

p u vp t Rp

p u v


Matrix form of cross product

20

0

0

y z z y z

z x x z z x

x y y z y x

a b a b a a

a b a b a b a a b a b

a b a b a a

( ) 0

( ) 0

a a b

b a b

a×b=|a||b|sin(η)u a=axi+ayj+azk

b=bxi+byj+bzk


The Essential Matrix

( , ,1)[ ( ')] 0 with

' ( ', ',1)

T

T

p u vp t Rp

p u v

' 0Tp Ep

' 0 with Tp Ep E t R Essential matrix


A Simple Stereo System

Zw=0

LEFT CAMERA

Left image:reference

Right image:target

RIGHT CAMERA

Elevation Zw

disparity

Depth Z

baseline


Parallel Cameras

ZT

fZxxTlr

OOll OOrr

PP

ppll pprr

TT

ZZ

xxll xxrr

ff

T is the stereo baseline

rlxx

TfZ

rlxxd Disparity:


Stereo View

Left View Right View

Disparity


Correlation Approach

For Each point (xl, yl) in the left image, define a window centered at the point

(xl, yl)LEFT IMAGE



… search its corresponding point within a search region in the right image

(xl, yl)RIGHT IMAGE



… the disparity (dx, dy) is the displacement when the correlation is maximum

(xl, yl)dx(xr, yr)RIGHT IMAGE


Stereo results

Ground truthScene

Data from University of Tsukuba

(Seitz)


Results with window correlation

Estimated depth of field Ground truth

(Seitz)


Results with better method

A state of the art methodBoykov et al., Fast Approximate Energy Minimization via Graph Cuts,

International Conference on Computer Vision, September 1999.

Ground truth

(Seitz)


Applications

courtesy of Sportvision

First-down line


ApplicationsVirtual advertising

courtesy of Princeton Video Image

Documents

EE 7700 Geometric Transformations. Bahadir K. Gunturk2 Geometric Transformation translation scale rotation & scale Rigid flow Rotation matrix Scale matrix