Solving Visibility with Epipolar Geometry

Tien-Tsin Wong

Chinese University of Hong Kong

View Interpolation

Given the reference imagesSynthesize the (desired) image in between

Tien-Tsin Wong (Nov. 1999)

View Interpolation (2)

Two sub-problems: Where should pixels be moved to?

Visibility

• Depth• Correspondences or optical flows

Tien-Tsin Wong (Nov. 1999)

Visibility

Traditional solution: depth-buffering

Depths may not be available (especially in real image)

Can we solve the visibility without depth?

Epipolar geometry

Tien-Tsin Wong (Nov. 1999)

Epipolar Geometry

Configuration of two cameras

Referencecamera Desired

camera

Consider another pixel i2p2 will occlude p1 only when p1, p2 & e are

coplanar, colinear and p2 is in between

Referenceimage Desired

image

Tien-Tsin Wong (Nov. 1999)

To identify pixels potentially occlude each other, the epipolar plane is intersected with the reference image to give the epipolar line

Only pixels on the same epipolar line may occlude each other

Although we don’t know where are p1 & p2, their projection on the epipolar line determines the visibility (e.g. i1 will never occludes i2)

If we always draw i1 before i2, the correct visibility is ensured without knowing the depth information

Epipolar Geometry (2)

Epipolarplane

positiveepipole

Tien-Tsin Wong (Nov. 1999)

Pattern of Drawing Order

By intersecting the epipolar planes with the reference image, a pattern of drawing order can be obtained

Tien-Tsin Wong (Nov. 1999)

Pattern of Drawing Order

A diverging pattern is formed if the direction of the epipolar ray reverses.

Tien-Tsin Wong (Nov. 1999)

Pixel-based Drawing Order

In fact, there are only three kinds of patterns

Tien-Tsin Wong (Nov. 1999)

Results

To illustrate the ordering, the triangle should be drawn earlier is shaded in darker color.

Reference image Desired image

Foreground surface should be brighter

Tien-Tsin Wong (Nov. 1999)

The End

Tien-Tsin Wong (Nov. 1999)