An extreme occurrence of the missing data

1

2

An extreme occurrence of the missing data

W I D E B A S E L I N E – no point in more than 2 images!

3

Difficult casesCoinciding camera centers

panorama zoom

Dominant planes

no problem

4

Uneven image capture

26 images325 image pairs

Some important, but very few matches

5

Uneven image capture

6

Our method can solve

all previous examples.

7

Algorithm

Technical contributionof this paper

• matches – uncalibrated EG [Matas et al. BMVC’02]

• focal length calibration

[Stewenius et al. CVPR’05], [Nister PAMI’04] [Chum]

• EG importance

• consistent rotations linearly

• bundle adjustment with constrained rotations

• consistent translations using SOCP [Kahl ICCV’05]

• dense stereo [Kostkova & Sara BMVC’03]

8

Calibrated RANSAC and planes

The six-point algorithm found only points on the wall. [Stewenius et al. CVPR’05]

Two-View Geometry Unaffected by a Dominant Plane. [Chum et al. CVPR'05]

use inliers as a pool for drawing samples in RANSAC on epipolar geometry

9

Full calibration

The “five-point algorithm” on all pairs. [Nister PAMI’04]

Partial calibration – unknown focal length

The “six-point algorithm” on all pairs. [Stewenius et al. CVPR’05]

mean focal length

10

Consistent rotations – previous work

[Uyttendaele et al., CG\&A '04] – dense video self-intersecting paths vanishing points

[Martinec, Pajdla CVPR'05] – gluing projective reconstructions

metric upgrade needed!

loosely coupled components – ambiguity!

11

Rotation registration into a reference framerotation matrices

rotations w.r.t. a reference framerelative rotation

consistent rotations

12

Consistent rotations – solution

fast: ~ 1 sec for 1000 image pairs

close to orthonormal

orthonormal

and solve

large & sparse matrix

rewrite as

eigenvalue problem global minimum

well conditioned

rotations – projection to orthonormal matrices

13

Refining rotations

• in each partial reconstruction:

14

Refining rotations

• in each partial reconstruction:

15

Refining rotations

• in each partial reconstruction:

replace rotations by the consistent ones,

16

Refining rotations

• in each partial reconstruction:

reprojection errors grow

bundle adjustment needed

change in relative rotation

replace rotations by the consistent ones,

17

Refining rotations

• in each partial reconstruction:

18

Refining rotations

• in each partial reconstruction:

• refine all reconstructions together,

each in independant coordinate frame,

but with corresponding rotations constrained to be same

re-estimate camera translations and points using [Kahl ICCV'05]

19

consistent rotations

same rotations,translations unknown

20

0.8 / 18 pxl

consistent rotations

low errors

stability

21

consistent rotations

0.8 / 18 pxl

0.20 / 1.6 pxl

refine

Refining rotations

22

Translations

consistent rotations

0.8 / 18 pxl

0.20 / 1.6 pxl

refine

0.24 / 1.3 pxl

consistent translations

[Kahl ICCV'05]

0.19 / 1.1 pxl

refine

23

Final reconstruction

24

Experiments

ICCV’05 Contest finals

mean / maximum error 3.01 / 4.87 meters

25

Experiments

ICCV’05 Contest finals

St. Martin rotunda – 104 images

26

support

Experiments

ICCV’05 Contest finals

Head2

St. Martin rotunda

correct surface

use triplets

importance

uneven image capture

few data

27

Summary

New algorithm for 3D reconstruction:

• EG importance

• consistent rotations linearly

• bundle adjustment with constrained rotations

Acknowledgements:• Ondrej Chum … code for EG unaffected by a dominant plane• Fred Schaffalitzki … code for the six-point algorithm (publicly available)• Lourakis et al. … base code for bundle adjustment (publicly available)• Jana Kostkova … routines for dense stereo• Richard Szeliski … the ICCV'05 Contest data (publicly available)

Difficult scenarios:

• coinciding camera centers

• only two-view matches

• uneven image capture, wide base-line

recent results on 260 viewspractical algorithm