December 11, 2002 MVA2002 1

Determining Shapes of Transparent Determining Shapes of Transparent Objects from Two Polarization ImagesObjects from Two Polarization Images

Daisuke Miyazaki

Masataka Kagesawa

Katsushi Ikeuchi

The University of Tokyo, Japan

December 11, 2002 MVA2002 2

Modeling transparent objectsModeling transparent objects

Polarization-based vision system

Unambiguous determination of surface normal using geometrical invariant

Transparentobject

VR

December 11, 2002 MVA2002 3

Related worksRelated works

Koshikawa 1979Koshikawa et al. 1987

Wolff 1990Wolff et al. 1991

Saito et al. 1999

Miyazaki et al. 2002

Our method

Rahmann et al. 2001

Need many light sourcesNot search corresponding points

Not solve ambiguity problem

Not needcamera calibration

Not needinfrared camera

Sphericaldiffuser

DOP

ThermalradiationBinocular

stereo

Optimizationmethod Searching

correspondingpoints

December 11, 2002 MVA2002 4

OutlineOutline

Rotatethe object

Targetobject

DOP(Degree Of

Polarization)images

Regionsegmentation

Searchcorresponding

points

3D model

December 11, 2002 MVA2002 5

PolarizationPolarization

PolarizerObjectAir

Incident light Reflected light

Surface normal

Transmitted light

Incidentangle

Reflectionangle

DOP(Degree Of Polarization): the ratio of how much the light polarized

DOP Origin

Unpolarized light 0 Sunlight / incandescent light

Perfectly polarized light 1 The light transmitted the polarizer

Partially polarized light 0~1 The light hit the object surface

Lightsource

Unpolarized light(DOP 0)

Pefectly polarized light(DOP 1)

Partially polarized light(DOP 0~1)

December 11, 2002 MVA2002 6

Object

ObservationObservation

Surface normal

Lightsource

Polarizer

Camera

Lightsource

Surfa

ce n

orm

al

P Q

P

Incident angle P

Reflection angle

Q Incident angleQ

Ref

lect

ion

angl

e

PPhase angle

Q Phase angle

December 11, 2002 MVA2002 7

Ambiguity of phase angle Ambiguity of phase angle

Determination of phase angle Propagate the determination from occluding boundary to the inner area (Assume C2 surface)

Inten

sity

255

0

IminP

3601P 2

P

Phase angleAzimuth angle

-ambiguity

December 11, 2002 MVA2002 8

Ambiguity of reflection angle Ambiguity of reflection angle

Reflection angleZenith angle

DO

P(D

eg

ree O

f Pola

rizatio

n)

1

P

0 1P 2

P

Brewsterangle

B 90

-ambiguity

Determination of reflection angle Explain in the following slides

December 11, 2002 MVA2002 9

Object rotationObject rotation

Rotate the object at a small angle Solve the ambiguity from two DOP images

taken from two directions

Rotate

Camera

Object

December 11, 2002 MVA2002 10

Region segmentationRegion segmentation

Measure DOP of the

object

DOP image

DOP 1: whiteDOP 0: black

Result ofregion segmentation

Divided into3 regions

Re

gio

n

segm

en

tation

Divide DOP image with curves of 1 DOP (Brewster angle)

December 11, 2002 MVA2002 11

Gauss’ mapGauss’ mapN

B-E region B-B region B-N region

N NNN

or

EF

B

B: Brewster curve N: North pole E: Equator F: Folding curve

B B B

E E E

F

December 11, 2002 MVA2002 12

B-E region & B-N regionB-E region & B-N region

B-E region (B<<90o)Definition: A region enclosed by occlu

ding boundariesDetermine the occluding boundary fro

m background subtraction

B-N region (0o<<B)Definition: A region where a point of 0o

is included is 0o or 90o when DOP is 0Assume there is no self-occlusion, so

is 0o when DOP is 0

December 11, 2002 MVA2002 13

B-B regionB-B region

B-B region (0o<<B or B<<90o)Definition: A region which is not the previous twoApply the following disambiguation method to this

region

December 11, 2002 MVA2002 14

Folding curveFolding curve

A curve (on G) that is a part of the boundary of the region (on G) and is not a Brewster curve (on G) is called a folding curve (on G)

Folding curve

Brewster curve

Equator

North pole

Gaussian sphere

G=Gaussian sphere

December 11, 2002 MVA2002 15

Parabolic curveParabolic curve

Theorem: Folding curve is parabolic curveParabolic curve = a curve where Gaussian

curvature is 0

Folding curve = geometrical invariantNorth pole

Folding curve

Equator

Gaussian sphereObject surface

Folding curve

December 11, 2002 MVA2002 16

Corresponding pointCorresponding point

Corresponding point folding curve great circle arg min DOP, s. t. surface normal // rotation plane

Correspondingpoint

Rotate the object

North side

Correspondingpoint

Rotate the object

South side

point

[= rotation direction]

December 11, 2002 MVA2002 17

Difference of DOPDifference of DOP

De

rivative

of

DO

PD

OP

+

–

1

0

B 90

0

sgn)(sgn)()(sgn

DO

P b

efo

re

rotation

DO

P a

fter

rotation

Ro

tation

ang

le

De

rivative

of

DO

P

Compare two DOPsat the pair of corresponding points

December 11, 2002 MVA2002 18

Acquisition systemAcquisition system

Object

Camera

Polarizer

Light

Light

Light

Optica

ldiffuse

r

December 11, 2002 MVA2002 19

PrecisionPrecision

Result of region segmentation

Estimated shape

Graph of DOP

Reflectionangle

DO

P

Error (Average absolute difference)

Error

DOP 0.17

Reflection angle 8.5

Height 2.6mm

Plastic transparent hemisphere [diameter 3cm]

December 11, 2002 MVA2002 20

Target objectTarget object

Photo [Acrylic bell-shaped object]

December 11, 2002 MVA2002 21

DOP imagesDOP images

DOP image when the object is not rotated

DOP image when the object is rotated at a small angle

Ro

tation d

irectio

n

We rotate the object about 8°

DOP0:whiteDOP1:black

December 11, 2002 MVA2002 22

Region segmentation resultRegion segmentation result

Result of region segmentation when the object is not rotated

Result of region segmentation when the object is rotated at a small angle

Ro

tation d

irectio

n

We rotate the object about 8°

December 11, 2002 MVA2002 23

Disambiguation of B-B regionDisambiguation of B-B region

De

rivative

of

DO

P

Positive

NegativeB 90

0

sgn)(sgn)()(sgn

Negative

Po

sitive

Su

rface n

orm

al w

as

Ro

tation d

irectio

n wa

s

0.0890.084

Negative

December 11, 2002 MVA2002 24

Rendered imageRendered image

Shading image

December 11, 2002 MVA2002 25

Rendered imageRendered image

Photo Raytracing image

December 11, 2002 MVA2002 26

ErrorError

Comparison of true value and estimated value

The diameter(width) of the object is 24mmError is 0.4mm (Average of the difference of the height)

True

Estimated

True value is made by hand

December 11, 2002 MVA2002 27

ConclusionsConclusions

A method to measure the surface shape of transparent object based on the analysis of polarization and geometrical characteristicsDetermined the surface normal with no ambiguityDetected a pair of corresponding points of

transparent surfaceDetermined the surface normal of the entire

surface at onceMeasured a transparent object which is not a

hemisphere

December 11, 2002 MVA2002 28

Future worksFuture works

Higher precision (dealing with interreflections) Estimation of refractive index More elegant method for determining phase a

ngle

December 11, 2002 MVA2002 29

(c) Daisuke Miyazaki 2002(c) Daisuke Miyazaki 2002All rights reserved.All rights reserved.

http://www.cvl.iis.u-tokyo.ac.jp/D. Miyazaki, M. Kagesawa, K. Ikeuchi, "Determining Shapes of Transparent Objects from Two Polarization Images," in Proceedings of IAPR Workshop on Machine Vision Applications, pp.26-31, Nara, Japan, 2002.12