24
Unit 1 Leisure Activities leisure: -- leisure activities -- leisure time use of free time

Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

Embed Size (px)

Citation preview

Page 1: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

Polarization-based Inverse Polarization-based Inverse Rendering from Single ViewRendering from Single View

Daisuke MiyazakiRobby T. Tan

Kenji HaraKatsushi Ikeuchi

Page 2: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

2

Modeling cultural assetsModeling cultural assets

Integrated framework for obtaining 3 types of information

Geometrical Photometrical Environmental

Page 3: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

3

Related workRelated work

Geometry Photometry Environment

Tominaga et.al. 2000

Zheng et.al. 1991

Nayar et.al. 1996

Sato et.al. 1999

Ramamoorthi et.al. 2001

Nishino et.al. 2001

Hara et.al. 2002

Proposed method

Page 4: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

4

OutlineOutline

1. Reflection componentsseparation

2. Shape from polarizationusing diffuse light

3. Light source estimationfrom intensity peak

4. Reflection parametersestimation by l.s.m. Minimize

Ks, σ

renderedimage

realimage

2

Page 5: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

1. Reflection components separation1. Reflection components separation

Page 6: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

6

Dichromatic reflection modelDichromatic reflection model

Incident lightSpecularly

reflected lightDiffusely

reflected light

Air

Object

Surfacenormal

Page 7: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

7

Reflection components separationReflection components separation

DiffuseInput Specular

[Tan2002]

•Shape•Illumination•Reflection parameters

Page 8: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

2. Shape from polarization2. Shape from polarization

Page 9: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

9

Related workRelated work

Object Reflection View

Koshikawa 1979 Opaque Specular 1

Wolff 1990 Opaque Diffuse 2

Rahmann et.al. 2001 Opaque Diffuse 2~5

Miyazaki et.al. 2002 Transparent Specular 2

Proposed method Opaque Diffuse 1

Page 10: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

10

PolarizationPolarization

Incident lightSpecularly

reflected lightDiffusely

reflected light

Air

Object

Page 11: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

11

Surface normalSurface normal

Object

Surface normal

Polarizer

Camera

Zenith angle

Azimuth angle

Page 12: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

12

Azimuth angleφ and intensity differenceAzimuth angleφ and intensity difference

Rotationangle

ofpolarizer

Inten

sity

255

0

Imax

3601 2

-ambiguity

Imin

Azimuth angle

Page 13: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

13

PropagationPropagation

[Ikeuchi&Horn1981]

Determination of azimuth angle Propagate φ from occluding boundary to inner part of object area (Assumption: smooth surface)

object

Cannot apply to “dimples”(=perfect concave)

Page 14: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

14

Zenith angleθ and DOPρZenith angleθ and DOPρ

0

1

90°

Zenith angle θ

DOP ρ

DegreeOfPolarization

ρ

θ

minmax

minmax

II

II

22222

22

sincos4sin122

sin1

nnnn

nn

Page 15: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

15

ModificationModification

0

0.5

90°

Zenithangle

θ

DOP ρ

DegreeOfPolarization

22222

22

sincos4sin122

sin1

nnnn

nn

uII

II

minmax

minmax

minmax

minmax

II

II

u: modification factor•Raises DOP•Assumption

•Closed smooth object•“u” is constant

Definition of DOP:

Modified DOP:

u

Page 16: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

16

Surface normalSurface normal

φ

θSurface normal

Page 17: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

17

HeightHeight

• Relaxation method

dxdyqy

Hp

x

H22

Minimize: where,

x

Hp

y

Hq

Gradient Height H

y

q

x

pyxHyxH ii

4

1),(),( )()1(Iteratively update:

[Ikeuchi1984]

221

1

qp

qp T

nSurface

normal

Page 18: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

3. Illumination estimation3. Illumination estimation

Page 19: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

19

Illumination sphereIllumination sphere

θ=0°

θ=90°θ=90°

θ=180°

Object

Light source is represented in polar coordinate system (θ, φ)

φ=0°

φ=90°

φ=180°

φ=270°

L1=(θ1, φ1)L2=(θ2, φ2)

L3=(θ3, φ3)

Page 20: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

20

Illumination estimationIllumination estimation

Detect position of intensity peakDetermine light source orientation from the peak

1.Project to (θ, φ)-space 2.Thresholding 3.Detect intensity peak

Page 21: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

4. Reflection parameters estimation4. Reflection parameters estimation

Page 22: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

22

Torrance-Sparrow reflection modelTorrance-Sparrow reflection model

Specular reflection

2

2

2

cos

1cos

α

θθ eKKI

rsid

Diffuse reflection

Incidentlight

Surfacenormal

ViewBisector

Object surface

αθi θr

Known: θi, θr, α

Unknown:•Diffuse reflection scale; Kd

•Specular reflection scale; Ks

•Surface roughness; σ

Page 23: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

23

Reflection parameters estimationReflection parameters estimation

Solve the following least-square problemby steepest-descent method

MinimizeKs, σ

renderedimage

realimage

2

2

2

2

cos

1 α

θeK

rs

Page 24: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

Experimental resultExperimental result

Page 25: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

25

InputInput

Intensity I

Azimuth angleφ

DOPρ

Page 26: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

26

Result of shape estimationResult of shape estimation

Page 27: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

27

Result of illumination estimationResult of illumination estimation

Actual illumination distribution Estimated illumination distribution

Page 28: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

28

Rendering resultRendering result

Input

Synthesizedimage Rendered image under

different illumination & view

Page 29: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

29

Result for another objectResult for another object

Input

Synthesizedimage

Estimatedshape

Rendered image underdifferent illumination & view

Page 30: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

30

ConclusionsConclusions

• Estimated geometrical, photometrical, environmental information in one integrated framework– Shape from polarization– Surface reflection parameters from iterative

computation– Illumination from intensity peak

Page 31: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

31

Application to digital archiving projectApplication to digital archiving project• Multiple View

• Modeling a statue in a room– IBR with

• surface normal• reflection parameters

Photorealistic preservation

Page 32: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

FinFin

Page 33: Polarization-based Inverse Rendering from Single View Daisuke Miyazaki Robby T. Tan Kenji Hara Katsushi Ikeuchi

(c) Daisuke Miyazaki 2003(c) Daisuke Miyazaki 2003All rights reserved.All rights reserved.

http://www.cvl.iis.u-tokyo.ac.jp/D. Miyazaki, R. T. Tan, K. Hara, K. Ikeuchi,

"Polarization-based Inverse Rendering from Single View," in Proceedings of International Symposium on the CREST Digital Archiving Project, pp.51-65, Tokyo,

Japan, 2003.05