All-Frequency Rendering with Dynamic, Spatially-Varying Reflectance

Jiaping Wang1 Peiran Ren1,3 Minmin Gong1 John Snyder2 Baining Guo1,3

1 Microsoft Research Asia 2 Microsoft Research 3 Tsinghua University

Complex, detailed reflectance Spatial/temporal variation All BRDF types:

parametric ↔ measuredisotropic ↔ anisotropicglossy ↔ mirror-like

Previous work

TemporalVariation

SpatialVariation

static

singleBRDF

SVBRDF

dynamic

Previous work

[Wang04]

[Ng03][Wang04][Liu04][Wang06][Tsai06][Krivanek08]

TemporalVariation

SpatialVariation

static

singleBRDF

SVBRDF

dynamic

Previous work

[Wang04]

[Green06][Green07]

[Green06]

TemporalVariation

SpatialVariation

static

singleBRDF

SVBRDF

dynamic

Previous work

[Wang04]

[Ben-Artzi06][Sun07][Ben-Artzi08]

[Green06]

[Ben-Artzi06]

TemporalVariation

SpatialVariation

static

singleBRDF

SVBRDF

dynamic

Previous work

[Wang04]

[Green06]

[Ben-Artzi06]

Our method:- per-pixel SVBRDF- dynamic SVBRDF- all BRDF types- all-frequency

[Wang09]

TemporalVariation

SpatialVariation

static

singleBRDF

SVBRDF

dynamic

Rendering Equation 2D lighting 4D visibility 6D reflectance (SVBRDF)

iinioiio d )max(0,),( ),,()(),( xVxLxR SVBRDFlight visibility cosine

o in

x

Precomputed Radiance Transfer

Dot product[Sloan et al. 2002]

iinoiiio d )max(0, ),,(),()(),( xxVLxR

light transferlight

Precomputed Radiance Transfer

Dot product[Sloan et al. 2002]

Triple product [Ng et al. 2004]

light transferlight

Light Transport & Precomputed

BRDF × cosinelight visibility

iinoiiio d )max(0, ),,(),()(),( xxVLxR

Precomputed Radiance Transfer

Dot product[Sloan et al. 2002]

Triple product [Ng et al. 2004]

Ours method

light transferlight

BRDF × cosinelight visibility

iinoiiio d )max(0, ),,(),()(),( xxVLxR

light BRDFvisibility cosine

LT & P LT & P

Precomputed Radiance Transfer

Dot product[Sloan et al. 2002]

Triple product [Ng et al. 2004]

Ours method

light transferlight

BRDF × cosinelight visibility

light BRDFvisibility cosine

LT & P LT & P LT & P

iinoiiio d )max(0, ),,(),()(),( xxVLxR

Algorithm Overview

Spherical Gaussians

o

Spherical Gaussians

Algorithm Overview

Spherical Gaussians SSDF

o

Environment

Dynamic, spatial varying BRDF

oOutline Reflectance Representation

Microfacet Model with SGs

Visibility Representation Signed Spherical Distance Function

Lighting & Rendering

oOutline Reflectance Representation

Microfacet Model with SGs

Visibility Representation Signed Spherical Distance Function

Lighting & Rendering

)1(),,;( pvpv eG

trivial rotationall-frequency signals

intensitysharpness

center

Spherical Gaussian (SG)

trivial rotationall-frequency signals

)1(),,;( pvpv eG

intensitysharpness

center

Spherical Gaussian (SG)

inner product: m

m

dd

eGGGG sinh4d)()(

21

212121

vvv

vector product:

)1(

212121m,;)()( pp

ppvvv meGGGGG mm

m

m

SG Mixtures

Sum of Multiple SGs:

Original SG, N = 7 SG, N = 3 SG, N = 1

),,;()(1

*iii

N

i

GF pvv

Microfacet BRDF Model

[Cook 82]

surface modeled by tiny mirror facets

Microfacet BRDF Model

[Cook 82]

surface modeled by tiny mirror facets

shadow term fresnel termnormal distribution

Represented by SG

single-lobe, analytic approximationCook-Torrance [Cook et al. 1981]

Ward [Ward 1992]

Blinn-Phong [Blinn 1977]

)1,2,;()( 2))(arccos( 2

mGeD m nhh nh

)1,2,;()( 2)()))(1( 222

nhh nhnh GeD

)1,,;()()( nGD n nhnhh

Parametric Models

Parametric BRDFs

nu=8, nv=128 nu=25, nv=400 nu=75, nv=1200

7-lo

be S

GM

grou

nd tr

uth

Anisotropic Parametric Models

Measured BRDFs

BRDF from [Matusik03] svBRDF from [Wang08] & [Lawance06]

Representation Efficiency Parametric BRDF

Texturing of original BRDF parameters isotropic : 7 float/texel: diffuse, specular, shininess Anisotropic: 8 float/texel: diffuse, specular, shininess u/v

Measured BRDFTexturing of SGs number

of SGsFloatsper SG

floats per texel

isotropic 1-3 4 4~12 + 3

anisotropic 2-7 6 12~42 + 3

)(hD

Normal Distributionin Half-vector Domain

ohhoh )(2)(

)()( 1 ii DW

)(iW

BRDF Slicein light-vector

o

BRDF Slices

Half-vector Domain

oioih

SG Warping

SG not closed under -1

approx. by per-SG warp of D*

))(()( 1** ii DW

DW

DDDDW

DDDW

opp

oppopp

4)(

)(2)(

)(* hD

))(( 1* iD

)(* iW

SG Warping

SG not closed under -1

approx. by per-SG warp of D*

)(* hD

))(( 1* iD

)(* iW

))(()( 1** ii DW

DW

DDDDW

DDDW

opp

oppopp

4)(

)(2)(

Parametric svBRDF Painting

oOutline Reflectance Representation

Microfacet Model with SGs

Visibility Representation Signed Spherical Distance Function

Lighting & Rendering

Visibility at one point

scene binary visibility function

x V(x,i)

Visibility Prerequisite Preserve sharp visibility boundary

inner product for Diffuse Term

vector product for Specular Term

iiinio d ),()max(0,)(),( xVLxRd

SGs ?

iiiinoio d )(),()max(0, ),,(),( LxVxxRs SGs ?

0)(if),arccos(min

1)(if),arccos(min)(

1)(

0)(

iit

iiti

t

t

V

VV

V

Vd

binary visibility, V(i)

i0

i1

SSDF, Vd (i)

Vd(i1)

Vd(i0)

Spherical Signed Distance Function

SSDF-SG Product

SSDF Visibility

SSDF-SG Product

p p

p

SG centered at p

SSDF Visibility

p≈

Approx. Visibility for p

p

Approx. Visibility for p

SSDF-SG Product

p p

p

SG centered at p

SSDF Visibility

p≈

Approx. Visibility for p

=0.329

Inner product vector product

Per-pixel Shading & Shadowing

oOutline Reflectance Representation

Microfacet Model with SGs

Visibility Representation Signed Spherical Distance Function

Lighting & Rendering

Point light

directional light

sr

GL ,ln,;)( 2* pii

22

2* ,

ln,;),(

xps

rxp

xpxpGxL

ii

Local Light Source

prefiltered MIPMAP[Kautz et al. 2000]

SGs (10 lobes)[Tsai and Shih 2006]

for diffuse shading for specular shading

Environment Light

prefiltered MIPMAP[Kautz et al. 2000]

SGs (10 lobes)[Tsai and Shih 2006]

for diffuse shading for specular shading

Environment Light

Environment + Local Lighting

Rendering Summary: Diffuse

Microfacet Model Environment Light

Rendering Summary: Diffuse

Microfacet Model Environment Light

Visibilityin SSDF

●

BRDF Slicein SGs

Cosine Termin SGs

Environmentin SGs

Rendering Summary: Specular

Microfacet Model Environment Light

BRDF Slicein SGs

Cosine Termin SGs

Visibilityin SSDF

PrefilteredEnvironment

●

Performance SummaryScene BRDF Type svBRDF

ResolusionsvBRDF

SizeEnv. FPS

Pt. FPS

Teapot CT (iso. 1 SG) 1024×1024 7.2MB 171 250Dragon Ward (iso. 1 SG) 512×512 1.8MB 165 231DishBall AS (aniso. 7 SGs) 512×1024 4.1MB 55 30

DishCardcard(iso. 2 SGs) 512×512 4.2MBsatin(aniso. 5 SGs) 850×850 22.4MB 48 35velvet (aniso. 2SGs) 850×850 9.4MB 168 145

Testing MachineIntel Core2 Duo 3.2G CPU, 4GB memorynVidia Geforce 8800 Ultra graphics card

Performance SummaryScene BRDF Type svBRDF

ResolusionsvBRDF

SizeEnv. FPS

Pt. FPS

Teapot CT (iso. 1 SG) 1024×1024 7.2MB 171 250Dragon Ward (iso. 1 SG) 512×512 1.8MB 165 231DishBall AS (aniso. 7 SGs) 512×1024 4.1MB 55 30

DishCardcard(iso. 2 SGs) 512×512 4.2MBsatin(aniso. 5 SGs) 850×850 22.4MB 48 35velvet (aniso. 2SGs) 850×850 9.4MB 168 145

Testing MachineIntel Core2 Duo 3.2G CPU, 4GB memorynVidia Geforce 8800 Ultra graphics card

bump maps dynamic BRDFs

anisotropic BRDFs

measured BRDFs

All-Frequency Visual Effects

parametric BRDFs

measured SVBRDFs

Reflectance Painting

Conclusion Overall method:

glossy to mirror-like, detailed, dynamic reflectanceall-frequency shadowsreal-time per-pixel shading

SG mixtures for microfacet-based reflectancecompact yet accuratefast rotation, warping, products

SSDFs for visibilityfast products with SG mixturesnon-ghosting spatial interpolation

Conclusion Overall method:

glossy to mirror-like, detailed, dynamic reflectanceall-frequency shadowsreal-time per-pixel shading

SG mixtures for microfacet-based reflectancecompact yet accuratefast rotation, warping, products

SSDFs for visibilityfast products with SG mixturesnon-ghosting spatial interpolation

Future work dynamic visibility inter-reflection anisotropic spherical Gaussian SSDF compression simpler SVBRDF acquisition

Future work dynamic visibility inter-reflection anisotropic spherical Gaussian SSDF compression simpler SVBRDF acquisition

Thank you for your attention.

Visibility Cuts [Cheslack-Postava et al.2008]

Light-cut frameworkNo highly glossy reflectanceHighly tessellationNot real-time

SG Scaling Shadowing and Fresnel terms

Assume low-frequency [Ashkmin01, Ngan05]

approx. by per-SG scale

))((4),()()(

nnFSS

W

WWW

W

W

opopop

pp

SSDF Compression

PCA:

M

jj

dj

d xwVxV1

)()(),( ii

binary visibility SSDF compressed SSDF

Error sources BRDF fitting error diffuse light fitting error SG warping error SSDF-SG product error SSDF compression error

SSDF-SG product errorvisibility function is approximateproduct is approximateinner product error typically less than 2%vector product error larger, but not visually

significanterror decreases as λ increases

SG mixturesN-lobe approximation

violet-acrylic NDF [Ngan et al. 2005]

original SG, N = 3L2 = 6.2%

SG, N = 2L2 = 8.2%

SG, N = 1L2 = 25%

),,;()(1

*iii

N

i

GF pvv

SG representation of lighting distant light → environment map (EM)

diffuse shading: ○ fit EM with SG mixture (10 lobes) [Tsai and Shih 2006]

specular shading: ○ prefilter EM with SGs of various λ [Kautz et al. 2000]

EM Prefiltering with SGs

……

MIPMAP of prefiltered cubemapsλ reduced by 1/4 per level

λ=21000,level 1

λ =329, level 4 λ =5.1,

level 7