Dual Representations for Light Field Compression

EE368C ProjectJanuary 30, 2001

Peter ChouPrashant Ramanathan

Outline Background Model-based Coding Surface Light Fields Trade-offs Duality Proposed Experiments

Light Fields and Compression What are light fields?

2-D array of images

Why is compression necessary? Light fields are very large data

setsMichelangelo’s Night96 GB raw image data

Stanford Computer Graphics Laboratory

Mouse Hemispherical Light FieldUniversity of Erlangen

Light Fields with Geometry Geometry used for light fields to

aid compression ex. model-based coding

Light fields are used with geometry for more realistic rendering ex. surface light fields

Model-based Coding Model-based Coding of Multi-Viewpoint

Imagery (Magnor and Girod, VCIP-2000) Eigen-Texture Method: Appearance

Compression based on 3D Model (Nishino, Sato, and Ikeuchi, CVPR-1999)

http://www.lnt.de/~magnor

Surface Light Fields Surface Light Fields for 3D Photography (Wood

et al., Siggraph 2000)

http://grail.cs.washington.edu/projects/slf/

Surface Light Fields (cont’d) Geometry acquired through range

scan For each point on surface, a

lumisphere represents radiance in all directions

Lumispheres are coded using either: function quantization (similar to VQ) principal function analysis (similar to

PCA)

Trade-offs Textures

+ coherency along 4D coordinate directions

– warping introduces artifacts, and possible loss of information

Surface Light Fields+ more intuitive representation for

compression– lumispheres are represented as

continuous functions

Duality View-dominant organization

(textures)

Geometry-dominant organization (surface light fields)

View 1

View 2

View N

Surface Points

Surface Point 1

Surface Point 2

Surface Point N

Views

Proposed Experiments I Compare the two organizations for

any difference in compression results

View 1

View 2

View N

Surface Points

Surface Point 1

Surface Point 2

Surface Point N

Views

Proposed Experiments II Reparameterize geometry-

dominant organization using local coordinate system w.r.t. surface normals

Surface Point 1

Surface Point 2

Surface Point N

Views

Normal Direction View

Proposed Experiments III Use image data directly, instead of

converting from warped texture data

Surface Point 1

Surface Point 2

Surface Point N

Views

image pixels

Workplan

Week 1 Week 2 Week 3 Week 4 Week 5 Acquire Data Familiarize with source code Get programs running Convert between representations (Experiment 1) Apply compression

Perform reparameterization (Experiment 2)

Analyze results Write report Prepare presentation

Use images directly (Experiment 3)

Peter

Prashant

Joint