Upload
gene
View
24
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Surface Light Fields for 3D Photography. Daniel Wood Daniel Azuma Wyvern Aldinger Brian Curless Tom Duchamp David Salesin Werner Stuetzle. 3D Photography. Goals Rendering and editing Inputs Photographs and geometry Requirements Estimation and compression. - PowerPoint PPT Presentation
Citation preview
Surface Light Fieldsfor 3D PhotographyDaniel Wood Daniel Azuma Wyvern AldingerBrian Curless Tom DuchampDavid Salesin Werner Stuetzle
3D PhotographyGoalsRendering and editing
InputsPhotographs and geometry
RequirementsEstimation and compression
View-dependent texture mappingDebevec et al. 1996, 1998Pulli et al. 1997
View-dependent texture mappingDebevec et al. 1996, 1998Pulli et al. 1997
Two-plane light fieldLevoy and Hanrahan 1996Gortler et al. 1996
Surface light fieldsWalter et al. 1997Miller et al. 1998Nishino et al. 1999
Lumisphere-valued texture mapsLumisphere
OverviewDataacquisitionEstimationandcompressionRenderingEditing
OverviewDataacquisitionEstimationandcompressionRenderingEditing
Scan and reconstruct geometryReconstructed geometryRange scans(only a few shown . . .)
Take photographsCamera positionsPhotographs
Register photographs to geometryGeometryPhotographs
Register photographs to geometryUser selected correspondences (rays)
Parameterizing the geometryBase meshScanned geometryMap
Sample base mesh facesBase meshDetailed geometry
Assembling data lumispheresData lumisphere
OverviewDataacquisitionEstimationandcompressionRenderingEditing
Pointwise fairingFaired lumisphereData lumisphere
Pointwise fairing resultsInput photographPointwise faired(177 MB)
Pointwise fairingMany input data lumispheresMany faired lumispheres
CompressionSmall set of prototypes
Compression / EstimationSmall set of prototypesMany input data lumispheres
Reflected reparameterization
Reflected reparameterization
Reflected reparameterization
Reflected reparameterizationBeforeAfter
Median removal+ReflectedMedian(diffuse)Median-removed(specular)+
Median removalMedian valuesSpecularResult
Function quantizationCodebook of lumispheresInput data lumisphere
Lloyd iterationInput data lumispheres
Lloyd iterationCodeword
Lloyd iterationPerturb codewords to create larger codebook
Lloyd iterationForm clusters around each codeword
Lloyd iterationOptimize codewords based on clusters
Lloyd iterationCreate new clusters
Function quantization resultsInput photographFunction quantized(1010 codewords, 2.6 MB)
Principal function analysisSubspace of lumispheresInput data lumispherePrototype lumisphere
Principal function analysisApproximating subspacePrototype lumisphere
Principal function analysis
Principal function analysis
Principal function analysis resultsInput photographPFA compressed(Order 5 - 2.5 MB)
Compression comparisonPointwise fairing(177 MB)Function quantization(2.6 MB)Principal functionanalysis (2.5 MB)
Comparison with 2-plane light field(uncompressed)Pointwise-fairedsurface light field (177 MB)Uncompressedlumigraph / light field (177 MB)
Comparison with 2-plane light field(compressed)Compressed (PFA)surface light field (2.5 MB)Vector-quantizedlumigraph / light field (8.1 MB)
OverviewDataacquisitionEstimationandcompressionRenderingEditing
View-dependent level-of-detail
Render texture domain and coordinates in false color
Evaluate surface light field
Interactive rendererscreen capture
OverviewDataacquisitionEstimationandcompressionRenderingEditing
Lumisphere filteringOriginal surface light fieldGlossier coat
Lumisphere filtering
Rotating the environmentOriginal surface light fieldRotated environment
DeformationOriginalDeformed
Deformation
SummaryEstimation and compressionFunction quantizationPrincipal function analysis
RenderingFrom compressed representationWith view-dependent level-of-detail
EditingLumisphere filteringGeometric deformations and transformations
Future workBetter geometry-to-image registration
More complex surfaces (mirrored, refractive, fuzzy) under more complex illumination
Derive geometry from images
Combining FQ and PFA
AcknowledgementsMarc Levoy and Pat Hanrahan(Thanks for the use of the Stanford Spherical Gantry)
Michael Cohen and Richard Szeliski
National Science Foundation
The end
Geometry (fish)Reconstruction: 129,000 facesMemory for reconstruction: 2.5 MBBase mesh: 199 facesRe-mesh (4x subdivided): 51,000 facesMemory for re-mesh: 1 MBMemory with view-dependence: 7.5 MB
Light field data and rep (fish)Time to acquire: 1 hourInput images: 661Raw data size: ~500 MBLumisphere representation: 3 times subdivided octehedronLumisphere size: 258 directions
Lumigraph (fish)Lumigraph images: 400x400Lumigraph viewpoints per slab: 8x8Number of slabs: 6Lumigraph size (w/o geom): 184 MBLumigraph VQ dimension: 16384Lumigraph VQ codewords: 2x2x2x2x3Compressed size (w/o geom): 8.1 MB
Compression (fish)Pointwise faired: Memory = 177 MBRMS error = 9FQ (2000 codewords)Memory = 3.4 MBRMS error = 23PFA (dimension 3)Memory = 2.5 MBRMS error = 24PFA (dimension 5)Memory = 2.9 MBRMS error = ?
Pre-processing times (fish)Compute times on ~450 MHz P-IIIRange scanning time: 3 hoursGeometry registration: 2 hoursImage to geometry alignment: 6 hoursMAPS (sub-optimal): 5 hoursAssembling data lumispheres: 24 hoursPointwise fairing: 30 minutesFQ codebook construction (10%): 30 hoursFQ encoding: 4 hoursPFA codebook construction (0.1%): 20 hoursPFA encoding: 2 hours
Construct codebook using Lloyd iterationIterate until convergence:
Assign all data lumispheres to closest codeword, forming clusters.
Compute new codeword for each cluster by cluster-wise fairing.
Then split all codewords and start over.
Data extrapolationPhotographSurface light field
Comparison with 2-plane light field(uncompressed)Pointwise-fairedsurface light field (177 MB)Uncompressed2-plane light field (177 MB)
Comparison with 2-plane light field(compressed)Principal function analysissurface light field (2.5 MB)Vector-quantized2-plane light field (8.1 MB)
DetailsInput photographPointwise fairing(177 MB)Function quantization (3.4 MB)Principal function analysis (2.5 MB)