View
222
Download
3
Embed Size (px)
Citation preview
CSCE 641 Computer Graphics: Image-based Rendering (cont.)
Jinxiang Chai
Outline
Light field rendering
Plenoptic sampling (light field sampling)
Layered depth image/Post-Rendering 3D Warping
View-dependent texture mapping
Unstructured lumigraph
Layered depth image [Shade et al, SIGGRAPH98]
Layered depth image:
- image with depths
Layered depth image [Shade et al, SIGGRAPH98]
Layered depth image:
- rays with colors and depths
Layered depth image [Shade et al, SIGGRAPH98]
Layered depth image: (r,g,b,depth)
- image with depths
- rays with colors and depths
Layered depth image [Shade et al, SIGGRAPH98]
Rendering from layered depth image
Layered depth image [Shade et al, SIGGRAPH98]
Rendering from layered depth image
- Incremental in X and Y
- Forward warping one pixel with depth
Layered depth image [Shade et al, SIGGRAPH98]
Rendering from layered depth image
- Incremental in X and Y
- Forward warping one pixel with depth
Layered depth image [Shade et al, SIGGRAPH98]
Rendering from layered depth image
- Incremental in X and Y
- Forward warping one pixel with depth
How to deal with occlusion/visibility problem?
How to form LDIs
Synthetic world with known geometry and texture - from multiple depth images
- modified ray tracer
Real images - reconstruct geometry from multiple images (e.g., voxel coloring, stereo reconstruction) - form LDIs using multiple images and reconstructed geometry
Kinect sensors - record both image data and depth data
Image-based Rendering Using Kinect Sensors
Capture both video/depth data using kinect sensors
Using 3D warping to render a video from a novel view point [e.g., Post-Rendering 3D Warping]
Demo: click here
Outline
Light field rendering
Plenoptic sampling (light field sampling)
Layered depth image/Post-Rendering 3D Warping
View-dependent texture mapping
Unstructured lumigraph
View-dependent surface representation
From multiple input image
- reconstruct the geometry
- view-dependent texture
View-dependent surface representation
From multiple input image
- reconstruct the geometry
- view-dependent texture
View-dependent surface representation
From multiple input image
- reconstruct the geometry
- view-dependent texture
View-dependent surface representation
From multiple input image
- reconstruct the geometry
- view-dependent texture
View-dependent texture mapping
[Debevec et al 98]
View-dependent texture mapping
Subject's 3D proxy points
V
C0 C2 C3C1
0
1
D
2
3
- Virtual camera at point D
- Textures from camera Ci mapped onto triangle faces
- Blending weights in vertex V
- Angle θi is used to compute the weight values:
wi = exp(-θi2/2σ2)
Videos: view-dependent texture mapping
Outline
Light field rendering
Plenoptic sampling (light field sampling)
Layered depth image/Post-Rendering 3D Warping
View-dependent texture mapping
Unstructured lumigraph
The Image-Based Rendering Problem
Synthesize novel views from reference images• Static scenes, fixed lighting
• Flexible geometry and camera configurations
The ULR Algorithm [Siggraph01]
Designed to work over a range of image and geometry configurations
Geometric Fidelity
# o
f Im
ag
es
VDTMVDTM
LFLF
The ULR Algorithm [Siggraph01]
Designed to work over a range of image and geometry configurations
Geometric Fidelity
# o
f Im
ag
es
VDTMVDTM
LFLFULRULR
The ULR Algorithm [Siggraph01]
Designed to work over a range of image and geometry configurations
Designed to satisfy desirable properties
Geometric Fidelity
# o
f Im
ag
es
VDTMVDTM
LFLFULRULR
Desired Camera
“Light Field Rendering,” SIGGRAPH ‘96
u0
s0
u
s
Desired color interpolated from “nearest cameras”
Desired Camera
“Light Field Rendering,” SIGGRAPH ‘96
u
s
Desired Camera
“The Scene”
“The Lumigraph,” SIGGRAPH ‘96
u
PotentialArtifact
“The Scene”
“The Lumigraph,” SIGGRAPH ‘96
Desired Property #2: Use of geometric proxy
Desired Camera
“The Lumigraph,” SIGGRAPH ‘96
“The Scene”
Desired Camera
“The Lumigraph,” SIGGRAPH ‘96
“The Scene”
Rebinning
Note: all images are resampled.
Desired Camera
Desired Property #3: Unstructured input images
“The Lumigraph,” SIGGRAPH ‘96
“The Scene”
Desired Property #4: Real-time implementation
Desired Camera
Desired Camera
View-Dependent Texture Mapping, SIGGRAPH ’96, EGRW ‘98
“The Scene”
Occluded
Out of view
Desired Camera
“The Scene”
Desired Property #5: Continuous reconstruction
View-Dependent Texture Mapping, SIGGRAPH ’96, EGRW ‘98
Desired Camera
“The Scene”
θ1
θ2
θ3
View-Dependent Texture Mapping, SIGGRAPH ’96, EGRW ‘98
Desired Camera
“The Scene”
θ1
θ2
θ3Desired Property #6: Angles measured w.r.t. proxy
View-Dependent Texture Mapping, SIGGRAPH ’96, EGRW ‘98
“The Scene”
Desired Camera
Desired Camera
“The Scene”
Desired Property #7: Resolution sensitivity
Demo