Upload
kim-spears
View
76
Download
3
Embed Size (px)
DESCRIPTION
Procam and Campro. Computer Science Columbia University. Shree K. Nayar. Support: NSF, ONR Procams 2006. PROCAMS. Shree K. Nayar ,. Cameras for Projectors (Campro): Display better or new images (Geometry, Shadows, Radiometry, Focus, Multi-Projectors). P. C. - PowerPoint PPT Presentation
Citation preview
Procam and Campro
Shree K. Nayar
Computer ScienceColumbia University
Support: NSF, ONR
Procams 2006PROCAMS Shree K. Nayar,
Convergence of Cameras and Projectors
Cameras for Projectors (Campro):
Display better or new images
(Geometry, Shadows, Radiometry, Focus, Multi-Projectors)
P
C
Projectors & Cameras for Apps (ProCamApp):
Enhance existing or create new application
(Immersion, Augmentation, Navigation, Surgery)
P
C
Projectors for Cameras (Procam):
Capture better or new images
(Quality, Matting, Relighting, Refocusing)
P
C
PROCAMS Shree K. Nayar,
Brief History of the Flash
Gas FlameIbbetson
1840
FlashbulbChauffour
1893
ElectricFlashbubVierkotter
1925
CommercialFlashbub
Ostermeier
1930
DigitalCameraFlash
2006
Flash/No Flash
Multi-Flash
Flash Matting
PROCAMS Shree K. Nayar,
Projector as a Programmable Flash
Depth from Projector Defocus
Image-Complete Depth Maps
(with Li Zhang, Siggraph 2006)
Separation of Direct and Global Illumination
Scattering Properties of Real Materials
(with Krishnan, Grossberg, Raskar, Siggraph 2006)
PROCAMS Shree K. Nayar,
Focal Plane SurfaceCamera
Depth from Camera Defocus
( Pentland 87, Subbarao 88, Darrell and Wohn 88,Nayar and Nakagawa 89, Ens and Lawrence 91, Bove 93, Nayar et al. 95, Watanabe et al. 98 )
Camera Defocus:
Captured
PROCAMS Shree K. Nayar,
Focal Plane Surface Projector
Lamp
Projector Defocus
Projector Defocus:
Projected
PROCAMS Shree K. Nayar,
Camera Defocus:
Captured
Camera Defocus vs. Projector Defocus
Projector Defocus:
Projected
PROCAMS Shree K. Nayar,
Focal Plane
LampSh
ift
Surface ProjectorCamera
Projector Defocus Model:
Temporal Defocus Method
Spectrum
Depth Estimation:PROCAMS Shree K. Nayar,
Projector
Half-mirror
Camera
Half-mirror
Projector
Camera
Experimental Setup
PROCAMS Shree K. Nayar,
Scene Depth Map
Green Screen Capture Depth-based Composition
Playing Cards: Object Insertion
PROCAMS Shree K. Nayar,
Multiple Projectors: (Bimber and Emmerling 2006)
Focused Projection at Multiple Depths
PROCAMS Shree K. Nayar,
CompensatedProjection
Front Plane Middle Plane Back Plane
Original Image
Normal Projection
Projector Defocus Compensation: Results
PROCAMS Shree K. Nayar,
Projector as a Programmable Flash
Depth from Projector Defocus
Image-Complete Depth Maps
(with Li Zhang, Siggraph 2006)
Separation of Direct and Global Illumination
Scattering Properties of Real Materials
(with Krishnan, Grossberg, Raskar, Siggraph 2006)
source
surface
P
Direct and Global Illumination
A
A : Direct
B
B : Interrelection
C
C : Subsurface
D
participating medium
D : Volumetric translucent surface
E
E : Diffusion
camera
Related Work
(Seitz et. al., ICCV 05)
• Inverse Light Transport
(Sen et. al., Siggraph 05)
T
• Dual Photography
Fast Separation of Direct and Global Images
PROCAMS Shree K. Nayar,
],[],[],[ icLicLicL gd
direct globalradiance
Direct and Global Components: Interreflections
surface
i
camera
source
P
g jiLjiAicL ],[],[],[
j
BRDF and geometryPROCAMS Shree K. Nayar,
High Frequency Illumination Pattern
surface
camera
source
fraction of activated source elements
],[],[],[ icLicLicL gd +
i
PROCAMS Shree K. Nayar,
High Frequency Illumination Pattern
surface
fraction of activated source elements
camera
source
],[],[],[ icLicLicL gd + - ],[],[ icLicL g )1(
i
PROCAMS Shree K. Nayar,PROCAMS Shree K. Nayar,
Other Global Effects
• Subsurface Scattering
• Volumetric Scattering
• Diffusion
PROCAMS Shree K. Nayar,
Diffuse Interreflections
SpecularInterreflections
Volumetric Scattering Subsurface
Scattering
Diffusion
PROCAMS Shree K. Nayar,
Photometric Stereo using Direct Images
Bowl
Shape
Source 1 Source 2 Source 3
Nayar et al., 1991
Direct
Global
PROCAMS Shree K. Nayar,
Hand
Direct Global
Skin: Hanrahan and Krueger
93,
Uchida 96, Haro 01, Jensen et
al. 01,
Cula and Dana 02, Igarashi et
al. 05, Weyrich et al. 05
PROCAMS Shree K. Nayar,
Summary
Coded Structured Light Shifted Sinusoids
Depth from Projector Defocus
(Minimum Images: 3)
Direct-Global Separation
(Minimum Images: 2)
PROCAMS Shree K. Nayar,
Open Questions
• What is Recoverable with a Projector and a Camera ?
• How Many Illumination Patterns?
• What are the Illumination Patterns?
PROCAMS Shree K. Nayar,
Cameras for Projectors Projectors for Cameras
Convergence of Cameras and Projectors
Projectors in 2006
SizePowerCost
x 1/10x 1/10x 1/10
PROCAMS Shree K. Nayar,