Scene3D: A Camera Simulation Pipeline For

Computational Photography and Imaging

Introduction

We built a multispectral camera simulation platform that utilizes 3D scenes,

that performs ray-tracing through lenses.

New features include diffraction and chromatic aberration simulation.

This tool can be used for rapid prototyping of novel computational photog-

raphy imaging systems such as flash/no-flash photography, 3D reconstruction.

Diffraction Simulation

Computational Photography Capabilities

Chromatic Aberration

No-flash Image Flash Image Combined Image

Flash/No-Flash Detail Transfer Algorithm [5, 11] Light-field [12]

Realistic Physically Based Renderings

Data Flow

Previous Dataflow

New Scene3D Dataflow

Motivation

Prototyping a novel imaging system is expensive, time-

consuming and cumbersome.

Experimental lenses, sensors, and cameras can be ex-

pensive and take time to manufacture.

We have little control over lighting conditions and object

placement in real-world scenes.

Andy L. Lin, Brian Wandell, Joyce Farrell

Diffraction Verification

An important lens artifact.

Arises from the variation of index of refraction

with wavelength[3].

Chromatic Aberration

Rendering on Slanted Bar

5x5 Camera Array Rendering

Scene Without

Chromatic Aberration

Scene With

Chromatic Aberration

PSF Comparisons Rendered PSF Cross-section Theoretical PSF Cross-section

The Heisenburg Uncertainty Ray Bending (HURB) [2] is used for diffraction simulation.

Rays hitting an aperture are deflected using a bivariate Gaussian distribution.

The spread of the distribution is dependent on wavelength and distance from aperture.

[1] C. Kolb, et al., "A realistic camera model for computer graphics," Computer Graphics (Proceedings of Siggraph '95), ACM SIGGRAPH, pp. 317-324, 1995. [2] E. R. Freniere, et al., "Edge diffraction in Monte Carlo ray tracing," Proceedings of SPIE, vol. 3780, 1999. [3] J. W. Goodman, Introduction to Fourier optics, 2nd ed. New York: McGraw-Hill, 1996. [4] J. M. DiCarlo, et al., "Illuminating illumination," in Proceedings of the Ninth Coloring Imaging Conference, ed Springfield, VA: IS&T, 2001, pp. 27-34. [5] G. Petschnigg, et al., "Digital photography with flash and no-flash image pairs," ACM Transactions on Graphics (TOG), vol. 23, pp. 664-672 2005. [6] Z. Zhou and E. R. Fossum, "Frame-Transfer CMOS Active Pixel Sensor with Pixel Binning," IEEE TRANSACTIONS ON ELECTRON DEVICES, vol. 44, pp. 1764-1768, October 1997.

[7] C. Zhou, et al., "Robust Stereo with Flash and No-flash Image Pairs," 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 342- 349 2012. [8] E. Eisemann and F. Durand, "Flash photography enhancement via intrinsic relighting," ACM Transactions on Graphics (TOG), vol. 23, pp. 673-678 2004. [9] A. Agrawal, et al., "Removing Photography Artifacts using Gradient Projection and Flash-Exposure Sampling," ACM Transactions on Graphics (TOG), vol. 24, pp. 828 - 835 205. [10] J. Sun, et al., "Flash Cut: Foreground Extraction with Flash and No-flash Image Pairs," IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR '07. , pp. 1- 8 2007. [11] Eisemann, Elmar, and Frédo Durand. "Flash photography enhancement via intrinsic relighting." ACM transactions on graphics (TOG). Vol. 23. No. 3. ACM, 2004. [12] Adelson, E.H., Bergen, J.R., "The Plenoptic Function and the Elements of Early Vision," In Computation Models of Visual Processing, M. Landy and J.A. Movshon, eds., MIT Press, Cambridge, 1991.

References

[1]