Upload
cameraculture-mit-media-lab
View
8.788
Download
0
Embed Size (px)
Citation preview
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Camera CultureAssociate Professor, MIT Media Lab
Photography Wishlist
http://raskar.info
Film-like Digital Photography
Fernald, Science [Sept 2006]
Shadow Refractive
Reflective
‘‘film-like’ Photographyfilm-like’ Photography
Lens
Detector
Pixels
Image
Slide by Shree NayarReproduce for the eye
Wish List TodayWish List Today
• ConsumersConsumers– Super-human visionSuper-human vision– Microscope like resolutionMicroscope like resolution– High speedHigh speed– See inside the body (health)See inside the body (health)– Auto-triggerAuto-trigger– Keep only ‘good’ picsKeep only ‘good’ pics– Find ‘relevant’ pics and better Find ‘relevant’ pics and better
archiving/accessarchiving/access
– Put photographer back in photo!Put photographer back in photo!
• CompaniesCompanies– CostCost– ResolutionResolution– Low-light sensitivity, HDRLow-light sensitivity, HDR– Stereo and 3DStereo and 3D– Mecha-free zoom/focusMecha-free zoom/focus– Auto-tagging for sharingAuto-tagging for sharing– RecognitionRecognition
Can you look around a corner ?
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Camera CultureAssociate Professor, MIT Media Lab
Computational Photography
Wish List
http://raskar.info
SynthesisLow Level Mid Level High
LevelHyper
realism
Raw
Angle, spectrum
aware
Non-visual Data, GPS
Metadata
Priors
Comprehensive
8D reflectance field
Digital
Epsilon
Coded
Essence
Computational Photography aims to make progress on
both axis
Camera ArrayHDR, FoV Focal stack
Decomposition problems
Depth
Spectrum
LightFields
Human Stereo Vision
Transient Imaging
Virtual Object Insertion
Relighting
Augmented Human
Experience
Material editing from single photo
Scene completion from photos
Motion Magnification
Phototourism
Resolution
Computational Photography
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Wish #1
Ultimate Post-capture Control
Marc Levoy
Digital Refocusing using Light Field Camera
125μ square-sided microlenses[Ng et al 2005]
Marc Levoy
Zooming into the raw photo
Mask based Light Field CameraMask Sensor
[Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan, Siggraph 2007 ]
MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan
Captured 2D Photo
MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan
x
y
u
v
121 Sub-aperture Views
MERL Mask-Enhanced Cameras: Heterodyned Light Fields & Coded Aperture Veeraraghavan, Raskar, Agrawal, Mohan & Tumblin
• Samples individual rays
• Predefined spectrum for lenses• Chromatic abberration
• High alignment precision
• For each lenslet, peripheral pixels are wasted
• Negligible Light Loss
• Samples coded combination of rays
• Supports any wavelength
• Reconfigurable f/#, Easier alignment
• No wastage• High resolution image for parts
of scene in focus
• 50 % Light Loss due to mask
Mask
Sensor
Microlens array
Sensor
Plenoptic Camera Heterodyne Camera
Motion Blur in Low Light
Fluttered Shutter CameraFluttered Shutter CameraRaskar, Agrawal, Tumblin Siggraph2006
Ferroelectric shutter in front of the lens is turnedopaque or transparent in a rapid binary sequence
Motion Blur in Low Light
Ramesh Raskar, Computational Illumination
Compact Programmable
Lights ?
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Wish #1
Ultimate Post-capture Control
•Digital Refocus and Motion blur
•Emulate studio light from compact flash
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Wish #2
Freedom from Form
Convert LCD into a big flat camera?Beyond Multi-touch: 3D Gestures
Large Virtual Camera for 3D Interactive HCI and Video Conferencing
Matthew Hirsch, Henry HoltzmanDoug Lanman, Ramesh Raskar
Siggraph Asia 2009
BiDi Screen
Touch + Hover using Thin, Depth Sensing LCD Sensor
Sensing Depth from Array of Virtual Cameras in
LCD
Lots of glass; Heavy; Bulky; Expensive
Shallow DoF with Simple Lens
Image Destabilization
Lens Sensor
Camera
Static Scene
Image Destabilization
Static Scene
Lens Motion Sensor Motion
Camera
Mohan, Lanman,Hiura, Raskar ICCP 2009
Small Aperture Photo
all-in-focus
Destabilized Small Aperture
focused in the front using destabilization
Adjusting the Focus Plane
focused in the middle using destabilization
Adjusting the Focus Plane
focused in the back using destabilization
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Wish #3
Understand the World
Convert single 2D photo into 3D ?Snavely, Seitz, SzeliskiU of Washington/Microsoft: Photosynth
Exploit Community Photo CollectionsU of Washington/Microsoft: Photosynth
Can you ‘see’ around a corner ?
Femto-Photography: Higher Dimensional Capture
FemtoFlash
UltraFast Detector
Computational OpticsSerious Sync
Bokode
Defocus blur of Bokode
Coding in Angle
Mohan, Woo, Smithwick, Hiura, Raskar [Siggraph 2009]
camera
sensor
circle of confusion circle of information
Bokode(angle)
Quote suggested by Kurt Akeley
Encoding in Angle, not space, time or wavelength
Product labels
Street-view Tagging
MIT media lab camera culture bokode.com mohan, woo, hiura, smithwick, raskar
cell-phone cameraclose to the Bokode
(10,000+ bytes of data)
capturing Bokodes
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Wish #3
Understand the World
•Identify/recognize Materials•3D Awareness
•Interact with information
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Wish #4
Sharing Visual Experience
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Wish #4
Sharing Visual Experience
•LifeLog Auto-summary•Privacy in public and
authentication• Great Photo-frames
6D Photo Frames
One Pixel of a 6D Display = 4D Display
1 2
11
2D 2D 2D
Single Pixel of
6D Frame
6D Photo Frames: Respond to Viewpoint + Ambient Light
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Wish #4
Sharing Visual Experience
•LifeLog Auto-summary•Privacy in public and
authentication •Hyper-real Photo Frames
•Print ‘material’
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Wish #5
Capturing Essence
Essence Photography
• Beyond physical realism• New Visual Art Forms
Yu, McMillan
What are the problems with ‘real’ photo in conveying information ?
Why do we hire artists to draw what can be photographed ?
Shadows
Clutter
Many Colors
Highlight Shape Edges
Mark moving parts
Basic colors
Depth Edges with MultiFlashDepth Edges with MultiFlashRaskar, Tan, Feris, Jingyi Yu, Turk – ACM SIGGRAPH 2004
Depth Discontinuities
Internal and externalShape boundaries, Occluding contour, Silhouettes
Our MethodCanny
Blind CameraBlind Camera
Sascha Pohflepp, U of the Art, Berlin, 2006
Scene Completion Using Millions of PhotographsHays and Efros, Siggraph 2007
Camera Culture
Ramesh Raskar
Wish #5
…………………
Photos of tomorrow: computed not recorded
http://scalarmotion.wordpress.com/2009/03/15/propeller-image-aliasing/
Raskar, Camera Culture, MIT Media Lab
Questions• What will a camera look like in 10,20 years?• How will a billion networked and portable cameras change
the social culture? • How will online photo collections transform visual social
computing?• How will movie making/new reporting change?• computational-journalism.com
2nd International Conference on Computational Photography
Papers due November 2,
2009
http://cameraculture.media.mit.edu/iccp10
• Ramesh Raskar and Jack Tumblin
• Book Publishers: A K Peters
• ComputationalPhotography.org
Camera Culture Group, MIT Media Lab Ramesh Raskar http://raskar.info
• Post-capture control• Emulate studio lights with compact flash• Focus and motion blur
• New forms• Flat camera, large LCDs as cameras• Image destabilization for larger aperture
• Understand the world• Real or fake• Place 2D photo into 3D• Look around corner• Bokode: long distance barcode
• Sharing• Lifelogs auto summary• Privacy/Verification• 6D photoframes
• Essence• New visual arts• Multi-flash camera• Delta-camera and Blind-camera
Computational Photography Wish ListSensor