View
13
Download
0
Category
Preview:
Citation preview
Acquisition
Some slides from: Yung-Yu Chuang (DigiVfx) Jan Neumann, Pat Hanrahan, Alexei Efros
Image Acquisition
Digital Camera
Film
Outline
• Pinhole camera• Lens• Lens aberrations• Exposure• Sensors• Noise• Sensing color
Camera trial #1
scene film
Put a piece of film in front of an object.
source: Yung-Yu Chuang
Pinhole camera
scene film
Add a barrier to block off most of the rays• It reduces blurring• The pinhole is known as the aperture• The image is inverted
barrier
pinhole camera
Modeling projection
• The coordinate system– Put the optical center (Center Of Projection) at the origin– Put the image plane (Projection Plane) in front of the COP– The camera looks down the negative z axis
• we need this if we want right-handed-coordinates
Modeling projection
• Projection equations– Compute intersection with PP of ray from (x,y,z) to COP– Derived using similar triangles (on board)
– We get the projection by throwing out the last coordinate:
In Homogenous Coordinates
• Projection is a matrix multiply using homogeneous coordinates:
Projection
Projection
http://users.skynet.be/J.Beever/pave.htm
Pinhole camera
scene film
Add a barrier to block off most of the rays• It reduces blurring• The pinhole is known as the aperture• The image is inverted
barrier
pinhole camera
Shrinking the Pinhole Aperture
Why make the aperture as small as possible?• Less light gets through• Diffraction effect
Shrinking the Pinhole Aperture
Sharpest image is obtained when:
pinhole diameter
Example: If
f = 50mm,
= 600nm (red),
d = 0.36mm
λfd 2=
λ
High-end commercial pinhole cameras
~$200
Exposure 4 seconds Exposure 96 minutes
Images copyright © 2000 Zero Image Co.
Pinhole Images
Outline
• Pinhole camera
• Lens• Lens Aberrations• Exposure• Sensors• Noise• Sensing color
Adding a lens
scene film
Adding a lens
scene filmlens
“circle of confusion”
A lens focuses light onto the film• There is a specific distance at which objects are “in focus”• Other points project to a “circle of confusion” in the image
(Thin) Lens
• Any object point satisfying this equation is in focus
Thin lens equation:
Circle of Confusion
)'('
iiidb −=
• Blur Circle Diameter b : Derive using similar triangles
d
aperturediameter
io
'i'o
Blur Circle, baperture
Aperture controls Depth of Field
• Changing the aperture affects depth of field– Smaller aperture:
• better DOF• increased exposure
f / 5.6
f / 32
Depth of Field
http://www.cambridgeincolour.com/tutorials/depth-of-field.htm
Thick Lens
• Corrects aberrations• Change zoom
Field of View (Zoom) Field of View (Zoom)
FOV depends on Focal Length
d = image sizeϕ/2 d/2
f
2
f
FOV depends on Focal Length
d = image size
2
FOV depends on Focal Length
d = image size
2
Smaller FOV larger Focal Length
FOV depends on Focal Length
image size
focalpoint
For closer objects: if focal point is larger but image distance and sizeremain unchanged – the objects in focus are more distant.
Simplified Zoom Lens in Operation
From wikipedia
Outline
• Pinhole camera• Lens
• Lens aberrations• Exposure• Sensors• Noise• Sensing color
Radial Distortion
• Radial distortion of the image– Caused by imperfect lenses– Deviations are most noticeable for rays that pass
through the edge of the lens
No distortion Pin cushion Barrel
Correcting radial distortion
from Helmut Dersch
No Distortion Barrel Distortion Pincushion Distortion
ru = rd + k1 rd3
• Radial distance from Image Center:
Radial Distortions
ru = undistorted radius
rd = distorted radius
Before After
http://www.grasshopperonline.com/barrel_distortion_correction_software.html
Correcting Radial Distortions
photo by Robert Johnes
Vignetting
B
A
L3 L1L2
• More light passes through lens L3 for scene point A than scene point B
• Results in spatially non-uniform brightness (in the periphery of the image)
Vignetting
Chromatic Aberration
longitudinal chromatic aberration transverse chromatic aberration (axial) (lateral)
longitudinal chromatic aberration (axial)
Canon EF 85/1.2 L USM
transverse chromatic aberration (lateral)
Cosina 3.5-4.5/19-35 @ 20 mm
Chromatic Aberration
Good lens
Carl Zeiss Distagon2.8/21
http://www.vanwalree.com/optics/chromatic.html
Chromatic Aberration
Near Lens Center Near Lens Outer Edge • Rays parallel to the axis do not converge
• Outer portions of the lens yield smaller focal lengths
Spherical aberration
Spherical aberration
Spherical lens are free of chromatic aberration but do not focus well.Parabolic lens does.
Spherical aberration
Astigmatism
Different focal length for inclined rays
Astigmatism
Change in size and shape of blur patches
Comapoint off the axis depicted as comet shaped blob
Reading: http://www.dpreview.com
Lens Glare
• Stray inter-reflections of light within the optical lens system• Happens when very bright sources are present in the scene
Outline
• Pinhole camera• Lens• Lens aberrations
• Exposure• Sensors• Noise• Sensing color
Exposure• Two main parameters:
– Aperture (in f stop)
– Shutter speed (in fraction of a second)
Shutter Leaf Shutter
• Advantages– Uniform illumination– Entire frame illuminated at once
• Disadvantages– Illumination not constant over time
– Limitations on shutter speed
Focal Plane Shutter
• Advantages– Cost effective (one shutter needed
for all lenses– Can achieve very fast shutter speeds
(~1/10000 sec)
• Disadvantages– May cause time distortion
Aperture• Aperture is the diameter of the lens opening,
usually specified by f-stop, f/D, a fraction of the focal length.– f/2.0 on a 50mm means that the aperture is 25mm– f/2.0 on a 100mm means that the aperture is 50mm
• When a change in f-stop occurs, the light is either doubled or cut in half.
• Lower f-stop, more light (larger lens opening)
• Higher f-stop, less light (smaller lens opening)
Aperture
Constant speed.
Shutter speed – Rule of Thumb
• 1 step in the shutter speed scale corresponds to 1 stop in the aperture scale.
• Handheld camera: shutter speed = 1 / f
• Stabilized gear: 2-3 shutter speeds slower
• Typical speeds: 1/1000 s, 1/500 s, 1/250 s, 1/125 s, 1/60 s, 1/30 s, 1/15 s, 1/8 s, 1/4 s, 1/2 s 1 s
f/22 f/4
Depth of Field
Small Aperture Large Aperture
(Low speed) (High speed)
Aperture vs. Shutter
1/30 sec. @ f/22
Aperture vs. Shutter
1/6400 sec. @ f/2.5
Small Aperture Large Aperture
(Low speed) (High speed)
Motion Blur
Dynamic Range Short exposure10-6 106
10-6 106
Real worldradiance
Pictureintensity
dynamic range
Pixel value 0 to 255
Long exposure10-6 106
10-6 106
Real worldradiance
Pictureintensity
dynamic range
Pixel value 0 to 255
Varying shutter speeds
HDR – High Dynamic Range Outline
• Pinhole camera• Lens• Lens aberrations• Exposure
• Sensors• Noise• Sensing color
Spatial Sampling
• When a continuous scene is imaged on the sensor, the continuous image is divided into discrete elements - picture elements (pixels)
Spatial Sampling
0x
Sampling
• The density of the sampling denotes the separation capability of the resulting image
• Image resolution defines the finest details that are still visible by the image
• We use a cyclic pattern to test the separation capability of an image
Sampling Frequency
Sampling Frequency
1D Example:0
Nyquist Frequency
• Nyquist Rule: To observe details at frequency f (wavelength d) one must sample at frequency > 2f (sampling intervals < d/2)
•• The Frequency 2f is the NYQUIST frequency.
• Aliasing: If the pattern wavelength is less than 2d erroneous patterns may be produced.
Aliasing - Moiré Patterns Quantization
Digitizers (Quantization) Image Sensors• CCD
– Charge Coupled Device
• CMOS– Complementary Metal Oxide Semiconductor
MOS (Metal Oxide Semiconductor)
• Photosensitive element• Charge acquired depends on the number of
photons which reach the element• CCD devices are arrays of this basic element
Photoelectric Effect
photon phot
on
Hole Electron
Incr
easi
ng e
nerg
y
Valence Band
Conduction Band
1.26eV
• Thermally generated electrons are indistinguishable from photo-generated electrons ‘Dark Current’.
Quantum Efficiency• Not every photon hitting a pixel creates a free electron• Quantum Efficiency (QE) =
– electrons collected / photons hitting the pixel
• QE heavily depends on the wavelength
• QE < 100% degrades the SNR of a camera
• Typical max QE values : 25% (CMOS) … 60% (CCD)
pe SNRQESNR =
QE [%]
lambda [nm]
blue green red
CCD (Charge Coupled Device)
• Boyle and Smith, 1969, Bell Labs• Converts light into electrical signal (pixels)
CCD Readout
•Charge Amplifier
Bucket Brigade• Integration• Charge Shift and
Read-out
CMOS (Complementary Metal-Oxide Semiconductor)
• Each pixel owns its own charge-voltage conversion• No need for external shutter (electronic shutter)• The chip outputs digital bits• Much faster than CCD devices
CCD vs. CMOS• Mature technology• Specific technology• High production cost• High power consumption• Higher fill factor• Blooming• Sequential readout
• Recent technology• Standard IC technology• Cheap• Low power• Less sensitive• Per pixel amplification• Random pixel access• Smart pixels• On chip integration
with other components
Sensor Parameters
• Fill factor– The area in the sensor that is truly sensitive to light– Shift registers and others can reduce it up to a 30%
• Well capacity– The quantity of charge that can be stored in each pixel– Close relation with pixel dimensions
• Integration time:– Exposure time that is required to excite the CCD elements– Depends on the scene brightness
• Acquisition time:– Time needed to transfer the information gathered by the CCD– Depends on the number of pixels in the sensor
Fill Factor
• The ratio between the light sensitive pixel area and the total pixel area.
Total pixel area:5µm x 5µm
Fill factor ≈ 40%
PhotoSensing
Area
Outline
• Pinhole camera• Lens• Lens aberrations• Exposure• Sensors
• Noise• Sensing color
Sensor noise
Noise Sources
• Photon noise / Shot Noise (Poisson)
• Dark Noise (Constant)
•Thermal noise (Poisson)
• Resetting (fixed)
• Read-out noise
• Blooming
(After T. Lomheim, The Aerospace Corporation)http://www.stw.tu-ilmenau.de/~ff/beruf_cc/cmos/cmos_noise.pdf
Photon Shot Noise
• Light is quantum in nature • Noise due to statistics of the detected photons
themselves• The probability distribution for N photons to be
counted in an observation time T is Poisson
• F = fixed average flux (photons/sec)
( ) ( )!
,|N
eFTTFNPFTN −
=
0 2 0 4 0 6 0 8 0 1 0 00
0 .0 5
0 .1
0 .1 5
0 .2
Poisson Distribution, FT = 5
N
Poisson Distribution : std equals sqrt of Mean.
photonsshot NFT ==2σ
0 2 0 4 0 6 0 8 0 1 0 00
0 .0 2
0 .0 4
0 .0 6
0 .0 8
0 .1
0 .1 2
0 .1 4
Poisson Distribution, FT = 10
N
0 2 0 4 0 6 0 8 0 1 0 00
0 .0 2
0 .0 4
0 .0 6
0 .0 8
0 .1
Poisson Distribution, FT = 20
N
Poisson Distribution, FT = 50
0 2 0 4 0 6 0 8 0 1 0 00
0 .0 1
0 .0 2
0 .0 3
0 .0 4
0 .0 5
0 .0 6
N
• As FT grows, Poisson distribution approaches Gaussian distribution.
• Signal To Noise (SNR) Increases with Mean.
NNNNSNR
2
2
2==
σ=
Photon Noise
• More noise in bright parts of the image
• You can identify the white and black regions from the noise image
Photon NoisePhoton Noise more noticeable in dark images.
Dark Current Noise
• Electron emission when no light• Dark current noise is high for long exposures• To remove (some) of it
– Calibrate the camera (make response linear)– Capture the image of the scene as usual– Cover the lens with the lens cap and take another
picture– Subtract the second image from the first image
Original image + Dark Current Noise Image with lens cap on
Result of subtractionCopyright Timo Autiokari,
Dark Current Noise
Sensor noise
ideal relationship betweenelectrons and impinging photons
Photon Noise (QE = 50)Light Signal (QE = 50)
CCD capacity limit
Outline
• Pinhole camera• Lens• Lens aberrations• Exposure• Sensors• Noise
• Sensing color
Sensing Color
beam splitter
light
3 CCD Bayer pattern
Foveon X3TM
Multi-Chip
wavelengthdependent
Field Sequential Field Sequential
Field Sequential
Fuji Corporation
Color Filter Array (CFA)
Color filter array
Color filter arrays (CFAs)/color filter mosaics
Bayer pattern
Bayer’s pattern
Demosaicking CFA’s Color filter array
red green blue output
X3 technology
red green blue output
Foveon X3 sensor
X3 sensorBayer CFA
Cameras with X3
Sigma SD14 Polaroid X530 Hanvision HVDUO5M/10M
Out of production
Color processing
After color values are recorded, more color processing usually happens:
– White balance
– Non-linearity to approximate film response or match TV monitor gamma
White Balance
automatic white balancewarmer +3
Gamma Correction• Gamma correction applied by the converter
redistributes the pixel luminance values so that limited brightness range captured by the sensor is “mapped” to match our eye’s sensitivity. Gamma = 2.2 is a good match to distribute relative brightness in a print or in a video display.
Gamma =1 vs. Gamma = 2.2 Space of response curves
Outline
• Pinhole camera• Lens• Lens aberrations• Exposure• Sensors• Noise• Sensing color
• Summary
Camera pipeline
Sensor Response
• The response of a sensor is proportional to the radiance and the throughput
Measurement Equation
• Scene Radiance L(x,ω,t,λ)• Optics (x’,ω’) = T(x,ω,λ)• Pixel Response P(x,λ)• Shutter S(x,ω,t)
Degradation Due To Sampling
• Sampling in space
• Sampling in intensity
• Sampling in color
• Sampling in time
• Sampling in frequencyExposure
Quantization
Pixels
Lens and pixel PSF (point-spread-function)
Color Filter Array (CFA)
Recommended