Embed Size (px)
Citation preview
ME5286 – Lecture 2 (Theory)
Last Lecture
• What is Computer vision: deals with the formation, analysis and interpretation of Images
• Evolving field in AI• Enabler in Robotics as a smart sensor
#2
ME5286 – Lecture 2 (Theory)
Outline for this Lecture
• Image Formation• Cameras and Lenses• Human Visual System• Digital Cameras• Digital Color Images, sampling and
quantization
#3
ME5286 – Lecture 2 (Theory)
ME5286 – Lecture 2 (Theory)
Visible Spectrum
Light waves extend in wavelength from about 400 to 700 nanometers
#5
ME5286 – Lecture 2 (Theory)
Quantum Theory of Light
• Newton proposed that light is a stream of particles traveling in a straight line. Each particle is called a quantum and each quantum of light is a photon. Thus the intensity of light is measured in number of photons. – the visible spectrum is from 380 nm (violet) to 760 nm (red)
• refraction occurs when light enters a different medium causing the velocity of the light to change, this change bends the direction of the light
• Short wavelengths (violet) of light are refracted more than longer wavelengths (red). This is why a spectrum is formed from white light passing through a prism and it also causes the problem of chromatic aberration
#6
ME5286 – Lecture 2 (Theory)
The Electromagnetic Spectrum
• Radio Waves - communication • Microwaves - used to cook• Infrared - “heat waves”• Visible Light - detected by your eyes• Ultraviolet - causes sunburns• X-rays - penetrates tissue• Gamma Rays - most energetic
ME5286 – Lecture 2 (Theory)
The Multi-Wavelength Sun
X-Ray UV Visible
Infrared RadioComposite
ME5286 – Lecture 2 (Theory)
Image Formation
#9
ME5286 – Lecture 2 (Theory)
Light is everywhere#10
ME5286 – Lecture 2 (Theory)
Image Formation: Simple Model
Digital Camera
The Eye
Film
ME5286 – Lecture 2 (Theory)
Image formation• There are two parts to the image formation
process:
(1)The geometry, which determines where in the image plane the projection of a point in the scene will be located.
(2) The physics of light, which determines the brightness of a point in the image plane.
f(x,y) = i(x,y) r(x,y)Simple model:i: illumination, r: reflectance
ME5286 – Lecture 2 (Theory)
Image formation
• Let’s design a camera– Idea 1: put a piece of film in front of an object– Do we get a reasonable image? Blurring …
FilmObject
ME5286 – Lecture 2 (Theory)
Pinhole camera
• Add a barrier to block off most of the rays– This reduces blurring– The opening known as the aperture– How does this transform the image?
FilmObject Barrier
ME5286 – Lecture 2 (Theory)
History of Imaging: Camera Obscura
Device that led to Photography and the Camera"When images of illuminated objects ... penetrate through a small hole into a very dark room ... you will see [on the opposite wall] these objects in their proper form and color, reduced in size ... in a reversed position, owing to the intersection of the rays". Leonardo da Vinci
Slide credit: David Jacobs
History 1544
ME5286 – Lecture 2 (Theory)
Camera Obscura
• The first camera– How does the aperture size affect the image?– How does the size of the box affect the image?
ME5286 – Lecture 2 (Theory)
“Pinhole” camera model• The simplest device to form an image of a
3D scene on a 2D surface.• Rays of light pass through a "pinhole" and
form an inverted image of the object on the image plane.
perspective projection:center of projection
(x,y)
fXxZ
=fYyZ
=(X,Y,Z)
f: focal length, distance from pinhole to image plane
ME5286 – Lecture 2 (Theory)
Pinhole and the Perspective Projection
(x,y)
screen scene
Is an image being formedon the screen?
YES! But, not a “clear” one.
image plane
effective focal length, f’opticalaxis
y
x
zpinhole
),,( zyx=r
zy
fy
zx
fx
==''
''
zfrr
=''
)',','(' fyx=r
ME5286 – Lecture 2 (Theory)
What is the effect of aperture size?
Large aperture: light from the source spreads across the image (i.e., not properly focused), making it blurry!
Small aperture: reducesblurring but (i) it limits the amount of light entering the camera and (ii) causes light diffraction.
ME5286 – Lecture 2 (Theory)
Shrinking the aperture
• Why not make the aperture as small as possible?– Less light gets through
#20
ME5286 – Lecture 2 (Theory)
Shrinking more the aperture
• What happens if we keepdecreasing aperture size?
• When light passes through a small hole, it does not travel in a straight line and is scattered in many directions (i.e., diffraction)
ME5286 – Lecture 2 (Theory)
Shrinking the aperture
•Pinhole too big - many directions are averaged, blurring the image
•Pinhole too small -diffraction effects blur the image
•Generally, pinhole cameras are dark, because a very small set of rays from a particular point hits the screen.
#22
ME5286 – Lecture 2 (Theory)
Problems with Pinholes
• Pinhole size (aperture) must be “very small” to obtain a clear image.
• However, as pinhole size is made smaller, less light is received by image plane.
• If pinhole is comparable to wavelength of incoming light, DIFFRACTION effects blur the image!
• Sharpest image is obtained when:
pinhole diameter
Example: If f’ = 50mm,
= 600nm (red),
d = 0.36mm
λ'2 fd =
λ
ME5286 – Lecture 2 (Theory)
Traditional Photography
• A chemical process, little changed from 1826
• Taken in France on a pewter plate
• … with 8-hour exposure
The world's first photograph
24
ME5286 – Lecture 2 (Theory)
Lens Based Camera Obscura, 1568
History of Imaging: Adding a Lens
ME5286 – Lecture 2 (Theory)
First Camera Design#26
ME5286 – Lecture 2 (Theory)
The Reason for Lenses
Gather more light from each scene point
#27
ME5286 – Lecture 2 (Theory)
Adding a lens
• Pinhole replaced by a Lens
• Lens redirect light rays emanating from the object
• Lens improve image quality, leading to sharper images.
ME5286 – Lecture 2 (Theory)
Lenses
• A lens focuses parallel rays onto a single focal point– focal point at a distance f beyond the plane of the lens
• f is a function of the shape and index of refraction of the lens
– Aperture of diameter D restricts the range of rays• aperture may be on either side of the lens
– Lenses are typically spherical (easier to produce)
focal point
F
optical center(Center Of Projection)
#29
ME5286 – Lecture 2 (Theory)
Properties of “thin” lens (i.e., ideal lens)
• Light rays passing through the center are not deviated.• Light rays passing through a point far away from the center
are deviated more.
focal point
f
ME5286 – Lecture 2 (Theory)
Properties of “thin” lens (i.e., ideal lens)
• All parallel rays converge to a single point.• When rays are perpendicular to the lens, it is
called focal point.
focal point
f
ME5286 – Lecture 2 (Theory)
Properties of “thin” lens
• The plane parallel to the lens at the focal point is called the focal plane.
• The distance between the lens and the focal plane is called the focal length (i.e., f) of the lens.
focal point
f
ME5286 – Lecture 2 (Theory)
Thin lenses
• Thin lens equation:
– Any object point satisfying this equation is in focus– What is the shape of the focus region?– How can we change the focus region?– Thin lens applet: http://www.phy.ntnu.edu.tw/ntnujava/index.php (by Fu-Kwun Hwang )
FilmObject Lens
Focal point
Not quite right…
ME5286 – Lecture 2 (Theory)
object
fuv
image
Thin lens equationAssume an object at distance u from the lens plane:
ME5286 – Lecture 2 (Theory)
Thin lens equation (cont’d)
Using similar triangles:
y’/y = v/u
fuv
y’y
image
ME5286 – Lecture 2 (Theory)
Thin lens equation (cont’d)
fuv
y’y
y’/y = (v-f)/f
Using similar triangles:
image
ME5286 – Lecture 2 (Theory)
Thin lens equation (cont’d)
fuv
1 1 1u v f+ =
image
Combining the equations:
ME5286 – Lecture 2 (Theory)
Adding a lens
• A lens focuses light onto the film– There is a specific distance at which objects are “in
focus”• other points project to a “circle of confusion” in the image
– Changing the shape of the lens changes this distance
“circle of confusion”
FilmObject Lens
ME5286 – Lecture 2 (Theory)
Circle of Confusion#39
ME5286 – Lecture 2 (Theory)
Thin lens assumption
FilmObject Lens
Focal point
The thin lens assumption assumes the lens has no thickness, but this isn’t true…
By adding more elements to the lens, the distance at which a scene is in focus can be made roughly planar.
ME5286 – Lecture 2 (Theory)
Lens Aperture#41
ME5286 – Lecture 2 (Theory)
depth of field
• The size of blur circle is proportional to aperture size.
ME5286 – Lecture 2 (Theory)
depth of field trade off
• Changing aperture size (controlled by diaphragm) affects depth of field.– A smaller aperture increases
the range in which an object is approximately in focus (but need to increase exposure time).
– A larger aperture decreases the depth of field (but need to decrease exposure time).
ME5286 – Lecture 2 (Theory)
Depth of field trade off
• Changing the aperture size affects depth of field– A smaller aperture increases the range in which the object is
approximately in focus
f / 5.6
f / 32
http://en.wikipedia.org/wiki/Depth_of_field
FilmAperture
ME5286 – Lecture 2 (Theory)
Depth of Field
http://www.cambridgeincolour.com/tutorials/depth-of-field.htm
The range of depths over which the world is approximately sharp (i.e., in focus).
ME5286 – Lecture 2 (Theory)
Varying aperture size
Large aperture = small DOF Small aperture = large DOF
ME5286 – Lecture 2 (Theory)
Another Example
Large aperture = small DOF
ME5286 – Lecture 2 (Theory)
Field of View (Zoom)
ff
• The cone of viewing directions of the camera.• Inversely proportional to focal length.
ME5286 – Lecture 2 (Theory)
Field of View (Zoom)
ME5286 – Lecture 2 (Theory)
Lens Flaws: Chromatic Aberration
• Lens has different refractive indices for different wavelengths.• Could cause color fringing:
– i.e., lens cannot focus all the colors at the same point.
ME5286 – Lecture 2 (Theory)
Chromatic Aberration - Example
ME5286 – Lecture 2 (Theory)
Lens Flaws: Radial Distortion
• Straight lines become distorted as we move further away from the center of the image.
• Deviations are most noticeable for rays that pass through the edge of the lens.
ME5286 – Lecture 2 (Theory)
Lens Flaws: Radial Distortion
No distortion Pin cushion Barrel
ME5286 – Lecture 2 (Theory)
Lens Flaws: Tangential Distortion
• Lens is not exactly parallel to the imaging plane!
ME5286 – Lecture 2 (Theory)
• Cameras are a Copy of the Human Eye !
Human Eye#55
ME5286 – Lecture 2 (Theory)
Human Eye vs. the Camera
• We make cameras that act “similar” to the human eye
ME5286 – Lecture 2 (Theory)
Image formation in the eye
Light receptor
radiant energy
electrical impulses
Brain
57
ME5286 – Lecture 2 (Theory)
http://www.cas.vanderbilt.edu/bsci111b/eye/human-eye.jpg
Human Eye
• The eye has an iris like a camera
• Focusing is done by changing shape of lens
• Retina contains cones (mostly used) and rods (for low light)
• The fovea is small region of high resolution containing mostly cones
• Optic nerve: 1 million flexible fibers
Slide credit: David Jacobs
ME5286 – Lecture 2 (Theory)
Human Eye: Pupil• Hole or opening where light enters
– Or, the diameter of that hole or opening• Pupil of the human eye
– Bright light: 1.5 mm diameter– Average light: 3-4 mm diameter– Dim light: 8 mm diameter
• Camera– Wider aperture admits more light– Though leads to blurriness in the
objects away from point of focus
59
ME5286 – Lecture 2 (Theory)
Human Eye : Lens
• Focusing is achieved by varying the shape of the lens (i.e., flattening of thickening).
ME5286 – Lecture 2 (Theory)
Human Eye: Retina• Retina contains light sensitive cells that convert light
energy into electrical impulses that travel through nerves tothe brain.
• Brain interprets the electrical signals to form images.
ME5286 – Lecture 2 (Theory)
Retina up-close
Light
ME5286 – Lecture 2 (Theory) © Stephen E. Palmer, 2002
Conescone-shaped less sensitiveoperate in high lightcolor vision
Two types of light-sensitive receptors
Rodsrod-shapedhighly sensitiveoperate at nightgray-scale vision
ME5286 – Lecture 2 (Theory)
Rod / Cone sensitivity
The famous sock-matching problem…
ME5286 – Lecture 2 (Theory)
Human Eye - Color
• Three different types of cones; each type has a special pigment that is sensitive to wavelengths of light in a certain range:– Short (S) corresponds to blue– Medium (M) corresponds to green– Long (L) corresponds to red
• Ratio of L to M to S cones: – approx. 10:5:1
• Almost no S cones in the center of the fovea
400 450 500 550 600 650R
ELA
TIV
E A
BS
OR
BA
NC
E (%
)
W AVELENGTH (nm.)
100
50
440
S
530 560 nm.
M L
ME5286 – Lecture 2 (Theory)
Visual Cortex #66
ME5286 – Lecture 2 (Theory)
Visual Perception
• Modern view is that visual transformation is a creative process– Vision transforms light stimuli on the retina
into mental constructs of a stable 3D world– Visual perception is a 3D perception of the
world that is invariant to a wide range of changes in illumination, size, shape, and brightness of the image
#67
ME5286 – Lecture 2 (Theory)
Digital Image Formation
ME5286 – Lecture 2 (Theory)
Digital cameras• A digital camera replaces
film with a sensor array.
– Each cell in the array is light-sensitive diode that converts photons to electrons
– Two common types• Charge Coupled Device (CCD) • Complementary metal oxide
semiconductor (CMOS)
ME5286 – Lecture 2 (Theory)
What is a digital image?
8 bits/pixel
0
255
ME5286 – Lecture 2 (Theory)
sceneradiance
(W/sr/m )
∫sensorirradiance
sensorexposure
Lens Shutter
2
∆t
analogvoltages
digitalvalues
pixelvalues
CCD ADC Remapping
Image Acquisition Pipeline
ME5286 – Lecture 2 (Theory)
Digital Camera: Properties• Focus – Shifts the depth that is in focus.
• Focal length – Adjusts the zoom, i.e., wide angle or telephoto
lens.
• Aperture – Adjusts the depth of field and amount of light let into
the sensor.
• Exposure time – How long an image is exposed. The longer an
image is exposed the more light, but could result in motion blur.
• ISO – Adjusts the sensitivity of the “film”. Basically a gain
function for digital cameras. Increasing ISO also increases noise.
ME5286 – Lecture 2 (Theory)
Camera exposure• ISO number
– Sensitivity of the film or the sensor – Can go as high as 1,600 and 3,200
• Shutter speed– How fast the shutter is opened and closed
• f/stop– The size of aperture– 1.0 ~ 32
73
ME5286 – Lecture 2 (Theory)
Illumination
• 1 Lux = 1 Lumen/m2 = 0.093 ft-candles• Office = 100-1,000 Lux• Sunlight = 50,000 – 200,000 Lux• Cloudy day = 1,1000 Lux• Twilight = 1-10 Lux• Full moon = 0.1 – 1 Lux• Night sky = 10-9 – 10-8
#74
ME5286 – Lecture 2 (Theory)
Exposure#75
ME5286 – Lecture 2 (Theory)
Exposure Time#76
ME5286 – Lecture 2 (Theory)
Aperture & Shutter control Exposure#77
ME5286 – Lecture 2 (Theory)
Aperture vs. Shutter#78
ME5286 – Lecture 2 (Theory)
Long Exposure
10-6 106
10-6 106
Real world
Picture
0 to 255
High dynamic range
ME5286 – Lecture 2 (Theory)
Short Exposure
10-6 106
10-6 106
Real world
Picture
0 to 255
High dynamic range
ME5286 – Lecture 2 (Theory)
Varying Exposure
