Upload
guillermo-fajardo
View
232
Download
0
Embed Size (px)
Citation preview
7/28/2019 Lec003 Filtering Convolution
1/18
CSE397/497-011 Real-time Image Processingfor Autonomous Robot Systems
Department of
Computer Science & Engineering
P.C. Rossin
College of Engineering
and Applied Science
Lecture 003:
Image Filtering & Convolution
CSE397/497 Real-time Image Processing
JR Spletzer
Administration
Lab sessions to start on Thursday
If you dont hear from me via e-mail, meet in PL322
Lab example
7/28/2019 Lec003 Filtering Convolution
2/18
CSE397/497 Real-time Image Processing
JR Spletzer
References Supporting Todays Lecture
Digital Image Processing, R. Gonzalez & R. Woods
HIPR2 Image Processing Learning Resources
http://homepages.inf.ed.ac.uk/rbf/HIPR2/
Introductory Techniques for 3-D Computer Vision, E.
Trucco & A. Verri
CSE397/497 Real-time Image Processing
JR Spletzer
Images are Discrete Functions!
Digital images are discrete functions that correspond to
the average scene luminance as perceived by the camera
over a period of time
Discrete spatially
Discrete quantization
7/28/2019 Lec003 Filtering Convolution
3/18
CSE397/497 Real-time Image Processing
JR Spletzer
Images are Discrete Signals!
Signal is visible light (the scene radiance)
Collector is our CCD array
While the signal is continuous, we sample at discrete
intervals (e.g. 30 Hz)
Both spatial and frequency representations
Image formed in the spatial domain
Can obtain frequency info through Discrete Fourier
Transform (DFT) of the image
We will focus on the spatial domain for now
CSE397/497 Real-time Image Processing
JR Spletzer
Since Images are Signals
They can be corrupted
Subject to random and additive noise (e.g. from
electronics)
Often assumed to be zero-mean Gaussian
Mathematical convenience
Some theoretical basis
2
2
2
)(
22
1)(
=
x
exf
7/28/2019 Lec003 Filtering Convolution
4/18
CSE397/497 Real-time Image Processing
JR Spletzer
Effects of Gaussian Noise (=3)
CSE397/497 Real-time Image Processing
JR Spletzer
Also from Impulsive Noise
Aka salt & pepper noise
Less chronic but more acute than Gaussian noise
Causes:
Transmission errors
Faulty CCD elements
External noise in AD conversion
Algorithm artifacts
7/28/2019 Lec003 Filtering Convolution
5/18
CSE397/497 Real-time Image Processing
JR Spletzer
Effects of Salt & Pepper Noise
CSE397/497 Real-time Image Processing
JR Spletzer
Noise Filtering
Q: Given a noise corrupted image, how to we go
about attenuating the noise while minimizing the
impact on the true signal ?
A: Signal processing.
We will look at filters for handling both random and
impulsive noise
First, we need to go way back in our minds to
calculus and rediscover
7/28/2019 Lec003 Filtering Convolution
6/18
CSE397/497 Real-time Image Processing
JR Spletzer
Convolution
Definition:
Q: What is the result of the convolution of 2
functions?
A: Another function.
dtgfgf )()(
= dtgfgfth )()()(
CSE397/497 Real-time Image Processing
JR Spletzer
So What Exactly does Convolution Do?
For a given value oft
Take the mirror ofg
Shift it by a given value oft
Multiply byf()
Integrate from
Repeat for every value oft from
Since our images are only defined over a finite
region, our range oft will be limited
= dtgfth )()()(
)( g
)( tg
)()( tgf
],[
],[
7/28/2019 Lec003 Filtering Convolution
7/18
CSE397/497 Real-time Image Processing
JR Spletzer
Sample Convolutions
http://mathworld.wolfram.com/Convolution.html
CSE397/497 Real-time Image Processing
JR Spletzer
Fun with Gaussians!
The convolution of 2 gaussians function is itself a
gaussian
Convolution can be used to motivate the central limit
theorem
Convolve 2 equally sized rectangle functions
* =
7/28/2019 Lec003 Filtering Convolution
8/18
CSE397/497 Real-time Image Processing
JR Spletzer
More fun with Gaussians!
Now convolve the rectangle function with the resultof the initial convolution
Is this starting to look familiar?
* =
CSE397/497 Real-time Image Processing
JR Spletzer
Discrete Convolution
For discrete functions, integration is replaced by a
summation
Discrete convolution can then be defined as:
In our case, the functionfwill correspond to our
image andg the filter kernel that we will use to
suppress noise
=
)()()( tgfth
7/28/2019 Lec003 Filtering Convolution
9/18
CSE397/497 Real-time Image Processing
JR Spletzer
Kernels are just Discrete Functions
A filter kernel or mask is an n x m array of numbers
It is no different than an image in that it is a discrete
function defined over the n x m array and 0
everywhere else
1-D Example:
Kernel is [-1 0 1]
2551313
1515012
111110
000
000
010
1)1(11)0(11)1(10
=
++=
Im2Im1
CSE397/497 Real-time Image Processing
JR Spletzer
So What Exactly does Convolution Do?
For a given value oft
Take the mirror ofg
Shift it by a given value oft
Multiply byf()
Integrate from
Repeat for every value oft from
Since our images are only defined over a finite
region, our range oft will be limited
= dtgfth )()()(
)( g
)( tg
)()( tgf
],[
],[
Original Image
Kernel
Resulting
Image
7/28/2019 Lec003 Filtering Convolution
10/18
CSE397/497 Real-time Image Processing
JR Spletzer
Linear Filter Takes a weighted average of the neighborhood. For a
3x3 we have
Also referred to as the box filter
Noise Filtering: Mean Smoothing
=
111
111
111
9
1k
CSE397/497 Real-time Image Processing
JR Spletzer
Noise Filtering: Mean Smoothing
In the frequency domain, this corresponds to the sinc
function
This acts as a low-pass filter by weighting frequencies
in the main lobes higher
Because of the secondary lobes, some noise can still
enter the filtered image
7/28/2019 Lec003 Filtering Convolution
11/18
CSE397/497 Real-time Image Processing
JR Spletzer
Noise Filtering: Gaussian Smoothing
Linear filter, low-pass filter
Based upon Gaussian distributions
Typically use a 5x5 (min and max)
Subtends 98.8% of the area when =1 pixel
Because of discretization, some high frequency noise is
not attenuated
CSE397/497 Real-time Image Processing
JR Spletzer
Generating Gaussian Kernels
1
1 1
1 2 1
1 3 3 1
1 4 6 4 1
Discrete 1-D kernel coefficients can be generated from
Pascals triangle
2-D coefficients can be obtained from convolving 2 1-D
kernels (horizontal & vertical components)
[ ]
=
121
242
121
16
1
1
2
1
121
7/28/2019 Lec003 Filtering Convolution
12/18
CSE397/497 Real-time Image Processing
JR Spletzer
3x3 Gaussian
Aka Tent Filter
Reduces high frequencies
more
Weights center sample more
Other samples weighted
linearly
[ ]
=
121242
121
16
1
12
1
121
* Filtering, R. Lin 2004
CSE397/497 Real-time Image Processing
JR Spletzer
Properties of Convolution
Commutative
Associative
Distributive
Linear
fggf =
hgfhgf = )()(
hfgfhgf +=+ )(
)()()( agfgafgfa ==
7/28/2019 Lec003 Filtering Convolution
13/18
CSE397/497 Real-time Image Processing
JR Spletzer
Separability of the 2-D Gaussian
12 IGI =
=
=
=
h k
kh
h k
kh
h k
kjhiIee
kjhiIe
kjhiIkhG
),(
),(
),(),(
1
22
12
1
2
2
2
2
2
22
CSE397/497 Real-time Image Processing
JR Spletzer
Separability of the 2-D Gaussian
This means that rather than convolve the image with
a 2-D n x n Gaussian kernel, we can first convolve it
by a horizontal 1-D gaussian, followed by a 1-D
vertical kernel
Theoretically, computation then will only increase by
a factor of2n vice n2
In practice, we will use the 2-D convolution because
Moving image data to/from memory costs instructions
We typically will use a 5x5 (or even 3x3) kernel
7/28/2019 Lec003 Filtering Convolution
14/18
CSE397/497 Real-time Image Processing
JR Spletzer
Synthetic Example: Gaussian Smoothing
CSE397/497 Real-time Image Processing
JR Spletzer
Gaussian Filtering Demo
7/28/2019 Lec003 Filtering Convolution
15/18
CSE397/497 Real-time Image Processing
JR Spletzer
Noise Filtering: Median Filter
Non-linear Filter useful for impulsive noise
Generate an n x n neighborhood around each pixel in
the original image
Take the median value of this neighborhood as the
value in the new image
n is typically small (3-5 are common)
));:,:(1(),(2
;2
mjmjmimiimmedianjiim
nfloorm
++=
=
CSE397/497 Real-time Image Processing
JR Spletzer
Noise Filtering: Median Filter
1010111110
1010111110
14142551313
14141515012
1010111110
1010111110
1010111110
14142551313
14141515012
1010111110
(10,11,11,12,13,13,15,150,255)
1010111110
1010111110
14142551313
1414151312
1010111110
Image 1
Image 2
7/28/2019 Lec003 Filtering Convolution
16/18
CSE397/497 Real-time Image Processing
JR Spletzer
Synthetic Image Example
CSE397/497 Real-time Image Processing
JR Spletzer
Handling Edge Effects
No perfect solution
Only convolve valid areas
Mirror the borders outside the image
Pad the image border with zeros & adapt kernel
1010111110
1010111110
14142551313
14141515012
1010111110
000000
0
0
0
0
0
1010111110
1010111110
14142551313
14141515012
1010111110
7/28/2019 Lec003 Filtering Convolution
17/18
CSE397/497 Real-time Image Processing
JR Spletzer
Putting the Pieces Together: Laboratory 1
Construct a vision system for automatic scene
surveillance
Part 1: Establish the Background of the Scene
Part 2: Separate the background from the
foreground
Part 3: Eliminate outliers from the data
CSE397/497 Real-time Image Processing
JR Spletzer
POP QUIZ : How do we do this???
Part 1:
?
Part 2:
?
?
Part 3:
?
7/28/2019 Lec003 Filtering Convolution
18/18
CSE397/497 Real-time Image Processing
JR Spletzer
Other Applications of Convolution: Edge Detection
Convolution with an appropriate kernel can yield the
image gradient
Discontinuities in the gradient correspond to edges.
Well talk more about this later
CSE397/497 Real-time Image Processing
JR Spletzer
Summary
Images are discrete digital signals subjected to additive,
random and impulse noise
We can attenuate this noise through a variety of filters
(mean, gaussian, median, et al)
This is accomplished by convolving the original image with
a discrete kernel generated from the filter
Convolution associativity can be used to speed upprocessing when multiple convolutions are required
(sometimes)
Because we are operating in a discrete world, results are
imperfect