Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
1
Copyright (c) Benny Thörnberg 1:44
Machine visionImage analysis I
W
V
Benny Thörnberg
Assistant professor
in electronics
Copyright (c) Benny Thörnberg 2:44
Outline
•Uniform sampling and quantization
•Spatial and frequency domains
•Image filtering in frequency domain
•Spatial image processing
•Point processing
•Sliding window
•Geometric transformations
•Affine transformations
•Pixel interpolation
2
Copyright (c) Benny Thörnberg 3:44
Outline
•Region based image segmentation
•Global thresholding
•Local adaptive thresholding
•Contour based image segmentation
•Sobel and Laplacian operators
•Subpixel precise image segmentation
•Connected components and connectivity
•Component labeling
•Feature extraction
Copyright (c) Benny Thörnberg 4:44
Spatial domain - Uniform sampling and
Quantization
Reference: R.C. Gonzales and R.E. Woods, Digital Image Processing, Addison-Wesley
3
Copyright (c) Benny Thörnberg 5:44
Spatial domain - Uniform sampling and
Quantization
Reference: R.C. Gonzales and R.E. Woods, Digital Image Processing, Addison-Wesley
Gray-level and spatially
continuous imageUniformly sampled image with
digitized gray-levels
Copyright (c) Benny Thörnberg 6:44
Uniform sampling and Quantization
−−−−
−
−
=
)1,1()1,1()0,1(
)1,1()1,1()0,1(
)1,0()1,0()0,0(
),(
MNfNfNf
Mfff
Mfff
yxf
K
M
L
L
•Digitization of spatial coordinates (x,y) is called image sampling and
amplitude digitization is called gray-level quantization
{ } +∈∧−∈∧∈ ZRIIyxf R 12..0),( 2
Fundamental data structures for an image
•Image processing that operates directly on pixels in one single image is
called spatial image processing.
4
Copyright (c) Benny Thörnberg 7:44
Frequency domain – DFT - IDFT
{ } ∑∑−
=
−
=
+−
⋅≡=1
0
1
0
)(2
],[],[],[M
x
N
y
N
vy
M
uxj
eyxfvuFyxfFπ
2-dimensional discrete Fourier transform DFT
{ } ∑−
=
−
⋅≡=1
0
2
][][][N
x
N
uxj
exfuFxfF
π1-dimensional discrete Fourier transform DFT
{ } ∑∑−
=
−
=
+− ⋅≡=
1
0
1
0
)(21 ],[],[],[
M
u
N
v
N
vy
M
uxj
evuFyxfvuFFπ
2-dimensional inverse discrete Fourier transform IDFT
Copyright (c) Benny Thörnberg 8:44
Frequency domain - Examples
Spatial domainAmplitude spectrum
of Frequency domain
F
F
5
Copyright (c) Benny Thörnberg 9:44
Frequency domain - Examples
Spatial domain
F
F
Amplitude spectrum
of Frequency domain
Copyright (c) Benny Thörnberg 10:44
Image filtering in Frequency domain
Reference: R.C. Gonzales and R.E. Woods, Digital Image Processing, Addison-Wesley
6
Copyright (c) Benny Thörnberg 11:44
Image smoothing in Frequency domain
Reference: R.C. Gonzales and R.E. Woods, Digital Image Processing, Addison-Wesley
Copyright (c) Benny Thörnberg 12:44
Image smoothing in Frequency domain
Reference: R.C. Gonzales and R.E. Woods, Digital Image Processing, Addison-Wesley
Original Radii=5
Radii=30
Radii=230Radii=80
Radii=15
2:nd order Butterworth LPF
7
Copyright (c) Benny Thörnberg 13:44
Spatial image enhancement – point processing
( )( )12,0,5.0maxmin)( −++= bbaggf
Linear gray level transformation and quantization
12 −b
12 −b
g
f(g)
a
b
a: contrastb: brightness
Copyright (c) Benny Thörnberg 14:44
Spatial image enhancement – point processing
Histogram equalization
•The purpose with applying histogram equalization is to improve contrast
and normalize the image histogram
•We want to apply a gray level transform such that an arbitrary image will
have a uniform Probability Distribution Function.
•The PDF is usually not defined for an image but we can estimate the PDF
from calculation of the histogram.
8
Copyright (c) Benny Thörnberg 15:44
Spatial image enhancement – point processing
)(rTs =
Histogram equalization
Assume a monotonically increasing gray level transformation function
Hence, there exist a reverse function 1010)(1 ≤≤∧≤≤∧= −rssTr
)(1
)()(sTr
rsds
drrpsp
−=
⋅=(*) Relation between pdf of r and s is given by
)()()(0
rcdfdwwprTs
r
r === ∫Consider the following transformation
Then )(rpdr
dsr=
1)(
1)()(
)(1
=
⋅=
−= sTrr
rsrp
rpspWith (*), we have
Copyright (c) Benny Thörnberg 16:44
Spatial image enhancement – point processing
Histogram equalization - Example
9
Copyright (c) Benny Thörnberg 17:44
Spatial image enhancement – point processing
Histogram equalization - Example
Global histogram equalization Local histogram equalization
using 7x7 neighborhood
Reference: R.C. Gonzales and R.E. Woods, Digital Image Processing, Addison-Wesley
Copyright (c) Benny Thörnberg 18:44
Spatial image enhancement – point processing
Image averaging
•The purpose of image averaging is to reduce temporal noise by calculating
a mean value image out of a sequence of images
),(),(),( yxyxfyxg η+=We consider additive Gaussian noise
∑ ==
N
i i yxgN
yxg1
),(1
),(We calculate the mean value image
{ } 2
),(
2
),(
1),(),( yxyxg
NyxfyxgE ησσ =∧=It follows that
10
Copyright (c) Benny Thörnberg 19:44
Spatial image enhancement – point processing
Image averaging
Averaging of 8 images Averaging of 128 imagesSingle noisy image
Copyright (c) Benny Thörnberg 20:44
Spatial image enhancement – Sliding Window
Reference: R.C. Gonzales and R.E. Woods, Digital Image Processing, Addison-Wesley
+
−= −=
∈∧
−−⋅=
∗=
∑ ∑
ZS
byaxfbacyxg
cfg
S
Sa
S
Sb
),(),(),(
A filtered output image g(x,y) is calculated
through convolution with the input image f(x,y)
and a mask c(x,y).
Describes a square sized filter mask c of size 2S+1 x 2S+1
Filter mask c is often symmetric and has odd number of
coefficients in both dimensions.
Describes a FIR filter in 2 dimensions
11
Copyright (c) Benny Thörnberg 21:44
Spatial image enhancement – Sliding Window
1/25 1/25 1/25 1/25 1/25
1/25 1/25 1/25 1/25 1/25
1/25 1/25 1/25 1/25 1/25
1/25 1/25 1/25 1/25 1/25
1/25 1/25 1/25 1/25 1/25
∗
Copyright (c) Benny Thörnberg 22:44
Spatial image enhancement – Sliding Window
1/25 1/25 1/25 1/25 1/25
1/25 1/25 1/25 1/25 1/25
1/25 1/25 1/25 1/25 1/25
1/25 1/25 1/25 1/25 1/25
1/25 1/25 1/25 1/25 1/25
Frequency response of a 5x5 mean filter
12
Copyright (c) Benny Thörnberg 23:44
Geometric transformations – Affine
transformations
Translation
+
+=
⋅
=
0
0
0
0
*
*
110
01
YY
XXY
X
Y
X
Y
X
New coordinates (X*,Y*) are calculated for all previous coordinates (X,Y)
such that a translational movement (X0,Y0) becomes visual.
+
+
=
⋅
=
11100
10
01
1
0
0
0
0
*
*
YY
XX
Y
X
Y
X
Y
X
The use of square matrixes simplifies sequential steps of several matrix operations
An affine transformation preserves lines, planes, parallel lines and parallel planes
Copyright (c) Benny Thörnberg 24:44
Geometric transformations – Affine
transformations
+−
+
=
⋅
−=
1
cossin
sincos
1100
0cossin
0sincos
1
*
*
θθ
θθ
θθ
θθ
YX
YX
Y
X
Y
X
=
⋅
=
11100
00
00
1
*
*
YS
XS
Y
X
S
S
Y
X
Y
X
Y
X
Scaling
Rotation clockwise
X
Y
θ
(X,Y)
(X* ,Y*)
13
Copyright (c) Benny Thörnberg 25:44
Geometric transformations – Pixel
interpolation
Problem definition
Reference: C. Steger, M. Ulrich and C. Wiedermann, Machine Vision Algorithms and Applications
Copyright (c) Benny Thörnberg 26:44
Geometric transformations – Pixel
interpolation
Reference: C. Steger, M. Ulrich and C. Wiedermann, Machine Vision Algorithms and Applications
Bilinear interpolation
( ) ( )00100111 )1()1()1(~ gaagbgaagbg −+−+−+=
14
Copyright (c) Benny Thörnberg 27:44
Geometric transformations
Reference: C. Steger, M. Ulrich and C. Wiedermann, Machine Vision Algorithms and Applications
Nearest neighbor versus Bilinear interpolation
Nearest neighbor
Bilinear interpolation
Copyright (c) Benny Thörnberg 28:44
Geometric transformations
Reference: C. Steger, M. Ulrich and C. Wiedermann, Machine Vision Algorithms and Applications
Aliasing effects at down scaling
Down scaled by a factor 3 using
bilinear interpolation
Same as above but preprocessed
with smoothing
15
Copyright (c) Benny Thörnberg 29:44
Geometric transformations
Reference: C. Steger, M. Ulrich and C. Wiedermann, Machine Vision Algorithms and Applications
Polar transformation Perspective transformation
Copyright (c) Benny Thörnberg 30:44
Image segmentation
Reference: C. Steger, M. Ulrich and C. Wiedermann, Machine Vision Algorithms and Applications
•We want to extract regions in the image that correspond to the image
objects of interest.
•Image segmentation takes an image as input and returns one or more
regions or subpixel precise contours as output.
•Gray-levels, motion (in case of video), spectral analysis (colors) and texture
are examples of image properties that can be used as input to the
segmentation process.
•Image is decomposed into smaller pieces based on some kind of strategy
16
Copyright (c) Benny Thörnberg 31:44
Image segmentation
25590 maxmin =∧= gg
Reference: C. Steger, M. Ulrich and C. Wiedermann, Machine Vision Algorithms and Applications
Global gray-level thresholding R
{ }maxmin ),(|),( gcrfgRcrS ≤≤∈=
Copyright (c) Benny Thörnberg 32:44
Image segmentation
Reference: C. Steger, M. Ulrich and C. Wiedermann, Machine Vision Algorithms and Applications
Local dynamic gray-level thresholding
{ }diffgcrgcrfRcrS ≥−∈= ),(),(|),(
Assume a gray-level image f(r,c) and a its smoothened correspondence g(r,c).
Then the dynamic thresholding of bright objects on a darker background is:
Smoothened
image g(r,c)
Original
image f(r,c)
Has ability to adapt to non homogeneous illumination, hence variations in background
17
Copyright (c) Benny Thörnberg 33:44
Image segmentation – Edge detection
•Illumination typically varies over large areas which makes large regions to grow or shrink if
we apply a global threshold.
•Another robust region segmentation can be achieved if we focus on finding the borders of
regions.
•Finding borders is often done as edge detection.
Copyright (c) Benny Thörnberg 34:44
Image segmentation – Edge detection
First order derivate
Second order derivate
Reference: R.C. Gonzales and R.E. Woods, Digital Image Processing, Addison-Wesley
18
Copyright (c) Benny Thörnberg 35:44
Image segmentation – Edge detection
( )22
∂
∂+
∂
∂=∇
y
f
x
ffmag
The magnitude of the gradient vector is an obvious way to enhance edges in a picture
Z9Z8Z7
Z6Z5Z4
Z3Z2Z1
However, since this is a “computational demanding” operation,
we need to find approximations.
The Sobel operator is probably the most well known and used approximation
( ) ( ) ( ) ( ) ( )741963321987 2222 zzzzzzzzzzzzfmag ++−+++++−++≈∇
The Sobel operator can be implemented as two consecutive computational steps using the
following masks.
121
000
-1-2-1
10-1
20-2
10-1 The Sobel operators constitute both a
smoothing filter and a gradient
operator.
Copyright (c) Benny Thörnberg 36:44
Image segmentation – Sobel operator
19
Copyright (c) Benny Thörnberg 37:44
Image segmentation – Laplacian operator
The Laplacian is a second order derivate and is defined as:
)(4 86425
2zzzzzf +++−≈∇Laplacian is approximated as :
2
2
2
22
y
f
x
ff
∂
∂+
∂
∂=∇
0-10
-14-1
0-10
The Laplacian is very sensitive to noise and is for that reason preferable combined with a
smoothing filter (Gaussian). Zero crossings should be used to detect the edges.
Copyright (c) Benny Thörnberg 38:44
Subpixel-precise image segmentation
Reference: C. Steger, M. Ulrich and C. Wiedermann, Machine Vision Algorithms and Applications
A continuous image function is estimated from
bilinear interpolation of gray levels between four
pixel centers.
Subpixel precise thresholding
20
Copyright (c) Benny Thörnberg 39:44
Subpixel-precise edge detection
Reference: C. Steger, M. Ulrich and C. Wiedermann, Machine Vision Algorithms and Applications
The Laplacian operator can be combined with Gaussian smothing and subpixel precise zero
crossing in order to detect edges at subpixel precision.
Subpixel precise
thresholding on gray values
Laplacian and subpixel precise
zero crossing
Copyright (c) Benny Thörnberg 40:44
Extraction of connected components
{ } ),()1,(),1,(),,1(),,1()(4 yxpyxyxyxyxpN =∧−+−+=
Four-connectivity
p
),()1,1(),1,1(),1,1(),1,1(
),1,(),1,(),,1(),,1()(8 yxp
yxyxyxyx
yxyxyxyxpN =∧
−−+−−+++
−+−+=
Eight-connectivity
p
21
Copyright (c) Benny Thörnberg 41:44
Extraction of 4-connected components
Component labeling
P
r
t
First pass:
If(P=1) then
If( t=1 and r=1 and different labels) then
assign P to one of the labels and note equivalence
else If( t=1 or r=1) then
assign P to label of r or t
else
assign new label to P
else nothing
Second pass:
Look for all equivalent labels and assign those to unique labels
Copyright (c) Benny Thörnberg 42:44
Extraction of connected components
Component labeling
22
Copyright (c) Benny Thörnberg 43:44
Feature extraction
•Region features
- Compactness
- Anisometry
- Moments (can be normalized with respect to area
and/or position)
- Area, (p,q)=(0,0)
- Center of gravity
∑∈
=Ryx
qp
qp yxcrgm),(
, ),(
•Gray value features
- Max, min, mean, variance
- Moments
∑∈
=Ryx
qp
qp yxm),(
,