Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Texture
Raul Queiroz Feitosa
Gilson A. O. P. Costa
10/3/2018 Texture 2
Objective
To introduce basic techniques to describe
texture.
10/3/2018 Texture 3
Content
Introduction
Texture Filters
Gray Scale Co-occurrence Matrix
Gabor Filter Bank
Local Binary Patterns
What is texture?
“Texture is a phenomenon that is widespread, easy
to recognize, and hard to define.”
10/3/2018 Texture 4
1 Computer Vision a Modern Approach, 2nd Ed., Forsyth.D.A., Ponce,. J., 2011
What is texture?
“If the brightness is
interpreted as elevation
in a representation of
the image as a surface,
then the texture is a
measure of the surface
roughness1.”
10/3/2018 Texture 5
1 Image Processing Handbook, 3rd Ed. 1999,
Jhon Huss
Sentinel 1 image sample
Venice, Italy (2016)
10/3/2018 Texture 6
10/3/2018 Texture 7
Content
Introduction
Texture Filters
Gray Scale Co-occurrence Matrix
Gabor Filter Bank
Local Binary Patterns
Texture filters
Compute some statistic on a neighborhood 𝑁 𝑝
around the pixel of interest 𝑝, whose value is assigned
to pixel 𝑝 in the output matrix.
Example of statistics:
Max-min
Standard deviation
Entropy
10/3/2018 Texture 8
mn = max𝑞∈𝑁 𝑝
𝐼 𝑞 − min𝑞∈𝑁 𝑝
𝐼 𝑞
𝜎 =1
# 𝑁 𝑝 − 1 𝐼 𝑞 − 𝐼 𝑞 2
𝑞∈𝑁 𝑝
1/2
𝐸 = − 𝑃 𝐼 𝑞
𝑞∈𝑁 𝑝
log 𝑃 𝐼 𝑞
mean value
in the
neighborhood
probability of
I(q) in the
neighborhood
Texture filters
Exercise 1:
Learn how to use the MATLAB functions that implement the texture filters
shown in previous slide:
rangefilt
stdfilt
entropyfilt
Show the results in a single figure using some of the Brodatz texture.
Look at the sensitivity to the neighborhood size.
Exercise 2:
Read the article Texture Segmentation using Texture Features from
MathWorks, and observe the sensitivity of the segmentation result to
neighborhood size.
10/3/2018 Texture 9
10/3/2018 Texture 10
Content
Introduction
Texture Filters
Gray Scale Co-occurrence Matrix
Gabor Filter Bank
Local Binary Patterns
Gray Level Co-occurrence Matrix
The GLCM is a tabulation
of how often different
combinations of pixel
brightness values (grey
levels) occur in an image.
10/3/2018 Texture 11
Haralick, R.M., K. Shanmugan, and I. Dinstein, "Textural Features for Image Classification",
IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC-3, 1973, pp. 610-621.
Robert. M. Haralick,
Distinguished Professor,
Universiy City of New York.
10/3/2018 Texture 12
Co-occurrence matrix
P(i,j,d,a) is the number of times two pixels at a
distance d forming an angle a occur, so that one
has gray level i and the other j. Example: d=1 (non symmetric)
)3,3(#)2,3(#)1,3(#)0,3(#3
)3,2(#)2,2(#)1,2(#)0,2(#2
)3,1(#)2,1(#)1,1(#)0,1(#1
)3,0(#)2,0(#)1,0(#)0,0(#0
3210
gray level
grey
level
image
1000
1300
0020
0122
HP
a=0º
0200
0122
0020
0003
VP
a=90º
0100
0220
0010
0012
LDP
a=45º
0200
0013
0011
0001
SDP
a=135º
0 0 1 1
0 0 1 1
0 2 2 2
2 2 3 3
10/3/2018 Texture 13
Co-occurrence matrix
A variant, called symmetric, does not consider the
order of pixel values. Thus P(i,j,d,a) = P(j,i,d,a)
and the co-occurrence matrix is symmetric.
)3,3(#)2,3(#)1,3(#)0,3(#3
)3,2(#)2,2(#)1,2(#)0,2(#2
)3,1(#)2,1(#)1,1(#)0,1(#1
)3,0(#)2,0(#)1,0(#)0,0(#0
3210
gray level
grey
level
2100
1601
0042
0124
HP
a=0º
0200
2222
0240
0206
VP
a=90º
0100
1420
0221
0014
LDP
a=45º
0200
0013
0121
0312
SDP
a=135º
Example: d=1 (symmetric)
image
0 0 1 1
0 0 1 1
0 2 2 2
2 2 3 3
Texture features from GLCM
In what follows the co-occurrence matrices are
normalized, so that their elements can be regarded
as probabilities, i.e., they must sum up to 1.
10/3/2018 Texture 14
𝑝 𝑖, 𝑗 =𝑃 𝑖, 𝑗
𝑃 𝑖, 𝑗𝑖,𝑗
10/3/2018 Texture 15
Texture features from GLCM
Descriptor Explanation Formula
Contrast A measure of the contrast between a pixel and its neighbor
over the whole image. Contrast is 0 for a constant image.
Correlation
A measure of how correlated a pixel is to its neighbor over
the whole image. Correlation is 1 or -1 for a perfectly
positively or negatively correlated image.
Energy
(Uniformity)
A measure of uniformity in range [0 1]. Energy is 1 for a
constant image.
Homogeneity
Measures the spatial closeness of the distribution of
elements to the diagonal. The range of values is [0 1].
Homogeneity is 1 for a diagonal GLCM.
𝑖 − 𝑗 2
𝑖,𝑗
𝑝 𝑖, 𝑗
𝑖 − 𝜇𝑖 𝑗 − 𝜇𝑗 𝑝 𝑖, 𝑗
𝜎𝑖𝜎𝑗𝑖,𝑗
for σi, σj ≠ 0
𝑝 𝑖, 𝑗 2
𝑖,𝑗
𝑝 𝑖, 𝑗
1 + 𝑖 − 𝑗𝑖,𝑗
𝜇𝑖 = 𝑖 𝑝 𝑖, 𝑗
𝑗𝑖
𝜇𝑗 = 𝑗 𝑝 𝑖, 𝑗
𝑖𝑗
𝜎𝑖2 = 𝑖 − 𝜇𝑖
2 𝑝 𝑖, 𝑗
𝑗𝑖
𝜎𝑗2 = 𝑗 − 𝜇𝑗
2 𝑝 𝑖, 𝑗
𝑖𝑗
where
MATLAB Example
% Read grayscale image and display it. The example converts the true color
% image to a grayscale image and then rotates it 90° for this example.
circuitBoard = rot90(rgb2gray(imread('board.tif')));
imshow(circuitBoard)
% Define offsets of varying direction and distance. The example specifies a set of
% horizontal offsets that only vary in distance.
offsets0 = [zeros(40,1) (1:40)'];
% Create the GLCMs.
glcms = graycomatrix(circuitBoard,'Offset',offsets0)
% Derive statistics from the GLCMs using the graycoprops function.
stats = graycoprops(glcms,'Contrast Correlation');
% Plot correlation as a function of offset.
figure, plot([stats.Correlation]);
title('Texture Correlation as a function of offset');
xlabel('Horizontal Offset')
ylabel('Correlation')
% The plot contains peaks at offsets 7, 15, 23, and 30. What does it mean?
% Taken from MATLAB help.
10/3/2018 Texture 16
MATLAB Example
% The plot contains peaks at offsets 7, 15, 23, and 30.
10/3/2018 Texture 17
GLCM
Exercise 3:
Write a MATLAB function that calculates the
GLCM features, having as input parameters
the input image in gray scale
the window size
the set of displacements and directions
Hint: try the MATLAB function blkproc.
10/3/2018 Texture 18
10/3/2018 Texture 19
Content
Introduction
Texture Filters
Gray Scale Co-occurrence Matrix
Gabor Filter Bank
Local Binary Patterns
Texture Representation using Filters
Local feature representation may be obtained by
filtering the image by a set of filters at various
scales and then preparing a summary.
The summary at a pixel is a description of the
texture in the neighborhood around that pixel.
10/3/2018 Texture 20
Gabor Filter Bank
Gabor filter kernels look like sine/cosine multiplied
by a Gaussian.
10/3/2018 Texture 21
Gabor Filter Bank
Gabor filter kernels look like sine/cosine multiplied
by a Gaussian.
10/3/2018 Texture 22
Gabor Filter Bank
Gabor filters come in pairs, often referred to as
quadrature pairs: symmetric and antisymmetric
components.
10/3/2018 Texture 23
cos 𝑘𝑥𝑥 + 𝑘𝑦𝑦 exp −𝑥2 + 𝑦2
2𝜎2 sin 𝑘𝑥𝑥 + 𝑘𝑦𝑦 exp −
𝑥2 + 𝑦2
2𝜎2
Gabor Filter Bank
Gabor filter responds strongly at points in an image
where there are components that locally have a
particular spatial frequency and orientation.
10/3/2018 Texture 24
Gabor Filter Bank
Exercise 4:
Read the help of the MATLAB (2016) functions
gabor that creates a Gabor filter bank.
imgaborfilt that computes the Gabor features.
Look at the texture features produced by each filter
and compare with the input image. Hint: Test the
functions for the bands of an IKONOS image or
any other gray scale image you wish.
10/3/2018 Texture 25
Gabor Filter Bank
Exercise 5: for MATLAB 2015
Read the MathWorks example of working with
Gabor filter for image segmentation.
Test it with images containing more than two
textures, such as,
the IKONOS image
a Sentinel 1 image.
10/3/2018 Texture 26
10/3/2018 Texture 27
Content
Introduction
Texture Filters
Gray Scale Co-occurrence Matrix
Gabor Filter Bank
Local Binary Patterns
10/3/2018 Texture 28
Local Binary Patterns – LBP
Each pixel is assigned to a pattern given by the sign of the
difference between the central pixel intensity and the intensity
of its m neighbors arranged over a circle of radius R.
interpolation is used whenever the sampling point does not fall
in the center of a pixel
m=8, R=1 m=16, R=2 m=8, R=2
10/3/2018 Texture 29
Local Binary Patterns – LBP
Illumination invariance is achieved by considering only the
signal of difference
Scale invariance is achieved by using multiple values for R
Rotation invariance is achieved by considering the pattern
circular and, for instance, by taking the rotation with the
minimum value. Example: all patterns below are considered
equivalent …
1 0 0 1 0 1 0 1
1 1 0 0 1 0 1 0
0 1 1 0 0 1 0 1
1 0 1 1 0 0 1 0
0 1 0 1 1 0 0 1
1 0 1 0 1 1 0 0
0 1 0 1 0 1 1 0
0 0 1 0 1 0 1 1 … to this one.
10/3/2018 Texture 30
Face Recognition with LBP
1. Compute LBP face representation (for example with
m=8, R=2)
x
y
X
Y
10/3/2018 Texture 31
Face Recognition with LBP
2. Compute the LBP histogram for each n×n non
overlapping block
. . .
0HX 1HX
kHX
10/3/2018 Texture 32
Face Recognition with LBP
3. Compute the χ2 distance between corresponding
blocks
. . .
. . .
c Y
b
X
b
Y
b
X
b
Y
b
X
b
cHcH
cHcHHH
2
2 ,
χ2( 0HX,0HY) χ2( 1HX,1HY) χ2( kHX, kHY) . . .
- - -
= = =
10/3/2018 Texture 33
Face Recognition with LBP
4. Compute the weighted sum of block distances
for instance,
with black squares indicate weight 0.0, dark gray 1.0,
light gray 2.0 and white 4.0.
b
Y
b
X
b
bYX HHwHH ,, 22
T. Ahonen, A. Hadid and M. Pietikäinen, Face description with local binary patterns: application to face recognition, IEEE
Transactions on Pattern Analysis and Machine Intelligence, vol. 28 no. 12, pp. 2037-2041, 2006
Face Recognition with LBP
5. In the so called uniform LBP only codes
containing up to 2 transitions 0 to 1 or 1 to 0
are considered. This reduces the number of
possible codes from 256 to 59.
10/3/2018 Texture 34
Face Recognition with LBP
Exercise 6: for MATLAB 2015 upwards
Download the file containing the LBP
implementation in MATLAB. Using the functions
Compute_Uniform_LBP and LBP_Discrepancy
determine the recognition rate for the face images in
file Images. mat.
Hint: for each image taken as a probe, determine the
most similar one. Compute the average percentage
of correct identification.
10/3/2018 Texture 35
10/3/2018 Texture 36
LBP in Remote Sensing
Texture (Local Binary Patterns – LBP)
LBP×GLCM for classification of Remote Sensing imagery
M. Musci*, R. Q. Feitosa, M. L. F. Velloso, T. Novack, G. A. Costa, Assessment of binary coding techniques for texture
characterization in remote sensing imagery , IEEE Geoscience and Remote Sensing Ltters, vol10(6, pp. 1607-1611, 2013.
Reference
1. Computer Vision A Modern Approach, 2nd Ed., Forsyth & Ponce, 2011,
2. T. Ahonen, A. Hadid and M. Pietikäinen, Face description with local
binary patterns: application to face recognition, IEEE Transactions on
Pattern Analysis and Machine Intelligence, vol. 28 no. 12, pp. 2037-2041,
2006
3. M. Musci, R. Q. Feitosa, M. L. F. Velloso, T. Novack, G. A. Costa,
Assessment of binary coding techniques for texture characterization in
remote sensing imagery , IEEE Geoscience and Remote Sensing Ltters,
vol10(6, pp. 1607-1611, 2013.
4. Oxford Visual Geometry Group:
http://www.robots.ox.ac.uk/~vgg/research/texclass/index.html
10/3/2018 Texture 37
10/3/2018 Texture 38
Texture
END