Upload
others
View
10
Download
0
Embed Size (px)
Citation preview
2012-09-10 E0005E, Lecture 2 1
INSTITUTIONEN FÖR SYSTEMTEKNIK
LULEÅ TEKNISKA UNIVERSITET
2012-09-10 E0005E, Lecture 2 2
Histogram equalization, geometric transformations and neighborhood operations
Matthew Thurley
E0005E Industrial Image Analysis
2012-09-10 E0005E, Lecture 2 3
Course Details
• Course web page - http://www.ltu.se/edu/course/E00/E0005E/E0005E-Industriell-bildanalys-1.36192?l=en
- Course guide – what you need to know
- Lecture notes
- Instructions for each development phase
- Other information
• Course textbook - Digital Image Processing, Rafael Gonzalez and Richard woods.
Pearson Education, 3rd edition
• Fronter for submission of reports
Last Lecture
• Digital Images
• Image Sampling & Quantisation
- Spatial Resolution
- Intensity Resolution
• Histograms
• Point Operations
- Lookup tables
- Contrast stretching
- Thresholding
2012-09-10 E0005E, Lecture 2 4
This Lecture
• Histogram Equalisation (you used this in Lab 1)
• Geometric Transformations
• Point Operations
• Neighbourhood Operations
• Noise Filtering
- Linear filtering
- Median Filtering
- Edge Preserving Filtering
2012-09-10 E0005E, Lecture 2 5
2012-09-10 6
Histogram Equalization
Speaking Notes: Histograms are the basis for many spatial domain processing algorithms,
image enhancement, thresholding, segmentation. They are simple to calculate and good for
real-time processing
Figures from DIP 3.3 Histogram Processing
2012-09-10 E0005E, Lecture 2 7
Histogram Equalization
2012-09-10 E0005E, Lecture 2 8
Histogram Equalization
Arbitrary histogram of an image
expressed as a probability function
If we adjusted the image intensities so that all
intensities occured at equal probabilties
(uniform distribution) then structure in the
image should be significantly more visible.
Quiz
Give a single intensity transformation function for spreading the intensities of an image so the lowest intensity is 0 and the highest L-1
What effect would setting to zero the lower-order bit planes have on the histogram of the image in general?
What effect on the histogram would setting the high-order bit planes to zero have instead?
2012-09-10 E0005E, Lecture 2 9
2012-09-10 E0005E, Lecture 2 10
Histogram Equalization Howto
• Calculate Intensity Histogram
• Calculate the Probability Function
• Calculate the cumulative sum of the probability function
• Scale the cumulative probability function by (L-1) the number of intensities
• The scaled cumulative probability function is the lookup table to achieve histogram equalisation
Intensity Frequency Probability
2012-09-10 E0005E, Lecture 2 11
Histogram Equalization Example
This is the scaled cumulative probability function, scaled from range [0..1] to [0..7], the range of intensities.
2012-09-10 E0005E, Lecture 2 12
Histogram Equalization
Many not be invertible as the left graph shows
GEOMETRIC TRANSFORMATIONS
E0005E Industrial Image Analysis
Matthew Thurley
2012-09-10 E0005E, Lecture 2 13
2012-09-10 E0005E, Lecture 2 14
Geometric Transformations
• Geometric transformations permit elimination of distortion in a captured image.
• Allow for matching of two different images of the same object
• Problem: Transform the distorted image to the correct geometric shape
• Textbook: linear only example in DIP 2.6 pg. 87
• Textbook: “Image Processing, Analysis, and Machine Vision” Sonka et. al. 4.2
Geometric Distortion in Images
• What we see in the image depends on the position of the camera
• Objects in the image may be affected by a noticable perspective distortion (such as this whiteboard, where the right side of the board looks shorter than the left side)
• It might be easier to read if we could transform the image so that the whiteboard appears rectangular again.
Images courtesy of Zhang & He (2003)
Transformed
Geometric Distortion in Images
• We might have a data projector which projects a rectangular image but we want to position the projector at an angle to the wall.
• The projected image will be affected by a noticable distortion (such as this image from a projector attached to the ceiling)
• It will be easier to read if we could transform the image so that when it is projected onto the wall it appears rectangular (modern data projectors can do this transformation)
• The transformation is function that can convert an image from one shape into another
Images courtesy of Wikipedia: Keystone effect
The image we want to see
Image as it appears on the wall
Undistorted image
Transforming the Image
• Here is a general equation to convert an arbitrary four sided polygon to another arbitrary four sided polygon
• We apply this equation to our images to calculate the corresponding ( , ) in the distorted image from any (i,j) in the undistorted image.
• We will use this to correct the distortion
• (i,j)
• ( , ) j ^ ^
i
i
j
j ^ ^
i Distorted image
Undistorted image
Transforming the Image
• The general equation contains 8 unknown values denoted a1, a2, a3, a4 and b1, b2, b3, b4 which we need to define so that the equation will transform between our images
• To solve for these 8 unknowns we need to create 8 simultaneous equations where we know the values of i,j, ,
• (i,j)
• ( , ) j ^ ^
i
i
j
Distorted image
j ^ ^
i
Transforming the Image
Undistorted image • We define 4 points in the undistorted image (i,j) , and 4 matching points in the distorted image ( , ) and we have all the values we need to create our 8 simultaneous equations
• Substituting these values into the general equation we can get our 8 simulataneous equations
•
(0,18)
j ^ ^
i
i
j
Distorted image
(0,0)
• •
• (60,0)
(60,40) (0,40)
•
•
• •
(54,0)
(60,32)
(15,32)
)32,15()40,0(
)32,60()40,60(
)0,54()0,60(
)18,0()0,0(
)ˆ,ˆ(),(
jiji
Undistorted image •
(0,18)
i
j
Distorted image
(0,0)
• •
• (60,0)
(60,40) (0,40)
•
•
• •
(54,0)
(60,32)
(15,32)
)32,15()40,0(
)32,60()40,60(
)0,54()0,60(
)18,0()0,0(
)ˆ,ˆ(),(
jiji
Solve the Equation
The Transformation Method
A. Create a new image that will be our new undistorted image.
B. For each pixel (i,j) in our new image
1. Use the transform function to calculate the corresponding position ( , ) in the distorted image
2. Get the color of the pixel at ( , )
3. Set the color of pixel (i,j) to this value
j ^ ^
i
j ^ ^
i
1820
7
10
3
400
3ˆ
8
3
10
9
800
3ˆ
jiijj
jiiji
Undistorted image
• (i,j)
• ( , ) j ^ ^
i
i
j
Distorted image
Correcting Distortion: Whiteboard example
Images courtesy of Zhang & He (2003)
Transformed
• (i,j)
• (i,j)
• ( , ) j ^ ^
i
2012-09-10 E0005E, Lecture 2 23
Geometric Transformations
• The recorded image is distorted (warped) due to some non-linear optic transfer function.
• Problem: Transform the distorted image to the right geometric shape
Distorted geometry Correct geometry
2012-09-10 E0005E, Lecture 2 24
Geometric Transformations
Barrel-distorted image A set of corresponding point pairs,
‘anchor points’
2012-09-10 E0005E, Lecture 2 25
Geometric Transformations
Points should be uniformly
distributed around the perimter of
the image Coordinates in the
correct image
Coordinates in the
distorted image
2012-09-10 E0005E, Lecture 2 26
Geometric Transformations Notes If the transformation is bi-linear (linear along rows and columns) such as in the earlier example, then m=1 and we require only 4 points (perhaps 3) Non-linear requires higher values of m (m=2 in this example) If the transform does not change rapidly based on position, then low order values of m, 2 or 3 can be used requiring 6 or 10 points.
2012-09-10 E0005E, Lecture 2 27
2012-09-10 E0005E, Lecture 2 28
2012-09-10 E0005E, Lecture 2 29
Geometric Transformations
• As part of development phase 2 you will work with an example with only 4 ‘anchor-points’
• So your transform equations will have the simpler linear form
2012-09-10 E0005E, Lecture 2 30
Geometric Transformations
A Need in Iron Ore Mining
• In mining the mountain is blasted to break it up into fragmented rocks
• An excavator digs out the fragmented rocks and dumps them in a shute that eventually leads to the processing plant for extraction of the iron ore
• The driver of the excavator would like to know if the bucket contains iron ore, or if it contains mostly waste rock
• The driver knows the approximate weight of the rock in the bucket
• If the volume of rock was known, then the rock density could be estimated
• High density would indicate iron ore, low density indicates waste rock
• Waste rock should not be put into the shute as it creates a lot of subsequent processing for no value
• Lots of waste rock would indicate it is time to stop excavating
• How to calculate volume? Image courtesy of LKAB, Underground excavator used at LKAB Kiruna
What is a height image
• A height image is another kind of digital image where the pixel color doesn’t represent the color of the objects in the scene, but the height of the objects.
• Below are two height maps of an excavator bucket viewed from the ceiling of the tunnel as the excavator drove underneath.
• The left image is a bucket containing rocks, the right image is an empty bucket. (White values are higher than gray values)
A solution for improved sustainability and productivity in mining
• As the images were collected at different times the excavator is not in the same position or orientation in the two images.
• Therefore we can apply our algorithm for correcting distortion in images
• Choose the empty bucket to be our undistorted image, the full bucket to be our distorted image
• Use the corner points in both images to build the transformation function and apply
Full bucket Empty bucket Full bucket transformed to match the empty bucket
A solution for improved sustainability and productivity in mining
• Subtract empty bucket from the transformed full bucket to get the volume image
• Calculate the bucket volume by adding all the height values in the volume image multiply by the known area of pixel
• Use the volume with the known weight of the bucket to estimate density and therefore rock type (iron ore or waste rock)
Empty bucket Volume image Full bucket transformed to match the empty bucket
POINT OPERATIONS AGAIN
E0005E Industrial Image Analysis
Matthew Thurley
2012-09-10 E0005E, Lecture 2 35
2012-09-10 36
Point operations (lecture 1)
• A point operation is an operation where the gray level gi at a certain pixel (i,j) is replaced with a new gray level go according to some mapping go = F(gi)
(i,j)
gi = I(i,j)
go = F(gi)
I(i,j) = go
E0005E, Lecture 2
2012-09-10 37
Point operations (lecture 1)
• Point operations are implemented as a Look Up Table
- If the mapping F(g) is independent of pixel position
2 9 0 1
0 7 4 3
9 8 1 6
3 0 6 2
Original image New image
Look Up Table
0 2 4 8 16 16 16 16 32 64
0 1 2 3 4 5 6 7 8 9
E0005E, Lecture 2
NEIGHBOURHOOD OPERATIONS
E0005E Industrial Image Analysis
Matthew Thurley
2012-09-10 E0005E, Lecture 2 38
Liner versus Non-linear operations
• Some operations, like point-operations will be described as linear operations, and other operations will be described as non-linear.
• Consider a general operator H, that produces an output image Ig, given an input image If, then
Ig = H[If]
H is a linear operator if
H[ a1.If1 + a2.If2 ] = a1.H[If1] + a2.H[If2]
= a1.Ig1 + a2.Ig2
This simply says that if we sum two scaled images and then apply the operator, it is the same as applying the operator to the images individually first and then scaling and summing
For example; the summation operator is linear, the max and min operators are non-linear (consider if a1 = -1)
2012-09-10 E0005E, Lecture 2 39
DIP 2.6.2
2012-09-10 40
Neighbourhood operations
• A square neighbourhood N(i0,j0) to the pixel (i0, j0)
the group of pixels (i0 – k, j0 – l )
where –m ≤ k ≤ m and –n ≤ l ≤ n
are odd constants
• Example
m = n = 1
Size 3x3
E0005E, Lecture 2
2012-09-10 41
Neighbourhood operations
• The graylevel gi at a certain pixel (i0, j0) is replaced with a new graylevel go according to some mapping
- go = F(N(i, j))
E0005E, Lecture 2
Operation on the whole
neighbourhood N
2012-09-10 42
Linear sliding-window operations
• Neighbourhood operation - linear sum of the graylevels in a neighbourhood N
E0005E, Lecture 2
2012-09-10 43
Non-linear sliding-window operations
• Maximum in a neighbourhood N
• Minimum in a neighbourhood N
E0005E, Lecture 2
• You need to consider how to handle the image boundary.
+ Can use only the part of the mask that intersects the image. Matlab does this with imfilter if you require a resultant image that is the same size.
+ For edge filters: Could pad the resultant edge image with zeros
A Note about Neighbourbood Operations
2012-09-10 E0005E, Lecture 2 44
NOISE FILTERING
E0005E Industrial Image Analysis
Matthew Thurley
2012-09-10 E0005E, Lecture 2 45
Noise Filtering – Image Restoration
• We now consider two types of noise and two noise filters
- Guassian white noise
- Salt and pepper noise
Textbook: DIP 5.3 Restoration in the Presence of Noise Only - Spatial Filtering
2012-09-10 E0005E, Lecture 2 46
2012-09-10 47
Noise – Additive (guassian) white noise
• At every pixel in the original image I a random value ξ is added to the original grayvalue.
- ξ = N(0, σ2)
- The random variable is normally-distributed and the noise values for any two pixels are independent.
• A noise contaminated image D is given by
D(i, j) = I(i, j)+ξ(i, j)
E0005E, Lecture 2
2012-09-10 48
Noise – additive white noise
E0005E, Lecture 2
D(i, j) I(i, j)
2012-09-10 49
Noise – filtering white noise
E0005E, Lecture 2
• Averaging filter mask reduce additive white noise
2012-09-10 50
Noise – additive white noise
E0005E, Lecture 2
D(i, j) I(i, j) Filtered image
2012-09-10 51
Noise – salt and pepper noise
• At certain pixels in the original image I the original grayvalue has been replaced with either a very high or a very low value.
• An binary expression of salt and pepper noise
E0005E, Lecture 2
2012-09-10 52
Noise – salt and pepper noise
E0005E, Lecture 2
D(i, j) I(i, j)
2012-09-10 53
Noise – filtering salt and pepper noise
E0005E, Lecture 2
• Median filter mask reduce salt and pepper noise
Median value (middle value)
Sorted array of graylevels
2012-09-10 54
Noise – salt and pepper noise
E0005E, Lecture 2
D(i, j) I(i, j) Filtered image
Neighbourhood Operations – Matlab Example
• Sliding window operations, both linear and non-linear can be performed in many different ways in MATLAB.
• To implement a median filter we first create a filter function > funcM = ’median(x(:))’;
• We can use this in a useful general function where m and n control the neighbourhood size > ImM = nlfilter(Im,[2*m+1 2*n+1],funcM)
• To implement a linear white noise filter shown earlier we create a different filter function, and then use that > funcA = ’((1/15)*x(:)’*[1 2 1 2 3 2 1 2 1]’)’;
> ImA = nlfilter(Im,[2*m+1 2*n+1],funcA)
2012-09-10 E0005E, Lecture 2 55
2012-09-10 56
Edges (more next lecture)
• Edges are one of the most important features in an image since the edges are strong indicators of object boundaries both for humans and in segmentation algorithms.
• Edge preserving filters attempt to filter noise while maintaining important edge information
• Textbook: Sonka 4.3.1 pg 73 Averaging using a rotating mask (very similar)
• Textbook: DIP 5.3.3 contains Adaptive reduction filter that is also variance based but more complicated
E0005E, Lecture 2
2012-09-10 57
Edge Preserving Filters
E0005E, Lecture 2
2012-09-10 58
Edge Preserving Filters
E0005E, Lecture 2
Neighborhood with vertical orientation
Neighborhood with horizontal orientation
2012-09-10 59
Direction-of-Least-Variance
• The sample variance of a vector (line) L is defined as
E0005E, Lecture 2
Consider the 10 pixels along the red line, 2 black, 2 white ...
Intensity profile along the line L (the red line)
2012-09-10 60
Direction-of-Least-Variance
• Check the variance in M (in this case 4) directions in the neighbourhood
E0005E, Lecture 2
Mean(L)
Do the smoothing in the direction of least variance, in this case vertically, perhaps using a filter
0 1 0
0 1 0
0 1 0
2012-09-10 61
DVL – example of corner detail
E0005E, Lecture 2
References & Further Reading
• Zhengyou Zhang & Li-wei He, Whiteboard Scanning and Image Enhancement, Microsoft Research, Technical Report MSR-TR-2003-39, June 2003, http://research.microsoft.com/en-us/um/people/zhang/papers/tr03-39.pdf
• Keystone effect, Wikipedia, Jan 2010, http://en.wikipedia.org/wiki/Keystone_effect
• Gonzalez R. and Woods R., Geometric Spatial Transformations and Image Registration, pg. 87, Digital Image Processing, Third Edition, Pearson Education, ISBN-10: 0-13-505267-X
• Sonka M., Hlavac V., Boyle R., Geometric Transformations, section 4.2, Image Processing, Analysis, and Machine Vision, Second Edition, International Thomson Publishing Inc., ISBN: 0-534-95393
• Contact me: [email protected]
This lecture contains some examples of how to include references and cite sources
Summing Up
• Consider the following three questions;
- What do I need to work on?
- What have I learnt today?
- What was the main point left unanswered today?
• Write your answers in the provided journal. Write the lecture number 2 on top of the page. Write your name and student number on the front of the book
2012-09-10 E0005E Industrial Image Analysis 63