53
Image Processing Filtering: Noise suppresion Edge detection

Image Processing Filtering: Noise suppresion Edge detection

Embed Size (px)

Citation preview

Page 1: Image Processing Filtering: Noise suppresion Edge detection

Image Processing

Filtering: Noise suppresionEdge detection

Page 2: Image Processing Filtering: Noise suppresion Edge detection

0 0 0 0 0 0

0 0 0 0 0 0

0 0 200 0 0 0

0 0 0 0 0 0

0 0 0 100 0 0

0 0 0 0 0 0

Image filteringLinear filtering = Applying a local function to the image using

a sum of weighted neighboring pixels.

Such as blurring:

0 0 0 0 0

0 0.11 0.11 0.11 0

0 0.11 0.11 0.11 0

0 0.11 0.11 0.11 0

0 0 0 0 0

* =0 0 0 0 0 0

0 22 22 22 0 0

0 22 22 22 0 0

0 22 33 33 11 0

0 0 11 11 11 0

0 0 11 11 11 0

Input image Kernel Output image

Page 3: Image Processing Filtering: Noise suppresion Edge detection

Image filtering

0 0 0 0 0 0

0 0 0 0 0 0

0 0 200 0 0 0

0 0 0 0 0 0

0 0 0 100 0 0

0 0 0 0 0 0

0 0 0 0 0

0 0.11 0.11 0.11 0

0 0.11 0.11 0.11 0

0 0.11 0.11 0.11 0

0 0 0 0 0

* =0 0 0 0 0 0

0 22 22 22 0 0

0 22 22 22 0 0

0 22 33 33 11 0

0 0 11 11 11 0

0 0 11 11 11 0

Input image f Filter h Output image g

Mean filter

Page 4: Image Processing Filtering: Noise suppresion Edge detection

Image filtering

• Linear filters can have arbitrary weights.• Typically they sum to 0 or 1, but not always.• Weights may be positive or negative.• Many filters aren’t linear (median filter.)

0 0 0 0 0

0 0 0 0 0

0 0 0 1 0

0 0 0 0 0

0 0 0 0 0

What does this filter do?

Page 5: Image Processing Filtering: Noise suppresion Edge detection

p1,1 p1,2 p1,3 p1,4 p1,5 p1,6

p2,1 p2,2 p2,3 p2,4 p2,5 p2,6

p3,1 p3,2 p3,3 p3,4 p3,5 p3,6

p4,1 p4,2 p4,3 p4,4 p4,5 p4,6

p5,1 p5,2 p5,3 p5,4 p5,5 p5,6

p6,1 p6,2 p6,3 p6,4 p6,5 p6,6

Animation of ConvolutionAnimation of Convolution

m1,1m1,2m1,3

m2,1m2,2m2,3

m3,1m3,2m3,3

Mask

Image after convolution

Original Image

To accomplish convolution of the whole image, we just Slide the mask

C4,2 C4,3 C4,4

Page 6: Image Processing Filtering: Noise suppresion Edge detection

Gaussian filter

* =Input image f

Filter hOutput image g

Compute empirically

Page 7: Image Processing Filtering: Noise suppresion Edge detection

Gaussian vs. mean filters

What does real blur look like?

Page 8: Image Processing Filtering: Noise suppresion Edge detection

--Noise EliminationThe noise is eliminated but the operation causes loss of sharp edge definition.

In other words, the image becomes blurred

Convolution for Noise EliminationConvolution for Noise Elimination

Page 9: Image Processing Filtering: Noise suppresion Edge detection

There are many masks used in Noise Elimination

Median Mask is a typical one

23 65 64

120 187 90

47 209 72

J=1 2 3

I=1

2

3

Rank: 23, 47, 64, 65, 72, 90, 120, 187, 209

median

Masked Original Image

The principle of Median Mask is to mask some sub-image, use the median of the values of the sub-image as its value in new image

Convolution with a non-linear maskConvolution with a non-linear maskMedian filteringMedian filtering

Page 10: Image Processing Filtering: Noise suppresion Edge detection

MedianMedian FilteringFiltering

Page 11: Image Processing Filtering: Noise suppresion Edge detection

Back to Linear filters:First and second derivatives

* =

Page 12: Image Processing Filtering: Noise suppresion Edge detection

First and second derivatives

Original First Derivative x Second Derivative x, y

What are these good for?

Page 13: Image Processing Filtering: Noise suppresion Edge detection

Subtracting filters

Original Second Derivative Sharpened

Page 14: Image Processing Filtering: Noise suppresion Edge detection

for some

Combining filters

*

*

0 0 0 0 0

0 0 0 0 0

0 -1 0 1 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 -1 0 0

0 -1 4 -1 0

0 0 -1 0 0

0 0 0 0 0

=

=It’s also true:

Page 15: Image Processing Filtering: Noise suppresion Edge detection

Combining Gaussian filters

More blur than either individually (but less than )

* = ?

Page 16: Image Processing Filtering: Noise suppresion Edge detection

Separable filters

* =

Compute Gaussian in horizontal direction, followed by the vertical direction.

Not all filters are separable. Freeman and Adelson, 1991

Much faster!

Page 17: Image Processing Filtering: Noise suppresion Edge detection

Sums of rectangular regions

243 239 240 225 206 185 188 218 211 206 216 225

242 239 218 110 67 31 34 152 213 206 208 221

243 242 123 58 94 82 132 77 108 208 208 215

235 217 115 212 243 236 247 139 91 209 208 211

233 208 131 222 219 226 196 114 74 208 213 214

232 217 131 116 77 150 69 56 52 201 228 223

232 232 182 186 184 179 159 123 93 232 235 235

232 236 201 154 216 133 129 81 175 252 241 240

235 238 230 128 172 138 65 63 234 249 241 245

237 236 247 143 59 78 10 94 255 248 247 251

234 237 245 193 55 33 115 144 213 255 253 251

248 245 161 128 149 109 138 65 47 156 239 255

190 107 39 102 94 73 114 58 17 7 51 137

23 32 33 148 168 203 179 43 27 17 12 8

17 26 12 160 255 255 109 22 26 19 35 24

How do we compute the sum of the pixels in the red box?

After some pre-computation, this can be done in constant time for any box.

This “trick” is commonly used for computing Haar wavelets (a fundemental building block of many object recognition approaches.)

Page 18: Image Processing Filtering: Noise suppresion Edge detection

Sums of rectangular regions

The trick is to compute an “integral image.” Every pixel is the sum of its neighbors to the upper left.

Sequentially compute using:

Page 19: Image Processing Filtering: Noise suppresion Edge detection

Sums of rectangular regions

A B

C D

Solution is found using:

A + D – B - C

Page 20: Image Processing Filtering: Noise suppresion Edge detection

Spatially varying filters

Some filters vary spatially.

Useful for deblurring.

Durand, 02

Page 21: Image Processing Filtering: Noise suppresion Edge detection

*

*

*

input output

Same Gaussian kernel everywhere.Slides courtesy of Sylvian Paris

Constant blur

Page 22: Image Processing Filtering: Noise suppresion Edge detection

*

*

*

input output

The kernel shape depends on the image content.Slides courtesy of Sylvian Paris

Bilateral filter

Maintains edges when blurring!

Page 23: Image Processing Filtering: Noise suppresion Edge detection

Borders

What to do about image borders:

black fixed periodic reflected

Page 24: Image Processing Filtering: Noise suppresion Edge detection

Sampling

Larry Zitnick

Page 25: Image Processing Filtering: Noise suppresion Edge detection

Up-sampling

How do we compute the values of pixels at fractional positions?

Page 26: Image Processing Filtering: Noise suppresion Edge detection

Up-Sampling as a Convolution Application ExampleUp-Sampling as a Convolution Application Example

1 3 2

4 5 6

Zerointerlace

1 0 3 0 2 0

0 0 0 0 0 0

4 0 5 0 6 0

0 0 0 0 0 0

Convolve

1 1 1

1 1 1

1 1 1

The value of a pixel in the enlarged image is the average of the value of around pixels. The difference between insert 0 and original value of pixels is “smoothed” by convolution

Original Image

(1+0+3+0+0+0+4+0+5) ÷(1+1+1+1+1+1+1+1+1) = 13/9

Page 27: Image Processing Filtering: Noise suppresion Edge detection

Up-Sampling as a Convolution Application ExampleUp-Sampling as a Convolution Application Example

Page 28: Image Processing Filtering: Noise suppresion Edge detection

Up-sampling: General solutions

f (x,y) f (x+1,y)

f (x+1,y+1)f (x,y+1)

f (x+0.8,y+0.3) f (x + a, y + b) = (1 - a)(1 - b) f (x, y) + a(1 - b) f (x + 1, y) + (1 - a)b f (x,y + 1) + ab f (x + 1, y + 1)

Bilinear sampling:

Bicubic sampling fits a higher order function using a larger area of support.

How do we compute the values of pixels at fractional positions?

Page 29: Image Processing Filtering: Noise suppresion Edge detection

Up-sampling

Nearest neighbor Bilinear Bicubic

Page 30: Image Processing Filtering: Noise suppresion Edge detection

Down-sampling

If you do it incorrectly your images could look like this:

Check out Moire patterns on the web.

Page 31: Image Processing Filtering: Noise suppresion Edge detection

Down-sampling

• Aliasing can arise when you sample a continuous signal or image– occurs when your sampling rate is not high enough to capture the

amount of detail in your image– Can give you the wrong signal/image—an alias– formally, the image contains structure at different scales

• called “frequencies” in the Fourier domain

– the sampling rate must be high enough to capture the highest frequency in the image

Page 32: Image Processing Filtering: Noise suppresion Edge detection

Solution

Filter before sampling, i.e. blur the image first.

With blur Without blur

Page 33: Image Processing Filtering: Noise suppresion Edge detection

Edge Detection

Larry Zitnick

Page 34: Image Processing Filtering: Noise suppresion Edge detection

What is an edge?

Cole et al. Siggraph 2008, results of 107 humans.

Page 35: Image Processing Filtering: Noise suppresion Edge detection

Edges are caused by a variety of factors

depth discontinuity

surface color discontinuity

illumination discontinuity

surface normal discontinuity

Origin of edges

Page 36: Image Processing Filtering: Noise suppresion Edge detection

Illusory contours

Page 37: Image Processing Filtering: Noise suppresion Edge detection

• The gradient of an image:

• The gradient points in the direction of most rapid change in intensity

Image gradient

Page 38: Image Processing Filtering: Noise suppresion Edge detection

The gradient direction is given by:

How does this relate to the direction of the edge?

The edge strength is given by the gradient magnitude

Image gradient

How would you implement this as a filter?

Page 39: Image Processing Filtering: Noise suppresion Edge detection

Sobel operator

-1 0 1

-2 0 2

-1 0 1

-1 -2 -1

0 0 0

1 2 1

Magnitude:

Orientation:

In practice, it is common to use:

Page 40: Image Processing Filtering: Noise suppresion Edge detection

Sobel operator

Original OrientationMagnitude

Page 41: Image Processing Filtering: Noise suppresion Edge detection

Effects of noise• Consider a single row or column of the image

– Plotting intensity as a function of position gives a signal

Where is the edge?

Page 42: Image Processing Filtering: Noise suppresion Edge detection

Where is the edge?

Solution: smooth first

Look for peaks in

Page 43: Image Processing Filtering: Noise suppresion Edge detection

• Check if pixel is local maximum along gradient direction– requires checking interpolated pixels p and r

Non-maximum suppression

Page 44: Image Processing Filtering: Noise suppresion Edge detection

Effect of (Gaussian kernel spread/size)

Canny with Canny with original

The choice of depends on desired behavior

• large detects large scale edges• small detects fine features

Page 45: Image Processing Filtering: Noise suppresion Edge detection

An edge is not a line...

How can we detect lines ?

Page 46: Image Processing Filtering: Noise suppresion Edge detection

Finding lines in an image

• Option 1:– Search for the line at every possible

position/orientation– What is the cost of this operation?

• Option 2:– Use a voting scheme: Hough transform

Page 47: Image Processing Filtering: Noise suppresion Edge detection

• Connection between image (x,y) and Hough (m,b) spaces– A line in the image corresponds to a point in Hough

space– To go from image space to Hough space:

• given a set of points (x,y), find all (m,b) such that y = mx + b

x

y

m

b

m0

b0

image space Hough space

Finding lines in an image

Page 48: Image Processing Filtering: Noise suppresion Edge detection

• Connection between image (x,y) and Hough (m,b) spaces– A line in the image corresponds to a point in Hough space– To go from image space to Hough space:

• given a set of points (x,y), find all (m,b) such that y = mx + b– What does a point (x0, y0) in the image space map to?

x

y

m

b

image space Hough space

– A: the solutions of b = -x0m + y0

– this is a line in Hough space

x0

y0

Finding lines in an image

Page 49: Image Processing Filtering: Noise suppresion Edge detection

Hough transform algorithm• Typically use a different parameterization

– d is the perpendicular distance from the line to the origin

– is the angle – Why?

Page 50: Image Processing Filtering: Noise suppresion Edge detection

Hough transform algorithm• Basic Hough transform algorithm

1. Initialize H[d, ]=02. for each edge point I[x,y] in the image

for = 0 to 180

H[d, ] += 1

3. Find the value(s) of (d, ) where H[d, ] is maximum4. The detected line in the image is given by

• What’s the running time (measured in # votes)?

Page 51: Image Processing Filtering: Noise suppresion Edge detection

Hough transform algorithm

http://www.cs.utah.edu/~vpegorar/courses/cs7966/

Page 52: Image Processing Filtering: Noise suppresion Edge detection

Hough transform algorithm

http://www.cs.utah.edu/~vpegorar/courses/cs7966/

Page 53: Image Processing Filtering: Noise suppresion Edge detection

Extensions• Extension 1: Use the image gradient

1. same2. for each edge point I[x,y] in the image

compute unique (d, ) based on image gradient at (x,y) H[d, ] += 1

3. same4. same

• What’s the running time measured in votes?

• Extension 2– give more votes for stronger edges

• Extension 3– change the sampling of (d, ) to give more/less resolution

• Extension 4– The same procedure can be used with circles, squares, or any other

shape