30
Image Processing - Lesson 3 Image Enhancement - Spatial Domain Smoothing filter Median filter Convolution 1D Discrete 1D Continuous 2D Discrete 2D Continuous Sharpening filter Image Enhancement

Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Embed Size (px)

Citation preview

Page 1: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Image Processing - Lesson 3

• Image Enhancement - Spatial Domain

• Smoothing filter

• Median filter

• Convolution

• 1D Discrete

• 1D Continuous

• 2D Discrete

• 2D Continuous

• Sharpening filter

Image Enhancement

Page 2: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Salt & Pepper Noise

Page 3: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Neighborhood Averaging

g(x,y) = (1/M) Σ f(n,m)

S = neighborhood of pixel (x,y)M = number of pixels in neighborhood S

Neighborhoods:

3 x 3

5 x 5

Sm)(n, ∈

Page 4: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Neighborhood Averaging - Example

Salt & PepperNoise

3 X 3 Average 5 X 5 Average

7 X 7 Average Median

Page 5: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Convolution

A, B = images B is typically smaller than A and is called the mask.

(A * B) (x) = Σ A(i)B(x-i)i

1 dimensional:

Example:

1 2 3

BA1 2 3 4 5

(A * B) (1)

1 2 3 4 5

3 2 1

x=0

1 10 4 5

* * *

+

x=0

x=1

3 4 3

(A * B) (2)

1 2 3 4 5

3 2 1

10 16 22

* * *

+

x=2

6 6 4

Page 6: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

• Option 1: Zero padding

• Option 2: Wrap around

• Option 3: Reflection

What happens near the edges?

0 0 0 1 2 3 4 5 0 0 0

4 10 16 22 22

3 4 5 1 2 3 4 5 1 2 3 4 5

19 10 16 22 23 5

3 2 1 1 2 3 4 5 5 4 3 2

7 10 16 22 27 5

1 2 3Convolution with

Page 7: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Why one image is reflected in the convolution:

0 3 2 1 00 1 0 ........ 1 2 3 ........* ........=

0 3 2 1 01 2 3 ........ 0 1 0 ........* ........=

With reflection:

Without reflection:

0 1 2 3 00 1 0 ........ 1 2 3 ........* ........=

0 3 2 1 01 2 3 ........ 0 1 0 ........* ........=

Reflection is needed so that convolution is commutative:

A * B = B * A

Page 8: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Convolution - Continuous Case

1/2

1

g(α)

1

1

f(α)

α

α

1/2

-1

g(-α)

α

1/2

g(x-α)

αx

1/2

-1

f(α)g(x-α)

αx

0≤x≤11/2

-1

f(x-α)g(α)

αx

1≤x≤2

1/2

x1 20

( )( ) ( ) ( )∫∞

∞−

−=∗ aaa dxgfxgf

Page 9: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

-1 1

0 1

Convolution - 2 Dimensions

A B*

(A * B) (x,y) = Σ Σ A(i,j) B(x-i,y-j)i j

10 5 20 20 20

10 5 20 20 20

10 5 20 20 20

10 5 20 20 20

10 5 20 20 20

*

-10 5 -15 0 0

-10 15 -10 20 20

-10 15 -10 20 20

-10 15 -10 20 20

-10 15 -10 20 20

=

Convolution - 2D Continuous Case:

(zero padding)

( ) ( ) ( ) dßdaßya,xgßa,fy)(x,gf −−=∗ ∫ ∫∞

∞−

∞−

Page 10: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Grayscale Convolution - Example

*

A B

A * B

Page 11: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Convolution Properties

• Complexity:– Assume A is n x n and B is k x k then

A*B takes O(n2k2) operations.• A*B = B*A• (A*B)*C = A*(B*C)

– If B and C are k x k then(A*B)*C takes O(2n2k2) operations.However A*(B*C) takes O(k4+n2k2) operations, which is faster if k<<n .

• Separability– In some cases it is possible to

decompose B (k x k) into B=C*D where C is 1 x k and D is k x 1. In such a case A*B takes O(n2k2)while (A*C)*D takes O(2n2k).

1 -1 *1

-1=

1 -1

-1 1

Page 12: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

• Image Average– In order to preserve the overall

average of A, the sum of B’s elements should equal 1

W1 W2 W3B

A

A*B

If W 1+W2+W3 =1 then Av(A)=Av(A*B)

( ) ==

=−−otherwise

yyandxxifyyxx

01

, 0000δ

0 0 00 1 00 0 0

A(x,y)*δ(x-x0,y-y0) = A(x-x0,y-y0)

),(d yx

Mask Constraints

Page 13: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Convolution Masks - Example: The Delta Kernel

A(x,y)*δ(x-x0,y-y0) = A(x-x0,y-y0)

A(x,y)*δ(x,y) = A(x,y)

0 0 00 1 00 0 0

0 0 10 0 00 0 0

δ(x,y) δ(x-1,y-1)

1 2 34 5 67 8 9

A

*0 0 10 0 00 0 0

δ(x-1,y-1)

=

6 4 59 7 83 1 2

0 4 50 7 80 0 0

A(x-1,y-1)

A(x-1,y-1)

(Zero padding)

(Wrap around)=

( ) ==

=−−otherwise

yyandxxifyyxx

01

, 0000δ

*

Page 14: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Grayscale Smoothing

Grayscale averaging = convolution with:

1/25 1/25 1/25 1/25 1/25

1/25 1/25 1/25 1/25 1/25

1/25 1/25 1/25 1/25 1/25

1/25 1/25 1/25 1/25 1/25

1/25 1/25 1/25 1/25 1/25

1/9 1/9 1/9

1/9 1/9 1/9

1/9 1/9 1/9

3 X 3

5 X 5

“Soft” Averaging: Convolution with a Gaussian

Discrete case:

0 1 0

1 2 1

0 1 0

(1/8) x

1 2 3 2 1

2 4 6 4 2

3 6 9 6 3

2 4 6 4 2

1 2 3 2 1

(1/81) x

A seperable kernel

2

22

2)y-(x

2 e2

1 σ

πσ

+

Page 15: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Normal vs Gaussian Grayscale Smoothing

OriginalNoisy image

3 X 3Average

3 X 3GaussianAverage

5 X 5GaussianAverage

7 X 7GaussianAverage

5 X 5Average

7 X 7Average

Page 16: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Median Filtering

S = neighborhood of pixel (x,y)

30 10 20

10 250 25

20 25 30

10, 10, 20, 20, 25, 25, 30, 30, 250

median

Median + Average: average the k central values.

10, 10, 20, 20, 25, 25, 30, 30, 250

24

medianSy)(x,

New value at (x,y) = {I(x,y)}

Page 17: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Salt & PepperNoise

3 X 3 Average 5 X 5 Average

7 X 7 Average Median

Median vs Average Filtering

Page 18: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Large Noise

Median Median x 2

Median x 7

Median x 4

Median x 6Median x 8

Multiple Median Filtering

Page 19: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Original Salt & Pepper Noise

Median Filter

Median Filtering - Failure

Page 20: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Original Salt & Pepper Noise

Median Filter

Oriented Median Filtering

Oriented Median Filter

Page 21: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Salt & Pepper noise

4x4 Average 7x2 Average

Oriented Filters

Page 22: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Oriented 6x2Average

Original Noisy Image

4x4 Average

Oriented Filtering - Example

Page 23: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Directional Smoothing

Define oriented masks:

1/n1/n

1/n1/n

1/n1/n

1/n1/n

θ

Choose neighborhood with smallest variance and replace pixel value with the average of that neighborhood.

Page 24: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Directional Smoothing - Example

Directional Smoothing(2x5, 5x2, diagonalx2)

Original + Noise 3x3 Average

Page 25: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Directional Smoothing - Example

Directional Smoothing(2x5, 5x2, diagonalx2)

Original + Noise 3x3 Average

Page 26: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Sharpening

• A sharpening filter is applied in order to enhance edges and fine details (high frequency) in an image:

– Assume A*G is a smoothing filtering.– A*(δ - G) contains the fine details of

the image.

– A+λA*(δ - G) = A*( (1+λ)δ-λG)=A*S(λ)amplifies fine details in the image.

– The parameter λ controls the amount of amplification.

G =

00

00

81

81

21

81

81

S(1) =

−−

00

00

81

81

23

81

81

Page 27: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

A

δ-G

A*S

Ringing effect in edge enhancement

Page 28: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

How can we enhance such an image?

Page 29: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Solution: Image Representation

2 1 3

5 8 7

0 3 5

= 1 0 0

0 0 0

0 0 0

2 + 1

0 1 0

0 0 0

0 0 0

0 0 1

0 0 0

0 0 0

+ 3 + 5

0 0 0

1 0 0

0 0 0

+

+ ...

= 3 + 5 +

+ 10 + 23 + ...

Page 30: Image Enhancement - University of Haifacs.haifa.ac.il/~dkeren/ip/lecture3.pdf · Image Processing - Lesson 3 • Image Enhancement - Spatial Domain • Smoothing filter • Median

Frequency Domain

Map of “Sizes and Orientations”

Evaluating an Image in terms of “sizes”:

Large Small

Thin & Horizontal

Vertical