Introduction and Overview · and image interpolation •Intensity transformations •Spatial filtering •Fourier transform and filtering in the frequency domain •Image restoration

Midterm Review

Image Processing

CSE 166

Lecture 10

Announcements

• Midterm exam is on May 4

CSE 166, Spring 2020 2

Topics covered

• Image acquisition, geometric transformations, and image interpolation

• Intensity transformations

• Spatial filtering

• Fourier transform and filtering in the frequency domain

• Image restoration

• Color image processing


Image acquisition


Geometric transformations


Geometric transformations


Euclideantransformation

Similaritytransformation

Affinetransformation

Composition and inversion of transformations


Composition of transformations Inverse of transformation

Firsttransformation

Secondtransformation

Thirdtransformation

Intensity transformations


Contrast stretchingfunction

Thresholdingfunction

Intensity transformations


Some basic transformation functions

Gamma transformation

CSE 166, Spring 2020 10


CSE 166, Spring 2020

γ < 1

Darkimage

11



γ > 1

Lightimage

12

Piecewise-linear transformations

• Contrast stretching

• Intensity-level slicing

• Bit-plane slicing

CSE 166, Spring 2020 13

Contrast stretching

CSE 166, Spring 2020 14

Intensity-level slicing

CSE 166, Spring 2020 15

Bit-plane slicing

CSE 166, Spring 2020 16

Histogram


Similar to probability density function (pdf)17

Histogram equalization

CSE 166, Spring 2020 18


CSE 166, Spring 2020 19


CSE 166, Spring 2020 20

Spatial filtering (2D)

CSE 166, Spring 2020 21

2D correlation

2D convolution

Correlation and convolution (2D)

CSE 166, Spring 2020 22

Smoothing kernels

CSE 166, Spring 2020 23

Average(box kernel)

Weighted average(Gaussian kernel)

Smoothing with box kernel

CSE 166, Spring 2020 24

3x3

11x11 21x21

Inputimage

Smoothing with Gaussian kernel

CSE 166, Spring 2020 25

Standard deviation σ Percent of total volume under surface

1 39.35

2 86.47

3 98.89

Volume under surface greater than 3σ is negligible

Smoothing with Gaussian kernel

CSE 166, Spring 2020 26

σ = 3.521x21

Input image σ = 743x43

Derivatives

CSE 166, Spring 2020 27

Sharpening filters

CSE 166, Spring 2020 28

Overview: Image processing in the frequency domain

CSE 166, Spring 2020 29

Image in spatial domain

f(x,y)

Image in spatial domain

g(x,y)

Fouriertransform

Image in frequency domain

F(u,v)

Inverse Fourier

transform

Image in frequency domain

G(u,v)

Frequency domain processing

Jean-Baptiste Joseph Fourier1768-1830

Review

• Complex numbers

• Euler’s formula

• Complex functions

CSE 166, Spring 2020 30

1D Fourier series


Sinesand

cosines

31

Periodicfunction

Weighted by magnitude

Shifted byphase

Period T

Sampling

CSE 166, Spring 2020 32

Sampling


Over-sampled

Critically-sampled

Under-sampled

1/ΔT

Fourier transform of function

Fourier transforms of sampled function

33

Fourier transform of sampled functionand extracting one period

CSE 166, Spring 2020 34

1D

2D

Over-sampled Under-sampled

Recovered Imperfect recoverydue to

interference

Aliasing

CSE 166, Spring 2020 35

Continuous Sampled

IdenticalDifferent

Sampled at same rate

Over-sampled

Under-sampledAlias: a

false identity

Aliasing


1D

2D

Aliasing

Original

36

Fourier transform of sampled functionand extracting one period

CSE 166, Spring 2020 37

1D

2D

Over-sampled Under-sampled

Recovered Imperfect recoverydue to

interference

2D discrete Fourier transform (DFT)

• (Forward) Fourier transform

• Inverse Fourier transform

CSE 166, Spring 2020 38

Centering the DFT


1D

2D

In MATLAB, use fftshift and ifftshift

39

Centering the DFT


Original

DFT(look at corners)

Shifted DFTLog of

shifted DFT

40

Contributions of magnitude and phaseto image formation


Phase

IDFT: Phase only

(zero magnitude)

IDFT: Magnitude

only (zero phase)

IDFT: Boy magnitude

and rectangle phase

IDFT: Rectangle magnitude

and boy phase 41

2D convolution theorem

• 2D discrete (circular) convolution

• 2D convolution theorem

CSE 166, Spring 2020 42

Filtering using convolution theorem


Filtering in spatial domain

using convolution

expectedresult

Filtering in frequencydomain

using productwithout

zero-padding

wraparounderror

43




using product

withzero-padding

no wraparounderror

Gaussian lowpass filter in

frequency domain

44

Fourier transform

Product

Inverse Fourier transform

Zero padding



Filtering in spatial

domain using

convolution


usingproduct

Identical results

DFT

45

Filtering in the frequency domain

• Ideal lowpass filter (LPF)

– Frequency domain

CSE 166, Spring 2020 46


• Ideal lowpass filter (LPF)

– Spatial domain

CSE 166, Spring 2020 47

H(u,v) h(x,y)


• Gaussian lowpass filter (LPF)

CSE 166, Spring 2020 48


• Butterworth lowpass filter (LPF)

CSE 166, Spring 2020 49

Gaussian LPF



Ideal LPF Butterworth LPF

50

Highpass filter (HPF)Frequency domain


Ideal HPF

Gaussian HPF

Butterworth HPF

51

Butterworth HPF

Highpass filter (HPF)Spatial domain


Ideal HPF Gaussian HPF

52



Ideal HPF Gaussian HPF Butterworth HPF

53

H (u)

u u

x x

H (u)

h (x)

1

16–– 3

h (x)

2 12 12 1

2 1 8 2 1

2 12 12 1

0 2 1 0

2 1 4 2 1

0 2 1 0

1 2 1

2

1

9–– 3

4 2

1 2 1

1 1 1

1 1 1

1 1 1



1D

Lowpass filter Sharpening filter54

Frequencydomain

Spatialdomain



2D

55

Lowpass filter Highpass filter Offset highpass filter

Bandreject filters

CSE 166, Spring 2020 56

ButterworthIdeal Gaussian

Model of image degradation

• Spatial domain

• Frequency domain

CSE 166, Spring 2020 57

Noiseimage

Originalimage

Degradedimage

Degradationfunction

Noiseimage

Originalimage

Degradedimage

Degradationfunction

Model of image degradation, then restoration

CSE 166, Spring 2020 58

Estimate of original imageOriginal image

Histograms of sample patches


Sample “flat” patches from images with noise

Identify closest probability density function (pdf) match:

59

Gaussian Rayleigh Uniform

Mean filters


Additive Gaussian

noise

Geometric mean

filtered

Arithmetic mean

filtered

60

X-ray image

Order-statistic filters


Additivesalt and peppernoise

1x median filtered

3x median filtered

2x median filtered

61

Comparing filters


Alpha-trimmed mean filtered

Median filtered

Arithmetricmean

filtered

Geometric mean filtered

62

Additive uniform +

salt and peppernoise

Adaptive filters


AdditiveGaussian

noise

Arithmetricmean filtered

Geometric mean

filtered

Adaptive noise reduction

filtered

63

Periodic noise


Conjugate impulses

64

Additive sinusoidal noise

DFT magnitude

Notch reject filters

CSE 166, Spring 2020 65

Notch reject filter

CSE 166, Spring 2020 66

Filter in frequency

domain

Estimate of original

image

Degraded image

DFT magnitudeConjugate

impulses

Conjugate impulses

Estimating the degradation function

• Methods

– Observation

– Experimentation

– Mathematical modeling

CSE 166, Spring 2020 67

Estimation of degradation function by experimentation

CSE 166, Spring 2020 68

Imaged (degraded) impulseImpulse of light

Estimation of degradation function by mathematical modeling


Atmospheric turbulence

model

69

Image restoration


Inversefiltering

Wienerfiltering

Degraded image

70

Motion blur and

additive noise

Constrained least squares filtering

Less noise

Much less noise

RGB color model


RGB coordinates

71

RGB color cube

HSI color model:Relationship to RGB color model


All colors with cyan

hue

RGB color cube rotated such that line joining black and white

(intensity axis) is vertical

72

HSI color model


RGB color cube rotated such that

observer is on intensity axis, beyond white looking towards

black

Shape does not matter, only angle from red73

RGB color cube

HSI intensity axis


HSI

RGB

CMYK

74

Color models

CMY

Pseudocolor image processing

• Intensity slicing

• Intensity to color transformations

CSE 166, Spring 2020 75

Intensity slicing

CSE 166, Spring 2020 76

Grayscale to 2 colors

Intensity slicing



77

Intensity slicing



Colorbar

78

Intensity to color transformations


Grayscale input image

RGBoutput image

79



X-ray grayscale input image

Misses explosive

RGB output images

Without explosive

With explosive

80



Multiple grayscale

input images

SingleRGB

output image

81


Near infrared (NIR)

Red (R)

NIR,G,B as RGB R,NIR,B as RGB82

Green (G) Blue (B)

Multiple satellite

grayscale input

images

Output RGB imagesCSE 166, Spring 2020

Full-color image processing

• Two approaches

– Process each channel independently

– Process color vector space

CSE 166, Spring 2020 83



Spatial filtering: process each channel independently

84



All RGB channels

HSI intensity channel only

Spatial filtering: image sharpening

85

Difference



Histogram equalization: do not process each channel independently

1. RGB to HSI2. Histogram

equalize HSI intensity channel

3. HSI to RGB86

4. HSI saturation channel adjustment

Next Lecture

• Basis vectors and matrix-based transforms

– After midterm exam

• Reading

– Sections 6.1 and 6.2: Wavelet and Other Image Transforms

CSE 166, Spring 2020 87

Documents

Introduction and Overview · and image interpolation •Intensity transformations •Spatial filtering •Fourier transform and filtering in the frequency domain •Image restoration