Download pdf - Spatial Transforms Spatial Transforms

1

ECE/OPTI 531 – Image Processing Lab for Remote Sensing Fall 2005

Spatial TransformsSpatial Transforms

Reading: Chapter 6Reading: Chapter 6

Fall 2005Spatial Transforms 2

Spatial Transforms

• Introduction• Convolution and Linear Filters• Spatial Filtering• Fourier Transforms• Scale-Space Transforms• Summary

2


Introduction• Spatial transforms provide a way to access

image information according to size, shape, etc.• Spatial transforms operate on different scales

– Local pixel neighborhood (convolution)– Global image (Fourier filters)– All scales (scale-space filters)

• Image Model for Spatial Filtering– Any digital image can be written as the sum of two

images,

– The Low-Pass (LP) image contains the large areavariations, which determines the global image contrast(“macro-contrast”)

– The High-Pass (HP) image contains the small areavariations, which determines the sharpness and localimage contrast (“micro-contrast”)


• LP and HP image components can be calculatedby either convolution or Fourier filters

image(x,y) - LP(x,y) = HP(x,y)

3×3 neighborhood

7×7 neighborhood

LP and HP Image Components

3


Spatial Transforms



Convolution Filters

• Local processing within a moving window

• Result of the calculation at each location is theoutput value at the center pixel of the movingwindow

• Processing within the window uses the originalpixels, not the previously-calculated values

output at: pixel 2, row2 pixel 3, row 2 pixel 2, row 3

4


Linear Filters

• Any type of calculation can be done within themoving window

• If the calculation satisfies the principal ofsuperposition (Chapter 3, Section 3.3.1), then thefilter is termed linear– “The output of the sum of two inputs is equal to the

sum of their respective outputs”• Discrete form of 2-D convolution

– Symbolically,


2-D Convolution

• Alternatively, we can write for a Wx×Wy window,

– which emphasizes that the convolution is a weightingof local pixels

1. flip the window in x and y2. shift the window3. multiply the window weights times the corresponding

image pixels4. add the weighted pixels and write to the output pixel5. repeat steps 2-3-4 (shift-multiply-add) until finished

2-D Convolution Procedure

5


Cascaded Linear Filters

• Sequential linear filters can be cascaded into asingle filter– Because convolution operator does not depend on the

order applied (cummutative and associative)– Example with two filters

– where wnet is the equivalent filter, which can be pre-computed efficiently because the kernels are generallymuch smaller than the image


1. generate column sum of weighted input pixels2. add column sums to obtain current output pixel

(leftmost output pixel in row only)3. move window one pixel along row4. subtract leftmost column sum and add new rightmost

column sum to obtain current output pixel5. repeat steps 3 and 4 until done

Box-Filter Algorithm (along rows)

Box-Filter Algorithm

• Very efficient for certain configurations ofwindow weights– For column-sums, the weights within each window row

must be constant (but do not have to be equal fromrow-to-row)

– Computational savings varies with window size, but atleast a factor of 2

6


Box-Filter Algorithm (cont.)

C1 C2 C3 C4C1 C2 C3

Graphical depiction of column calculation


Border Region• There is a problem with the moving

window when it runs out of pixelsnear the image border

• Several possible solutions:– repeat the nearest valid output

pixel– reflect the input pixels outside the

border and calculate theconvolution

– reduce the window size– set the border pixels to zero or

mean image DN– wrap the window around to the

opposite side of the image (circularconvolution, same as produced byFourier filtering)

• All solutions are a “trick” to getaround the problem; no realsolution

border region for 3x3 window

7


Spatial Transforms



Spatial Filters

• Low-Pass Filters (LPF)– Preserve slowly varying spatial details (signal mean)

and smooth sudden transitions (edges, noise)– Simple example: 3-pixel window with equal weights

(moving average)[1/3, 1/3, 1/3]

– Weights sum to one• High-Pass Filters (HPF)

– Remove the signal mean and preserve edges– Complimentary to the LPF

[-1/3, 2/3, -1/3] = [0, 1, 0] - [1/3, 1/3, 1/3]– Weights sum to zero

8


Example: 1-D LPF and HPF

0

20

40

60

80

100

120

140

0 10 20 30 40 50 60

input

1x3 LP

DN

index

-50

0

50

100

150

0 10 20 30 40 50 60

input

1x3 HP

DN

index

0

20

40

60

80

100

120

140

0 10 20 30 40 50 60

input

1x7 LP

DN

index

-50

0

50

100

150

0 10 20 30 40 50 60

input

1x7 HP

DN

index


2-D LPF and HPF Kernels

• Weight array for pixels within the movingwindow is known as the “kernel” of the spatialfilter

9


Filtered Image Histograms

• LP-filtered images– same mean– smaller standard deviation– similar histogram

• HP-filtered images– zero mean– much smaller standard

deviation– nearly symmetric

histogram0

200

400

600

800

1000

-100 -50 0 50 100 150 200

num

ber

of

pix

els

DN

0

200

400

600

800

1000

-100 -50 0 50 100 150 200

num

ber

of

pix

els

DN

0

500

1000

1500

2000

2500

3000

3500

-100 -50 0 50 100 150 200

num

ber

of

pix

els

DN

Original

LP-component HP-component


High-Boost Filters (HBF)

• Corresponds to a “boosting” of the high spatialfrequency components (details) of the image

• Kernel consists of edges extracted with a HPFand boosted by a gain K, then added back to theoriginal DN at the center of the window

10


Example: HBF Varying K


Directional Filters

• Emphasize (HP) or suppress(LP) spatial features in agiven direction

orientation of window

Directional enhancements in 3 directions

11


Gradient Filters

• Find image gradient (derivative) in any directionby combining the x and y gradients in a vectorcalculation

! gx

gyg

x

y

|g|


Example: Edge Detection

original Prewitt

Roberts Sobel

Roberts DNT = 60

DNT = 90 DNT = 120

Gradient magnitude of different filters Thresholding Roberts gradient to find edges

12


Statistical Filters

• Similar to convolution with moving window, butoutput is a statistical quantity, e.g.– local minimum or maximum– local standard deviation– local histogram mode (DN at peak)

• Morphological filters– Application to spatial segmentation and noise filtering– Operations are composed of two basic elements similar

to minimum and maximum filters, but defined for darkobjects in binary images

• dilation (minimum)• erosion (maximum)

– Two filters applied in series to binary image yield• opening = dilation[erosion(binary image)] Eq. 6–9• closing = erosion[dilation(binary image)] Eq. 6–10

– Can use non-rectangular windows to achieve templatematching (“structuring element”)


Example: Morphological Filters

original local minimum local maximum

thresholdedoriginal

dilation erosion

openingclosing

13


Median Filter

• Output at each pixel is the median of the DNswithin the window– insensitive to outliers– suppresses impulse (single pixel or line) noise

• 1-D example with 5-pixel moving window– input = 10, 12, 9, 11, 21, 12, 10, 10– output = .., 11, 12, 11, 11, ..

0

20

40

60

80

100

120

140

0 10 20 30 40 50 60

input

1X3 median

DN

index


Spatial Transforms


14


Fourier Transforms

• Old theory (18th century), with wide applicationto signal analysis

• Represent a function as a linear combination(superposition) of basis functions, namely sinesand cosines

• Fourier Synthesis– The two-component image model is a simple example

• Extend to many components, i.e. individual sines andcosines

• Classic example is synthesis of a 1-D square wave with asum of sines and cosines

– In our case, the functions are 2-D images, but thesynthesis is the same

• An image is a sum of 2-D sine and cosine components• The more components included in the sum, the better the

synthesis


-0.2

0

0.2

0.4

0.6

0.8

1

1.2

0 50 100 150 200

DC+fundamental

+third harmonic

+fifth harmonic

ampli

tude

position

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

0 50 100 150 200

fundamentalthird harmonicfifth harmonic

ampli

tude

position

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

0 50 100 150 200

amplitude

position

Square Wave Synthesis

Sine wave components

Partial component sum

Ideal square wave

15


2-D Image Synthesissynthesized

imageerrorimage

DC

+ 8 components

+ 2 components

+ 18 components

+ 32 components

+ 144 components

RMS error

127.5

74.2

43.1

29.1

16

0

(DN)synthesized

imageerror

image

DC

+ 84 components

+ 12 components

+ 220 components

+ 420 components

+ 2048 components

RMS error

8.96

6.45

4.54

3.07

1.67

0

DN


Discrete Fourier Transform

• 2-D discrete Fourier series (synthesis)– inverse Fourier transform

– complex exponential is the sum of a sine and cosine

• 2-D discrete Fourier transform (analysis)– forward Fourier transform

16


DFT (cont.)

• Eq. 6–14 and 6–16 are the Discrete FourierTransform (DFT) pair– f is in the spatial domain and F is in the spatial

frequency domain– The arrays in the DFT are assumed periodic in both

domains• Fig. 6–18 example “postage stamp” replication of arrays

Image Domain Spatial Frequency Domain


Frequency Domain

• Relations among frequency domain parameters– spatial frequency intervals (not normalized)

– spatial frequency intervals (normalized)

– spatial frequencies (normalized)

17


Fourier Components

• The Fourier transform produces a complex array,with real and imaginary components

• A complex number can also be written in termsof amplitude, Akl, and phase, φkl

• Both components are important– Amplitude determines contrast/brightness– Phase determines location


Fourier Components (cont.)

Re(Fkl)

Im(Fkl)

Akl

!kl

phase only (Akl = 1) amplitude only (!kl = 0)

inverse Fourier transform of onlyone component

18


Fourier Filters

• Filtering in the frequency domain is the productof two functions

– W is called the transfer function, or simply filter– W acts like a mask on the image frequency

components F, modifying their amplitudes and phases– for discrete case, applies only to circular convolution

• Writing in the amplitude/phase representation

– |W| is called the Modulation Transfer Function (MTF)and is the amplitude filter


Fourier Filter Data Flow

FT

FT

FT -1

filteredimage

originalimage

zero-paddedwindow

spatial domain Fourier domain spatial domain

imagespectrum

transfer

Fkl

Wkl

function

Gkl

filteredimage

spectrum

19


Transfer Functions

• An ideal LPF in the spatial frequency domain hasuniform MTF,

• It transmits all frequency components below thecutoff frequency kc unchanged, and removes allhigher frequency components


Example: MTFs

LPF HPF

HBF 3x3 Gaussian 3x3 Box Filter

MTFs for Truncated 3x3 Box Filters MTF Comparison for LPFs

Gaussian

20


Power Spectrum

• The square of the Fourier amplitude• Indicates the “strength” of each frequency

component• Certain image spatial patterns have distinctive

power spectra

high amplitude “cloud,” primarily at lowerfrequencies

nonlinear, aperiodic features

high amplitude “line” through DC,oriented orthogonal to the spatial patterns

linear, quasi-periodic features

high amplitude “spikes,” localized at thefrequencies of the patterns

periodic patterns

spatial frequency descriptionspatial description


Example: Power Spectra

desert

fields

streets

railroad

21


Spatial Transforms



Scale-Space Transforms

• Access an image at differentresolutions (scales)– Image pyramids– Define a “reduce” operation that

incorporates resolution reduction anddownsampling to smaller image

L0

L1

L2

L3

L5

L4

L0

L1

L2

L3

22


Reduction Filters

• The Burt-Adelson REDUCE function (convolutionweights) is separable

– where w in each direction is parametric function of a

– Gaussian-like if a=0.4,


Reduction Comparison

box pyramid Gaussian pyramid

Gaussian pyramid

produces smoother

(AKA “well-

behaved”) reduced-

resolution images

23


Expansion

• EXPAND the REDUCEdimage by interpolation

* w

down-sample up-sample

w

REDUCE EXPAND

level L-1

level L


Net Convolution Filter

level 2

level 3

level 1

29 x 29

13 x 13

5 x 5

24


*wH

level 0

level 1

*

*** *filter columns

wL

wL

wL wH

filter rows

wH

HxHy HxLy LxHyLxLy

down-sample rows

down-sample columns

level 2Wavelet Transforms

• Similar to resolutionpyramids

• Successiveconvolutions withLPFs and HPFs todecompose imageinto components atdifferent resolutionscales


Wavelet Components

* wL

wH

level 1

level 0

LxHy

down-sample

down-sample

Lx

• Calculation of one of the 4wavelet components ateach level

25


Spatial Transforms



Summary

• Convolution and Fourier transform filtering areequivalent (given the same border algorithm)

• Small neighborhood convolution filters canaccomplish noise removal (LPF, median filter),edge enhancement (HPF, HBF, gradient filter)

• Scale-space filters combine convolution filterswith resolution pyramid– Examples of scale-space filters include Gaussian (LPF),

Laplacian (HPF) and wavelets (LPF and HPF)