Chapter 5

Digital Image Processing Fundamentals

Learning Goals

• The human visual system

• Digitizing images

• Display of Images

Trading an eye for an ear

opticalaxiscornea

lensopticdisk

opticnerveretina

foveacentralis

iris

An eye is a Multi-mega pixel camera

• It has a lens (adjustable zoom)

• It has an automatic f-stop (iris 2-8 mm)

• It has a sensor plane (100 million pixels)

• The sensor has a transfer function senstive to mesopic range; 380 to about 700 nm

The eyes have a USB2 data rate!

• 250,000 neurons in the optic nerve

• variable voltage output on EACH nerve

• 17.5 million neural samples per second

• 12.8 bits per sample

• 224 Mbps, per eye (a 1/2 G bps system!).

• Compression using lateral inhibition between the retinal neurons

Response curves

• Eye has Gaussian response to light.

• Gives rise to biologically motivated image processing

Quantization of an Image

• Computers use digital cameras -> quantizationSNR ≤6b+ 4.8

SNR =10 log3×22 b( ) =20blog2 +10log3

Delta-function

⎩⎨⎧ =

=⋅ else 0

0 )0()()(

nxnnx δ

Sampling an Image

f s

f (x) Anti-aliasingFilter

Sampling=convolution w pulse train

F(u) * P(u) ≡ F(γ)P(u−γ)dγ−∞

∞

∫

F(u) * P(u) = f(x) δ(x−n / fs)n=−∞

∞

∑ e−j 2πux ⎡ ⎣ ⎢

⎤ ⎦ ⎥dx

−∞

∞

∫

Quantization Error is visible

Displays

• Color Monitors are made for people

Chapter 6-7

• Opening and Saving Images

Chapter 8 Convolutionwith a kernel, g(x)

h(x) = f(x) * g(x)= f(u)g(x−u)du−∞

∞

∫

h(x) = f(x) * g(x)= f (u)g(−∞

∞

∑ x−u)

Convolution

1D and 2D signal processing

Consider the delta function

⎩⎨⎧ =

=⋅ else 0

0 )0()()(

nxnnx δ

δ(t)− ε

ε

∫ dt = 1

Time-shift delta

)( knk −=δδ

kk kxx δδ ⋅=⋅ )(δ( t − td )

Sample the input (it’s a convolution!)

][][][])[*( nxkxknnxk∑∞

−∞=

=−= δδ

s(t) = δ(t−n / fs)n=−∞

∞

∑

vs (t) =v(t)s(t) =v(t) δ(t−n/ fs)n=−∞

∞

∑

What does sampling do to spectrum?

What is the spectrum? v(t) =a0 + (a1 cost+b1sint) + (a2 cos2t+b2sin2t)+K

Fourier Coefficients

a0 ,a1,b1, a2 ,b2K

v(t) =a0 + (a1 cost+b1sint) + (a2 cos2t+b2sin2t)+K

CTFT

V( f ) =F [v(t)] = v(t)e−2 πiftdt−∞

∞

∫

v(t) =F−1 V( f )[ ] = V( f)e2πiftdt−∞

∞

∫e iθ =cosθ + i sinθ

Euler’s identity

e iθ =cosθ + i sinθ

Sine-cos Rep

x(t) = an cos(2πnf0t) + bn sin(2πnf0t)n=1

∞

∑n=0

∞

∑

v(t) =a0 + (a1 cost+b1sint) + (a2 cos2t+b2sin2t)+K

Harmonic Analysisa0 =1

Tx(t)dt

0

T

∫

an =2T

x(t)cos(2πnf0t)dt0

T

∫

bn =2T

x(t)sin(2πnf0t)dt0

T

∫

Convolution=time-shift&multi

V * W( f ) ≡ V(λ )W( f −λ)dλ−∞

∞

∫

Convolution ThmV * W( f ) =F(v(t)w(t))

multiplication in the time domain =convolution in the frequency domain

Sample

vs (t) =v(t)s(t) =v(t) δ(t−n/ fs)n=−∞

∞

∑

Vs ( f ) =V(F) * fsδ( f −nfs )n=−∞

∞

∑

Vs ( f ) = fs V( f −nfs)n=−∞

∞

∑

Spectrum reproduced

Vs ( f ) = fs V( f −nfs)n=−∞

∞

∑

spectrum to be reproduced at intervalsf s

Summary

h(x) = f(x) * g(x)= f(u)g(x−u)du−∞

∞

∫

h(x) = f(x) * g(x)= f (u)g(−∞

∞

∑ x−u)

Example of 1D convoln

2D Convolution

h(x, y) = f * g= f (u,v)g(x−u,y−v)dudv−∞

∞

∫−∞

∞

∫

h(x,y) = f * g= f(u,v)g([x−u],[y−v])v=0

vmax−1

∑u=0

umax−1

∑

Region of Support

• The region of support is defined as that area of the .kernel which is non-zero

• linear convolution:=signal has infinite extent but kernel has finite support

• If function has finite region of support we have compact support

Real images have finite region of support

• But we treat them as periodic and infinite!

• We repeat kernels so that they have the same period as the images.

• We call this cyclic convolution.

Convolution in 2D

h(x, y) = f * g= f (u,v)g(x−u,y−v)dudv−∞

∞

∫−∞

∞

∫

h(x,y) = f * g= f(u,v)g([x−u],[y−v])v=0

vmax−1

∑u=0

umax−1

∑

[x −u] =(x−u)modumax

[y−v] =(y−v)modvmax

Avoid the Mod op

h(x,y) = f * g= f (x−u,y−v)g(u+uc ,v+vc)v=−vc

vc

∑u=−uc

uc

∑

What is wrong with avoiding the mod op?

• How do I find the center of the kernel?

Cyclic Convolution

Implementing Convolution for(int y = 0; y < height; y++) { for(int x = 0; x < width; x++) { sum = 0.0; for(int v = -vc; v <= vc; v++) for(int u = -uc; u <= uc; u++) sum += f[cx(x-u) ][cy(y-v)] * k[ u+uc][v+vc]; if (sum < 0) sum = 0; if (sum > 255) sum = 255; h[x][y] = (short)sum; } }

What happens to the image if you ignore the wrap?

Cyclic Convolution keeps the edges

Can you think of a better way to implement convolution?

• Keep the edges!

• Don’t use the mod operation.

• How about growing the image by the size of the kernel*2?

Convolution is slow, how can I speed it up?

• JAI!

• FFT!?

• Other ideas?

Chapter 9

• Spatial Filters

• Blurring

• Median Filtering

• Hi-pass filtering

• SpatialFilterFrame and JAI

Blurring=Low-pass Filters

Why Blur an Image?

• Remove Noise

• Other ideas?

Average=low-pass

public void average() { float k[][] = { {1, 1, 1}, {1, 1, 1}, {1, 1, 1} }; Mat.scale(k,1/9.0); convolve(k); }

Want Unity Gain

G = kijj=0

H−1

∑i=0

W−1

∑

Many variations on LP filters

public void lp3() { float k[][] = { {1, 1, 1}, {1, 12, 1}, {1, 1, 1} }; Mat.scale(k,1/20.0); convolve(k); }

Gaussian Blur

gaussian(x,y, xc, yc,σ ) =1

2πσ 2 e−

x−xc( )2− y−yc( )

2( )

2σ 2

gmax =1

2πσ 2

Why use Gaussian Blur?

• How the eye works

• Symmetric

• Differentiable

• Smooth

Classic bell curve

Larger Kernels=more blur

Median Filtering

• Middle of a list of samples listed in ascending order.

• Sort samples, return n/2

Why use median filtering?

• Discard outliers

• 0, 85, 90, 87 and 100. The mean is 72

• Median is 87.

• {0, 85, 87, 90, and 100}=87

How do I implement the median?

public static int median(int a[]) { quickSort(a); int mid = a.length/2-1; if ((a.length & 1) == 1) return a[mid]; return (int )((a[mid]+ a[mid+1]+0.5)/2); }

Salt and Pepperpublic void saltAndPepper(int n) { for (int i=0;i < n; i++) { int rx = rand(0,width-1); int ry = rand(0,height-1); r[rx][ry] = 255; g[rx][ry] = 255; b[rx][ry] = 255; rx = rand(0,width-1); ry = rand(0,height-1); r[rx][ry] = 0; g[rx][ry] = 0; b[rx][ry] = 0; } short2Image(); }

Median 3x3

public void medianCross3x3() { short k[][] = {

{0, 1, 0},{1, 1, 1},{0, 1, 0}};median(k);

}

Median on any kernel

public void median (short k[] []) { printMedian(k,"color median"); Timer t = new Timer(); t.start();

r = median(r,k);g = median(g,k);b = median(b,k);t.print("Median filter time");

short2Image();}

Median result

Median Octagon, less aggressive

public void medianOctagon5x5() {short k[][] = {{ 0, 1, 1, 1, 0},{ 1, 1, 1, 1, 1},{ 1, 1, 1, 1, 1},{ 1, 1, 1, 1, 1},{ 0, 1, 1, 1, 0}};median(k);

}

Median Octagon results

Median Filtering is not FREE!

• Image degradation

• Selective median filtering.

• How do you know when to apply it?

Use Sample Variance

σ 2 =1

n(ai − a )2

i= 0

n-1

∑

a =mean

ai = ith sample

σ 2 = sample variance and

n= number of samples

Coefficient of variationˆ C = σ / a

public static boolean outlierHere(int a[]) {return ( coefficientOfVariation(a) > .4);

}

Before and After 3x3 Median Filtering with Outlier Detection

High-pass Filters

Defining HP Kernels

public void hp1() { float k[][] = {

{ 0, -1, 0},{-1, 10, -1},{ 0, -1, 0}};Mat.normalize(k);

convolve(k); }

Lower Center Number, GREATER edge enhancement!

510

Project ideasAdd an interface that allows the user to outline a rectangular region of interest. Then perform processing only in that region.

10. Convolution-based Edge Detection

Why do edge detection?What is an edge?What is a good edge?How do you find an edge?

Laplacian

∇2 f (x, y) =∂ 2 f

∂x2 +∂ 2 f

∂y2

Why Laplacian?

because it is zero when the rate of intensity change in the image plane is zero or linear.

Log filters=prefilter+laplacian

−∇2 1

2πσ 2 ex 2 + y 2

2σ2

=1

πσ 4 1 −x2 + y2

2σ 2

⎡

⎣ ⎢ ⎤

⎦ ⎥e−

x 2 + y 2

2σ2

∇2 f (x, y) =∂ 2 f

∂x2 +∂ 2 f

∂y2

Mexican Hat (LoG Kernel)

The Log kernel

Oriented Filters are steerable

Changing Scale at 0 Degrees

Changing Phase at 0 degrees

Interactive Heuristic Edge Detection

Douglas A. Lyon, Ph.D.Chair, Computer Engineering Dept. Fairfield

University, CT, USA

Lyon@DocJava.com, http://www.DocJava.com

CGIM, 2002

Copyright 2002 © DocJava, Inc.

Background

• It is easy to find a bad edge!• We know a good edge when we see it!

The Problem

• Given an expert + an image.

• The expert places markers on a good edge.

• Find a way to connect the markers.

Motivation

• Experts find/define good edges

• Knowledge is hard to encode.

Approach

• Mark an important edge

• Pixels=graph nodes

• Search in pixel space using a heuristic

• A* is faster than DP

Assumptions

• User is a domain expert

• Knowledge rep=heuristics+markers

Applications

• Photo interpretation

• Path planning (source+destination)

• Medical imaging

Photo Interpretation

Echocardiogram

•3D-multi-scale analysis

Path Plans, the idea

Path Planning-pre proc.•hist+thresh

•Dil+Skel

Path Planning - Search

The Idea

• The mind selects from filter banks

• The mind tunes the filters

Gabor filter w/threshold

• The Strong edge is the wrong edge!

Sub bands for 3x3 Robinson

Sub Bands 7x7 Gabor

Gabor-biologically motivated

Summary

• Heuristics+markers= knowledge• Low-level image processing still needed• Global optimization is not appropriate for

all applications• Sometimes we only want a single, good

edge

http://www.DocJava.com