CSCE 641 Computer Graphics: Image Sampling and Reconstruction Jinxiang Chai

CSCE 641 Computer Graphics:Image Sampling and Reconstruction

Jinxiang Chai

Review: 1D Fourier Transform

A function f(x) can be represented as a weighted combination of phase-shifted sine waves

How to compute F(u)?

dxexfuF uxi 2)()(

dueuFxf uxi 2)()( Inverse Fourier Transform

Fourier Transform

Review: Box Function

21 02

1 1)(

x

xxf

)(sinsin

)( ucu

uuF

x

u

f(x)

|F(u)|

If f(x) is bounded, F(u) is unbounded

Review: Cosine

-1.5

-1

-0.5

0

0.5

1

1.5

1-1

If f(x) is even, so is F(u)

)2cos( x

Review: Gaussian

2222 ue -0.02

0.03

0.08

0.13

0.18

-0.02

0.03

0.08

0.13

0.18

2

2

2

2

1

x

e

If f(x) is gaussian, F(u) is also guassian.

2222 ue

Review: Properties

Linearity:

Time shift:

Derivative:

Integration:

Convolution:

)()()()( ubGuaFxbgxaf

)()( 020 uFexxf uxi

)()(

uuFx

xdf

u

uFdxxf

)()(

)()()()( uGuFxgxf

Outline

2D Fourier Transform

Nyquist sampling theory

Antialiasing

Gaussian pyramid

Extension to 2D

Fourier Transform:

Inverse Fourier transform:

dvduevuFyxf vyuxi )(2),(),(

dydxeyxfvuF vyuxi )(2),(),(

Building Block for 2D Transform)(2 vyuxie

Building block:

Frequency:

Orientation:

22 vu

),tan( uvaOriented wave fields

Building Block for 2D Transform

Higher frequency

)(2 vyuxie Building block:

Frequency:

Orientation:

22 vu

),tan( uvaOriented wave fields

Some 2D Transforms

From Lehar

Some 2D Transforms

Why we have a DC component?

From Lehar

Some 2D Transforms

Why we have a DC component?

From Lehar

Some 2D Transforms

Why we have a DC component?

From Lehar

Some 2D Transforms

Why we have a DC component?

From Lehar

Some 2D Transforms

Why we have a DC component?

- the sum of all pixel values

From Lehar

dydxeyxfF yxi )00(2),()0,0(

Some 2D Transforms

Why we have a DC component?

- the sum of all pixel values

Oriented stripe in spatial domain = an oriented line in spatial domain

From Lehar

dydxeyxfF yxi )00(2),()0,0(

2D Fourier Transform

Why?- Any relationship between two slopes?

2D Fourier Transform

Linearity

Why?- Any relationship between two slopes?

2D Fourier Transform

Why is the spectrum bounded?

Linearity

Why?- Any relationship between two slopes?

Online Java Applet

http://www.brainflux.org/java/classes/FFT2DApplet.html

2D Fourier Transform Pairs

u vk k y

v

x

u

T

kv

T

ku ),(

x

yxdf ),( ),( vuuF

Gaussian Gaussian

x yk k

yyxx TkyTkx ),(

y

yxdf ),( ),( vuvF

2D Image FilteringInverse

transformFourier

transform

From Lehar

2D Image FilteringInverse

transformFourier

transform

Low-pass filter

From Lehar

2D Image FilteringInverse

transformFourier

transform

Low-pass filter

high-pass filter

From Lehar

2D Image FilteringInverse

transformFourier

transform

Low-pass filter

high-pass filter

band-pass filter

From Lehar

Aliasing

Why does this happen?

Aliasing

How to reduce it?

Sampling Analysis

f(x)

x

T 2T…-2T -T… 0

fs(x)

x

Sampling

Sampling Analysis

f(x)

x

T 2T…-2T -T… 0

fs(x)

x

Sampling

Reconstruction

Sampling Analysis

What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal?

f(x)

x

T 2T…-2T -T… 0

fs(x)

x

Sampling

Reconstruction

Sampling Theory

• How many samples are required to represent a given signal without loss of information?

• What signals can be reconstructed without loss for a given sampling rate?

Sampling Analysis: Spatial Domain

k

kTxxf )()(

f(x)

T 2T…-2T -T… 0

fs(x)

x

x T 2T…-2T -T… 0 x

k

kTx )(

X

?

Sampling Analysis: Spatial Domain

k

kTxxf )()(

f(x)

T 2T…-2T -T… 0

fs(x)

x

x T 2T…-2T -T… 0 x

k

kTx )(

X

? What happens in Frequency domain?

Fourier Transform of Dirac Comb

k

kTx )(

T

Review: Dirac Delta and its Transform

00

0)(

x

xx

x

1)( uF1

u

f(x)

|F(u)|

Fourier transform and inverse Fourier transform arequalitatively the same, so knowing one direction gives you the other

1)( dxx

)0()()( fdxxxf

Review: Fourier Transform Properties

Linearity:

Time shift:

Derivative:

Integration:

Convolution:

)()()()( ubGuaFxbgxaf

)()( 020 uFexxf uxi

)()(

uuFx

xdf

u

uFdxxf

)()(

)()()()( uGuFxgxf

Fourier Transform of Dirac Comb

k

kTx )(

T

k

ukTie 2

Fourier Transform of Dirac Comb

kk

ukTi

T

kue )(2

Fourier Transform of Dirac Comb

k

kTx )( )(T

ku

k

T 1/T

k

ukTie 2

Moving the spikes closer together in the spatial domain moves them farther apart in the frequency domain!

Sampling Analysis: Spatial Domain

k

kTxxf )()(

f(x)

T 2T…-2T -T… 0

fs(x)

x

x T 2T…-2T -T… 0 x

k

kTx )(

X

? What happens in Frequency domain?

Sampling Analysis: Freq. Domain

1/T …-1/T… 0 u

n T

nu )(F(u)

u-fmax fmax

Sampling Analysis: Freq. Domain

1/T …-1/T… 0 u

n T

nu )(F(u)

u-fmax fmax

How does the convolution result look like?

Sampling Analysis: Freq. Domain

1/T …-1/T… 0 u

)(uF(u)

u-fmax fmax

Sampling Analysis: Freq. Domain

)()( uuF

1/T …-1/T… 0 u

)(uF(u)

u-fmax fmax

Sampling Analysis: Freq. Domain

)()()( uGuuF

1/T …-1/T… 0 u

)(uF(u)

u-fmax fmax

G(0)?

G(fmax)?

G(u)?

Sampling Analysis: Freq. Domain

)()()( uGuuF

1/T …-1/T… 0 u

)(uF(u)

u-fmax fmax

G(0) = F(0)

G(fmax) = F(fmax)

G(u) = F(u)

Sampling Analysis: Freq. Domain

)()( uuF

)(u

1/T …-1/T… 0 u

F(u)

u-fmax fmax

Fs(u)

u-fmax fmax-1/T 1/T

)

1(

Tu How about

Sampling Analysis: Freq. Domain

)()( uuF

)(u

1/T …-1/T… 0 u

F(u)

u-fmax fmax

Fs(u)

u-fmax fmax-1/T 1/T

)

1(

Tu How about

Sampling Analysis: Freq. Domain

n T

nuuF )()(

1/T …-1/T… 0 u

n T

nu )(F(u)

u-fmax fmax

Fs(u)

u-fmax fmax-1/T 1/T

Sampling Theory

• How many samples are required to represent a given signal without loss of information?

• What signals can be reconstructed without loss for a given sampling rate?

Sampling Analysis: Freq. Domain

n T

nuuF )()(

1/T …-1/T… 0 u

n T

nu )(F(u)

u-fmax fmax

Fs(u)

u-fmax fmax-1/T 1/T

How can we reconstruct the original signal?

Reconstruction in Freq. Domain

u-fmax fmax

Fs(u)

u-fmax fmax

box(u)

u-fmax fmax

)()( uboxuFs

Signal Reconstruction in Freq. Domain

T 2T…-2T -T… 0

x

fs(x)

f(x)

x

Fs(u)

u-fmax fmax

F(u)

u-fmax fmax

)()( uboxuFs

Inverse Fourier transform

Fourier transform

Signal Reconstruction in Spatial Domain

)sin()( xxf s

T 2T…-2T -T… 0

x

x

fs(x) sinc(x)

Aliasing

Why does this happen?

Sampling Analysis

T …-T… 0

f(x) F(u)

fs(x)

x

ux -fmax fmax

Fs(u)

u-fmax fmax

When does aliasing happen?

Sampling Analysis

T …-T… 0

f(x) F(u)

fs(x)

x

ux -fmax fmax

Fs(u)

u-fmax fmax

max21

fT

When does aliasing happen?

Sampling Analysis

T …-T… 0

f(x) F(u)

fs(x)

x

ux -fmax fmax

Fs(u)

u-fmax fmax

max21

fT

When does aliasing happen?

Sampling Analysis

T …-T… 0

f(x) F(u)

fs(x)

x

ux -fmax fmax

Fs(u)

u-fmax fmax

max21

fT

When does aliasing happen?

Sampling Analysis

What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal?

f(x)

x

T 2T…-2T -T… 0

fs(x)

x

Sampling

Reconstruction

Sampling Analysis

What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal?

f(x)

x

T 2T…-2T -T… 0

fs(x)

x

Sampling

Reconstruction

Sampling Rate ≥ 2 * max frequency in the signal• this is known as the Nyquist Rate

Antialiasing

Antialiasing

Increase the sampling rate to above twice the highest frequency

u-fmax fmax

Antialiasing

Increase the sampling rate to above twice the highest frequency

Introduce an anti-aliasing filter to reduce fmax

u-fmax fmax

Fs(u)

u-fmax fmax

Antialiasing

Increase the sampling rate to above twice the highest frequency

Introduce an anti-aliasing filter to reduce fmax

u-fmax fmax

Fs(u)

u-fmax fmax

Sinc Filter

F(u)

u-fmax fmax

Is this a good filter?

Sinc Filter

F(u)

u-fmax fmax

x

Inverse Fourier transform

)(sinsin

)( xcx

xxf

Is this a good filter?

Sinc Filter

Is this a good filter?F(u)

u-fmax fmax

x

Multiplying with a box function in frequency domain

Convolution with a sinc function in spatial domain

Inverse Fourier transform

)(sinsin

)( xcx

xxf

Sinc Filter

Is this a good filter?F(u)

u-fmax fmax

x

Multiplying with a box function in frequency domain

Convolution with a sinc function in spatial domain

Good: removes all frequency components above a given bandwidth

Bad:

- an infinite support in spatial domain and hard to do in the spatial domain

- Fluctuation causes ripples

- Negative weights (not good for filtering pixles)

Inverse Fourier transform

)(sinsin

)( xcx

xxf

Good Pre-filtering filters

Finite support in frequency domain - cut-off frequency

Finite support in spatial domain - implementation in spatial domain

Positive weights

- good for filtering pixels

Other Filter Choices

Mean filter

Triangular filter

Gaussian filter

x

f(x)

u

|F(u)|sinc(u)

Sin2c(u)

2222 ue

-0.02

0.03

0.08

0.13

0.18

-0.02

0.03

0.08

0.13

0.18

2

2

2

2

1

x

e

2222 ue

Cannot have a filter with finite support in both spatial and frequency domain (box, mean, triangular)

- box filter: finite in freq. and infinite in spatial domain

- mean and triangular: infinite in freq. and finite in spatial domain

Gaussian filter provides a good tradeoff between two domains

Other Filter Choices

Mean filter

Triangular filter

Gaussian filter

x

f(x)

u

|F(u)|sinc(u)

Sin2c(u)

2222 ue

-0.02

0.03

0.08

0.13

0.18

-0.02

0.03

0.08

0.13

0.18

2

2

2

2

1

x

e

2222 ue

Cannot have a filter with finite support in both spatial and frequency domain (box, mean, triangular)

- box filter: finite in freq. and infinite in spatial domain

- mean and triangular: infinite in freq. and finite in spatial domain

Gaussian filter provides a good tradeoff between two domains

Gaussian FilterFs(u)

u-fmax fmax

Gaussian FilterFs(u)

u-fmax fmax-0.02

0.03

0.08

0.13

0.18

Inverse Fourier transform

Gaussian FilterFs(u)

u-fmax fmax

Multiplying with gaussian function in frequency domain

Convolution with a gaussian function in spatial domain

-0.02

0.03

0.08

0.13

0.18

Inverse Fourier transform

Gaussian FilterFs(u)

u-fmax fmax

Multiplying with gaussian function in frequency domain

Convolution with a gaussian function in spatial domain

- Since the Gaussian function decays rapidly, it is reasonable to truncate the filter window

- Its standard deviation controls the smoothness of filtered signal: large σ in spatial domain = lower cutoff frequency

- Easy to implement it in the spatial domain

-0.02

0.03

0.08

0.13

0.18

Inverse Fourier transform

Filtering in Spatial Domain

=

Filter functionInput imageFiltered image

Gaussian Filtering in Spatial Domain

Discrete approximation to Gaussian function with σ =1.0

Filtering in Spatial Domain

Discrete approximation to Gaussian function with σ =1.0

Filtering in Spatial Domain

Discrete approximation to Gaussian function with σ =1.0

Filtering in Spatial Domain

Discrete approximation to Gaussian function with σ =1.0

Filtering in Spatial Domain

Discrete approximation to Gaussian function with σ =1.0

Filtered_I45 = Xwindownpixels

Gaussian Pre-filtering

G 1/4

G 1/8

Gaussian 1/2

Solution: filter the image, then subsample– Filter size should double for each ½ size reduction.

Why?

Gaussian Pre-filtering

G 1/4 G 1/8Gaussian 1/2

Solution: filter the image, then subsample– Filter size should double for each ½ size reduction.

Why?

Without Prefiltering

1/4 (2x zoom) 1/8 (4x zoom)1/2

Known as a Gaussian Pyramid– MipMap (Williams, 1983)

Image Pyramids

MipMap

Mip (latin phase): many things in a small place

Used for texture mapping

Small-scale texture used for rendering distant objects

Gaussian Pyramid

filter mask

Repeat– Filter– Subsample

Until minimum resolution reached – can specify desired number of levels (e.g., 3-level pyramid)

The whole pyramid is only 4/3 the size of the original image!

Laplacian Pyramid

Laplacian Pyramid (subband images)- Created from Gaussian pyramid by subtraction

- Encode fine to course structure in a different level

Laplacian Pyramid

What happens in frequency domain?

Where Are They Used?

Texture mapping - Mipmap

Image processing - Image composition/blending - Image compression

Computer vision - Image matching - Motion tracking - Optical flow estimation

Summary

2D Fourier Transform

Nyquist sampling theory - Sampling Rate ≥ 2 * max frequency in the signal

Antialiasing - prefiltering

Gaussian pyramid - Mipmap

Next Lecture

Image Processing

- Filtering noise

- Bilateral filter

- Feature extraction