CSCE 641 Computer Graphics:Image Sampling and Reconstruction
Jinxiang Chai
Review: 1D Fourier Transform
A function f(x) can be represented as a weighted combination of phase-shifted sine waves
How to compute F(u)?
dxexfuF uxi 2)()(
dueuFxf uxi 2)()( Inverse Fourier Transform
Fourier Transform
Review: Box Function
21 02
1 1)(
x
xxf
)(sinsin
)( ucu
uuF
x
u
f(x)
|F(u)|
If f(x) is bounded, F(u) is unbounded
Review: Cosine
-1.5
-1
-0.5
0
0.5
1
1.5
1-1
If f(x) is even, so is F(u)
)2cos( x
Review: Gaussian
2222 ue -0.02
0.03
0.08
0.13
0.18
-0.02
0.03
0.08
0.13
0.18
2
2
2
2
1
x
e
If f(x) is gaussian, F(u) is also guassian.
2222 ue
Review: Properties
Linearity:
Time shift:
Derivative:
Integration:
Convolution:
)()()()( ubGuaFxbgxaf
)()( 020 uFexxf uxi
)()(
uuFx
xdf
u
uFdxxf
)()(
)()()()( uGuFxgxf
Review: Properties
Linearity:
Time shift:
Derivative:
Integration:
Convolution:
)()()()( ubGuaFxbgxaf
)()( 020 uFexxf uxi
)()(
uuFx
xdf
u
uFdxxf
)()(
)()()()( uGuFxgxf
Review: Properties
Linearity:
Time shift:
Derivative:
Integration:
Convolution:
)()()()( ubGuaFxbgxaf
)()( 020 uFexxf uxi
)()(
uuFx
xdf
u
uFdxxf
)()(
)()()()( uGuFxgxf
)()()()( uGuFxgxf
Outline
2D Fourier Transform
Nyquist sampling theory
Antialiasing
Gaussian pyramid
Extension to 2D
Fourier Transform:
Inverse Fourier transform:
dvduevuFyxf vyuxi )(2),(),(
dydxeyxfvuF vyuxi )(2),(),(
Building Block for 2D Transform)(2 vyuxie
Building block:
Frequency:
Orientation:
22 vu
),tan( uvaOriented wave fields
Building Block for 2D Transform
Higher frequency
)(2 vyuxie Building block:
Frequency:
Orientation:
22 vu
),tan( uvaOriented wave fields
A function f(x,y) can be represented as a weighted combination of phase-shifted oriented wave fields.
Some 2D Transforms
From Lehar
Some 2D Transforms
From Lehar
Some 2D Transforms
From Lehar
Some 2D Transforms
From Lehar
Some 2D Transforms
From Lehar
Some 2D Transforms
From Lehar
Why we have a DC component?
Some 2D Transforms
Why we have a DC component?
- the sum of all pixel values
From Lehar
dydxeyxfF yxi )00(2),()0,0(
Some 2D Transforms
Why we have a DC component?
- the sum of all pixel values
Oriented stripe in spatial domain = an oriented line in spatial domain
From Lehar
dydxeyxfF yxi )00(2),()0,0(
2D Fourier Transform
Why?- Any relationship between two slopes?
2D Fourier Transform
Linearity
Why?- Any relationship between two slopes?
2D Fourier Transform
The spectrum is bounded by two slopes.
Linearity
Why?- Any relationship between two slopes?
Online Java Applet
http://www.brainflux.org/java/classes/FFT2DApplet.html
2D Fourier Transform Pairs
u vk k y
v
x
u
T
kv
T
ku ),(
x
yxdf ),( ),( vuuF
Gaussian Gaussian
x yk k
yyxx TkyTkx ),(
y
yxdf ),( ),( vuvF
2D Image FilteringInverse
transformFourier
transform
From Lehar
2D Image FilteringInverse
transformFourier
transform
Low-pass filter
From Lehar
2D Image FilteringInverse
transformFourier
transform
Low-pass filter
high-pass filter
From Lehar
2D Image FilteringInverse
transformFourier
transform
Low-pass filter
high-pass filter
band-pass filter
From Lehar
Aliasing
Why does this happen?
Aliasing
How to reduce it?
Sampling Analysis
f(x)
x
T 2T…-2T -T… 0
fs(x)
x
Sampling
Sampling Analysis
f(x)
x
T 2T…-2T -T… 0
fs(x)
x
Sampling
Reconstruction
Sampling Analysis
What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal?
f(x)
x
T 2T…-2T -T… 0
fs(x)
x
Sampling
Reconstruction
Sampling Theory
• How many samples are required to represent a given signal without loss of information?
• What signals can be reconstructed without loss for a given sampling rate?
Sampling Analysis: Spatial Domain
k
kTxxf )()(
f(x)
T 2T…-2T -T… 0
fs(x)
x
x T 2T…-2T -T… 0 x
k
kTx )(
X
?
continuous signal
discrete signal (samples)
comb function
Sampling Analysis: Spatial Domain
k
kTxxf )()(
f(x)
T 2T…-2T -T… 0
fs(x)
x
x T 2T…-2T -T… 0 x
k
kTx )(
X
? What happens in Frequency domain?
continuous signal
discrete signal (samples)
comb function
Sampling Analysis: Spatial Domain
k
kTxxf )()(
f(x)
T 2T…-2T -T… 0
fs(x)
x
x T 2T…-2T -T… 0 x
k
kTx )(
X
? What happens in Frequency domain?
continuous signal
discrete signal (samples)
comb function
Fourier Transform of Dirac Comb
k
kTx )(
T
Review: Dirac Delta and its Transform
00
0)(
x
xx
x
1)( uF1
u
f(x)
|F(u)|
Fourier transform and inverse Fourier transform arequalitatively the same, so knowing one direction gives you the other
1)( dxx
)0()()( fdxxxf
Review: Fourier Transform Properties
Linearity:
Time shift:
Derivative:
Integration:
Convolution:
)()()()( ubGuaFxbgxaf
)()( 020 uFexxf uxi
)()(
uuFx
xdf
u
uFdxxf
)()(
)()()()( uGuFxgxf
Fourier Transform of Dirac Comb
k
kTx )(
T
k
ukTie 2
Fourier Transform of Dirac Comb
kk
ukTi
T
kue )(2
Fourier Transform of Dirac Comb
k
kTx )( )(T
ku
k
T 1/T
k
ukTie 2
- Fourier transform of a Dirac comb is a Dirac comb as well.- Moving the spikes closer together in the spatial domain moves them farther apart in the frequency domain!
Sampling Analysis: Spatial Domain
k
kTxxf )()(
f(x)
T 2T…-2T -T… 0
fs(x)
x
x T 2T…-2T -T… 0 x
k
kTx )(
X
?What happens in Frequency
domain?
continuous signal
discrete signal (samples)
comb function
Review: Properties
Linearity:
Time shift:
Derivative:
Integration:
Convolution:
)()()()( ubGuaFxbgxaf
)()( 020 uFexxf uxi
)()(
uuFx
xdf
u
uFdxxf
)()(
)()()()( uGuFxgxf
)()()()( uGuFxgxf
Sampling Analysis: Freq. Domain
1/T …-1/T… 0 u
n T
nu )(F(u)
u-fmax fmax
Sampling Analysis: Freq. Domain
1/T …-1/T… 0 u
n T
nu )(F(u)
u-fmax fmax
How does the convolution result look like?
Sampling Analysis: Freq. Domain
1/T …-1/T… 0 u
)(uF(u)
u-fmax fmax
Sampling Analysis: Freq. Domain
)()( uuF
1/T …-1/T… 0 u
)(uF(u)
u-fmax fmax
Sampling Analysis: Freq. Domain
)()()( uGuuF
1/T …-1/T… 0 u
)(uF(u)
u-fmax fmax
G(0)?
G(fmax)?
G(u)?
Sampling Analysis: Freq. Domain
)()()( uGuuF
1/T …-1/T… 0 u
)(uF(u)
u-fmax fmax
G(0) = F(0)
G(fmax) = F(fmax)
G(u) = F(u)
Sampling Analysis: Freq. Domain
)()( uuF
)(u
1/T …-1/T… 0 u
F(u)
u-fmax fmax
Fs(u)
u-fmax fmax-1/T 1/T
)
1(
Tu How about
Sampling Analysis: Freq. Domain
)()( uuF
)(u
1/T …-1/T… 0 u
F(u)
u-fmax fmax
Fs(u)
u-fmax fmax-1/T 1/T
)
1(
Tu How about
Sampling Analysis: Freq. Domain
n T
nuuF )()(
1/T …-1/T… 0 u
n T
nu )(F(u)
u-fmax fmax
Fs(u)
u-fmax fmax-1/T 1/T
Sampling Theory
• How many samples are required to represent a given signal without loss of information?
• What signals can be reconstructed without loss for a given sampling rate?
Sampling Analysis: Freq. Domain
n T
nuuF )()(
1/T …-1/T… 0 u
n T
nu )(F(u)
u-fmax fmax
Fs(u)
u-fmax fmax-1/T 1/T
How can we reconstruct the original signal?
Reconstruction in Freq. Domain
u-fmax fmax
Fs(u)
u-fmax fmax
box(u)
u-fmax fmax
)()( uboxuFs
Signal Reconstruction in Freq. Domain
T 2T…-2T -T… 0
x
fs(x)
f(x)
x
Fs(u)
u-fmax fmax
F(u)
u-fmax fmax
)()( uboxuFs
Inverse Fourier transform
Fourier transform
Signal Reconstruction in Spatial Domain
)sin()( xxf s
T 2T…-2T -T… 0
x
x
fs(x) sinc(x)
Aliasing
Why does this happen?
Sampling Analysis
T …-T… 0
f(x) F(u)
fs(x)
x
ux -fmax fmax
Fs(u)
u-fmax fmax
When does aliasing happen?
Sampling Analysis
T …-T… 0
f(x) F(u)
fs(x)
x
ux -fmax fmax
Fs(u)
u-fmax fmax
max21
fT
When does aliasing happen?
Sampling Analysis
T …-T… 0
f(x) F(u)
fs(x)
x
ux -fmax fmax
Fs(u)
u-fmax fmax
max21
fT
When does aliasing happen?
Sampling Analysis
T …-T… 0
f(x) F(u)
fs(x)
x
ux -fmax fmax
Fs(u)
u-fmax fmax
max21
fT
When does aliasing happen?
Sampling Analysis
What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal?
f(x)
x
T 2T…-2T -T… 0
fs(x)
x
Sampling
Reconstruction
Sampling Analysis
What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal?
f(x)
x
T 2T…-2T -T… 0
fs(x)
x
Sampling
Reconstruction
Sampling Rate ≥ 2 * max frequency in the signal• this is known as the Nyquist Rate
Antialiasing
Antialiasing
Increase the sampling rate to above twice the highest frequency
u-fmax fmax
Antialiasing
Increase the sampling rate to above twice the highest frequency
Introduce an anti-aliasing filter to reduce fmax
u-fmax fmax
Fs(u)
u-fmax fmax
Antialiasing
Increase the sampling rate to above twice the highest frequency
Introduce an anti-aliasing filter to reduce fmax
u-fmax fmax
Fs(u)
u-fmax fmax
Sinc Filter
F(u)
u-fmax fmax
Is this a good filter?
Sinc Filter
F(u)
u-fmax fmax
x
Inverse Fourier transform
)(sinsin
)( xcx
xxf
Is this a good filter?
Sinc Filter
Is this a good filter?F(u)
u-fmax fmax
x
Multiplying with a box function in frequency domain
Convolution with a sinc function in spatial domain
Inverse Fourier transform
)(sinsin
)( xcx
xxf
Sinc Filter
Is this a good filter?F(u)
u-fmax fmax
x
Multiplying with a box function in frequency domain
Convolution with a sinc function in spatial domain
Good: removes all frequency components above a given bandwidth
Bad:
- an infinite support in spatial domain and hard to do in the spatial domain
- negative weights (not good for filtering pixels)
- fluctuation causes ripples
Inverse Fourier transform
)(sinsin
)( xcx
xxf
Good Pre-filtering filters
Finite support in frequency domain - cut-off frequency
Finite support in spatial domain - implementation in spatial domain
Positive weights
- good for filtering pixels
Other Filter Choices
Mean filter
Triangular filter
Gaussian filter
x
f(x)
u
|F(u)|sinc(u)
Sin2c(u)
2222 ue
-0.02
0.03
0.08
0.13
0.18
-0.02
0.03
0.08
0.13
0.18
2
2
2
2
1
x
e
2222 ue
Cannot have a filter with finite support in both spatial and frequency domain (box, mean, triangular)
- box filter: finite in freq. and infinite in spatial domain
- mean and triangular: infinite in freq. and finite in spatial domain
Gaussian filter provides a good tradeoff between two domains
Other Filter Choices
Mean filter
Triangular filter
Gaussian filter
x
f(x)
u
|F(u)|sinc(u)
Sin2c(u)
2222 ue
-0.02
0.03
0.08
0.13
0.18
-0.02
0.03
0.08
0.13
0.18
2
2
2
2
1
x
e
2222 ue
Cannot have a filter with finite support in both spatial and frequency domain (box, mean, triangular)
- box filter: finite in freq. and infinite in spatial domain
- mean and triangular: infinite in freq. and finite in spatial domain
Gaussian filter provides a good tradeoff between two domains
Gaussian Filters
Cannot have a filter with finite support in both spatial and frequency domain (box, mean, triangular)
- box filter: finite in freq. and infinite in spatial domain
- mean and triangular: infinite in freq. and finite in spatial domain
Gaussian filter provides a good tradeoff between two domains
Gaussian Filters
Cannot have a filter with finite support in both spatial and frequency domain (box, mean, triangular)
- box filter: finite in freq. and infinite in spatial domain
- mean and triangular: infinite in freq. and finite in spatial domain
Gaussian filter provides a good tradeoff between two domains
Gaussian FilterFs(u)
u-fmax fmax
Gaussian FilterFs(u)
u-fmax fmax-0.02
0.03
0.08
0.13
0.18
Inverse Fourier transform
Gaussian FilterFs(u)
u-fmax fmax
Multiplying with gaussian function in frequency domain
Convolution with a gaussian function in spatial domain
-0.02
0.03
0.08
0.13
0.18
Inverse Fourier transform
Gaussian FilterFs(u)
u-fmax fmax
Multiplying with gaussian function in frequency domain
Convolution with a gaussian function in spatial domain
- Since the Gaussian function decays rapidly, it is reasonable to truncate the filter window
-Easy to implement it in the spatial domain
- Its standard deviation controls the smoothness of filtered signal: large σ in spatial domain = lower cutoff frequency
-0.02
0.03
0.08
0.13
0.18
Inverse Fourier transform
Review: Gaussian
2222 ue -0.02
0.03
0.08
0.13
0.18
-0.02
0.03
0.08
0.13
0.18
2
2
2
2
1
x
e
If f(x) is gaussian, F(u) is also guassian.
2222 ue
Filtering in Spatial Domain
=
Filter functionInput imageFiltered image
Gaussian Filtering in Spatial Domain
Discrete approximation to Gaussian function with σ =1.0
filtering window: 5X5
?
Step 1: evaluate the filtering function at each grid
Step 2: Normalize each weight to ensure the sum of all weights is one
Gaussian Filtering in Spatial Domain
Discrete approximation to Gaussian function with σ =1.0
filtering window: 5X5
G(0,0)
Step 1: evaluate the filtering function at each grid
Step 2: Normalize each weight to ensure the sum of all weights is one
Gaussian Filtering in Spatial Domain
Discrete approximation to Gaussian function with σ =1.0
filtering window: 5X5
G(0,0)
?
Step 1: evaluate the filtering function at each grid
Step 2: Normalize each weight to ensure the sum of all weights is one
Gaussian Filtering in Spatial Domain
Discrete approximation to Gaussian function with σ =1.0
filtering window: 5X5
G(0,0)
Step 1: evaluate the filtering function at each grid
Step 2: Normalize each weight to ensure the sum of all weights is one
G(-2,-2)
Gaussian Filtering in Spatial Domain
Discrete approximation to Gaussian function with σ =1.0
Step 1: evaluate the filtering function at each grid
Step 2: Normalize each weight to ensure the sum of all weights is one
Filtering in Spatial Domain
Discrete approximation to Gaussian function with σ =1.0
Filtering in Spatial Domain
Discrete approximation to Gaussian function with σ =1.0
Filtering in Spatial Domain
Discrete approximation to Gaussian function with σ =1.0
Filtering in Spatial Domain
Discrete approximation to Gaussian function with σ =1.0
Filtered_I45 = Xwindownpixels
Gaussian Pre-filtering
G 1/4
G 1/8
Gaussian 1/2
Solution: filter the image, then subsample– Filter size should double for each ½ size reduction.
Why?
Gaussian Prefiltering
G 1/4 G 1/8Gaussian 1/2
Solution: filter the image, then subsample– Filter size should double for each ½ size reduction.
Why?
Without Prefiltering
1/4 (2x zoom) 1/8 (4x zoom)1/2
Known as a Gaussian Pyramid– MipMap (Williams, 1983)
Image Pyramids
MipMap
Mip (latin phase): many things in a small place
Used for texture mapping
Small-scale texture used for rendering distant objects
Gaussian Pyramid
filter mask
Repeat– Filter– Subsample
Until minimum resolution reached – can specify desired number of levels (e.g., 3-level pyramid)
The whole pyramid is only 4/3 the size of the original image!
Gaussian Pyramid
Laplacian Pyramid
Laplacian Pyramid (subband images)- Created from Gaussian pyramid by subtraction
- Encode fine to course structure in a different level
Laplacian Pyramid
What happens in frequency domain?
Where Are They Used?
Texture mapping - Mipmap
Image processing - Image composition/blending - Image compression
Computer vision - Image matching - Motion tracking - Optical flow estimation
Summary
2D Fourier Transform
Nyquist sampling theory - Sampling Rate ≥ 2 * max frequency in the signal
Antialiasing - prefiltering
Gaussian pyramid - Mipmap
Next Lecture
Image Processing
- Filtering noise
- Bilateral filter
- Feature extraction