View
232
Download
0
Tags:
Embed Size (px)
Citation preview
CSCE 641 Computer Graphics:Image Sampling and Reconstruction
Jinxiang Chai
Review: 1D Fourier Transform
A function f(x) can be represented as a weighted combination of phase-shifted sine waves
How to compute F(u)?
dxexfuF uxi 2)()(
dueuFxf uxi 2)()( Inverse Fourier Transform
Fourier Transform
Review: Box Function
21 02
1 1)(
x
xxf
)(sinsin
)( ucu
uuF
x
u
f(x)
|F(u)|
If f(x) is bounded, F(u) is unbounded
Review: Cosine
-1.5
-1
-0.5
0
0.5
1
1.5
1-1
If f(x) is even, so is F(u)
)2cos( x
Review: Gaussian
2222 ue -0.02
0.03
0.08
0.13
0.18
-0.02
0.03
0.08
0.13
0.18
2
2
2
2
1
x
e
If f(x) is gaussian, F(u) is also guassian.
2222 ue
Review: Properties
Linearity:
Time shift:
Derivative:
Integration:
Convolution:
)()()()( ubGuaFxbgxaf
)()( 020 uFexxf uxi
)()(
uuFx
xdf
u
uFdxxf
)()(
)()()()( uGuFxgxf
Outline
2D Fourier Transform
Nyquist sampling theory
Antialiasing
Gaussian pyramid
Extension to 2D
Fourier Transform:
Inverse Fourier transform:
dvduevuFyxf vyuxi )(2),(),(
dydxeyxfvuF vyuxi )(2),(),(
Building Block for 2D Transform)(2 vyuxie
Building block:
Frequency:
Orientation:
22 vu
),tan( uvaOriented wave fields
Building Block for 2D Transform
Higher frequency
)(2 vyuxie Building block:
Frequency:
Orientation:
22 vu
),tan( uvaOriented wave fields
Some 2D Transforms
From Lehar
Some 2D Transforms
Why we have a DC component?
From Lehar
Some 2D Transforms
Why we have a DC component?
From Lehar
Some 2D Transforms
Why we have a DC component?
From Lehar
Some 2D Transforms
Why we have a DC component?
From Lehar
Some 2D Transforms
Why we have a DC component?
- the sum of all pixel values
From Lehar
dydxeyxfF yxi )00(2),()0,0(
Some 2D Transforms
Why we have a DC component?
- the sum of all pixel values
Oriented stripe in spatial domain = an oriented line in spatial domain
From Lehar
dydxeyxfF yxi )00(2),()0,0(
2D Fourier Transform
Why?- Any relationship between two slopes?
2D Fourier Transform
Linearity
Why?- Any relationship between two slopes?
2D Fourier Transform
Why is the spectrum bounded?
Linearity
Why?- Any relationship between two slopes?
Online Java Applet
http://www.brainflux.org/java/classes/FFT2DApplet.html
2D Fourier Transform Pairs
u vk k y
v
x
u
T
kv
T
ku ),(
x
yxdf ),( ),( vuuF
Gaussian Gaussian
x yk k
yyxx TkyTkx ),(
y
yxdf ),( ),( vuvF
2D Image FilteringInverse
transformFourier
transform
From Lehar
2D Image FilteringInverse
transformFourier
transform
Low-pass filter
From Lehar
2D Image FilteringInverse
transformFourier
transform
Low-pass filter
high-pass filter
From Lehar
2D Image FilteringInverse
transformFourier
transform
Low-pass filter
high-pass filter
band-pass filter
From Lehar
Aliasing
Why does this happen?
Aliasing
How to reduce it?
Sampling Analysis
f(x)
x
T 2T…-2T -T… 0
fs(x)
x
Sampling
Sampling Analysis
f(x)
x
T 2T…-2T -T… 0
fs(x)
x
Sampling
Reconstruction
Sampling Analysis
What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal?
f(x)
x
T 2T…-2T -T… 0
fs(x)
x
Sampling
Reconstruction
Sampling Theory
• How many samples are required to represent a given signal without loss of information?
• What signals can be reconstructed without loss for a given sampling rate?
Sampling Analysis: Spatial Domain
k
kTxxf )()(
f(x)
T 2T…-2T -T… 0
fs(x)
x
x T 2T…-2T -T… 0 x
k
kTx )(
X
?
Sampling Analysis: Spatial Domain
k
kTxxf )()(
f(x)
T 2T…-2T -T… 0
fs(x)
x
x T 2T…-2T -T… 0 x
k
kTx )(
X
? What happens in Frequency domain?
Fourier Transform of Dirac Comb
k
kTx )(
T
Review: Dirac Delta and its Transform
00
0)(
x
xx
x
1)( uF1
u
f(x)
|F(u)|
Fourier transform and inverse Fourier transform arequalitatively the same, so knowing one direction gives you the other
1)( dxx
)0()()( fdxxxf
Review: Fourier Transform Properties
Linearity:
Time shift:
Derivative:
Integration:
Convolution:
)()()()( ubGuaFxbgxaf
)()( 020 uFexxf uxi
)()(
uuFx
xdf
u
uFdxxf
)()(
)()()()( uGuFxgxf
Fourier Transform of Dirac Comb
k
kTx )(
T
k
ukTie 2
Fourier Transform of Dirac Comb
kk
ukTi
T
kue )(2
Fourier Transform of Dirac Comb
k
kTx )( )(T
ku
k
T 1/T
k
ukTie 2
Moving the spikes closer together in the spatial domain moves them farther apart in the frequency domain!
Sampling Analysis: Spatial Domain
k
kTxxf )()(
f(x)
T 2T…-2T -T… 0
fs(x)
x
x T 2T…-2T -T… 0 x
k
kTx )(
X
? What happens in Frequency domain?
Sampling Analysis: Freq. Domain
1/T …-1/T… 0 u
n T
nu )(F(u)
u-fmax fmax
Sampling Analysis: Freq. Domain
1/T …-1/T… 0 u
n T
nu )(F(u)
u-fmax fmax
How does the convolution result look like?
Sampling Analysis: Freq. Domain
1/T …-1/T… 0 u
)(uF(u)
u-fmax fmax
Sampling Analysis: Freq. Domain
)()( uuF
1/T …-1/T… 0 u
)(uF(u)
u-fmax fmax
Sampling Analysis: Freq. Domain
)()()( uGuuF
1/T …-1/T… 0 u
)(uF(u)
u-fmax fmax
G(0)?
G(fmax)?
G(u)?
Sampling Analysis: Freq. Domain
)()()( uGuuF
1/T …-1/T… 0 u
)(uF(u)
u-fmax fmax
G(0) = F(0)
G(fmax) = F(fmax)
G(u) = F(u)
Sampling Analysis: Freq. Domain
)()( uuF
)(u
1/T …-1/T… 0 u
F(u)
u-fmax fmax
Fs(u)
u-fmax fmax-1/T 1/T
)
1(
Tu How about
Sampling Analysis: Freq. Domain
)()( uuF
)(u
1/T …-1/T… 0 u
F(u)
u-fmax fmax
Fs(u)
u-fmax fmax-1/T 1/T
)
1(
Tu How about
Sampling Analysis: Freq. Domain
n T
nuuF )()(
1/T …-1/T… 0 u
n T
nu )(F(u)
u-fmax fmax
Fs(u)
u-fmax fmax-1/T 1/T
Sampling Theory
• How many samples are required to represent a given signal without loss of information?
• What signals can be reconstructed without loss for a given sampling rate?
Sampling Analysis: Freq. Domain
n T
nuuF )()(
1/T …-1/T… 0 u
n T
nu )(F(u)
u-fmax fmax
Fs(u)
u-fmax fmax-1/T 1/T
How can we reconstruct the original signal?
Reconstruction in Freq. Domain
u-fmax fmax
Fs(u)
u-fmax fmax
box(u)
u-fmax fmax
)()( uboxuFs
Signal Reconstruction in Freq. Domain
T 2T…-2T -T… 0
x
fs(x)
f(x)
x
Fs(u)
u-fmax fmax
F(u)
u-fmax fmax
)()( uboxuFs
Inverse Fourier transform
Fourier transform
Signal Reconstruction in Spatial Domain
)sin()( xxf s
T 2T…-2T -T… 0
x
x
fs(x) sinc(x)
Aliasing
Why does this happen?
Sampling Analysis
T …-T… 0
f(x) F(u)
fs(x)
x
ux -fmax fmax
Fs(u)
u-fmax fmax
When does aliasing happen?
Sampling Analysis
T …-T… 0
f(x) F(u)
fs(x)
x
ux -fmax fmax
Fs(u)
u-fmax fmax
max21
fT
When does aliasing happen?
Sampling Analysis
T …-T… 0
f(x) F(u)
fs(x)
x
ux -fmax fmax
Fs(u)
u-fmax fmax
max21
fT
When does aliasing happen?
Sampling Analysis
T …-T… 0
f(x) F(u)
fs(x)
x
ux -fmax fmax
Fs(u)
u-fmax fmax
max21
fT
When does aliasing happen?
Sampling Analysis
What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal?
f(x)
x
T 2T…-2T -T… 0
fs(x)
x
Sampling
Reconstruction
Sampling Analysis
What sampling rate (T) is sufficient to reconstruct the continuous version of the sampled signal?
f(x)
x
T 2T…-2T -T… 0
fs(x)
x
Sampling
Reconstruction
Sampling Rate ≥ 2 * max frequency in the signal• this is known as the Nyquist Rate
Antialiasing
Antialiasing
Increase the sampling rate to above twice the highest frequency
u-fmax fmax
Antialiasing
Increase the sampling rate to above twice the highest frequency
Introduce an anti-aliasing filter to reduce fmax
u-fmax fmax
Fs(u)
u-fmax fmax
Antialiasing
Increase the sampling rate to above twice the highest frequency
Introduce an anti-aliasing filter to reduce fmax
u-fmax fmax
Fs(u)
u-fmax fmax
Sinc Filter
F(u)
u-fmax fmax
Is this a good filter?
Sinc Filter
F(u)
u-fmax fmax
x
Inverse Fourier transform
)(sinsin
)( xcx
xxf
Is this a good filter?
Sinc Filter
Is this a good filter?F(u)
u-fmax fmax
x
Multiplying with a box function in frequency domain
Convolution with a sinc function in spatial domain
Inverse Fourier transform
)(sinsin
)( xcx
xxf
Sinc Filter
Is this a good filter?F(u)
u-fmax fmax
x
Multiplying with a box function in frequency domain
Convolution with a sinc function in spatial domain
Good: removes all frequency components above a given bandwidth
Bad:
- an infinite support in spatial domain and hard to do in the spatial domain
- Fluctuation causes ripples
- Negative weights (not good for filtering pixles)
Inverse Fourier transform
)(sinsin
)( xcx
xxf
Good Pre-filtering filters
Finite support in frequency domain - cut-off frequency
Finite support in spatial domain - implementation in spatial domain
Positive weights
- good for filtering pixels
Other Filter Choices
Mean filter
Triangular filter
Gaussian filter
x
f(x)
u
|F(u)|sinc(u)
Sin2c(u)
2222 ue
-0.02
0.03
0.08
0.13
0.18
-0.02
0.03
0.08
0.13
0.18
2
2
2
2
1
x
e
2222 ue
Cannot have a filter with finite support in both spatial and frequency domain (box, mean, triangular)
- box filter: finite in freq. and infinite in spatial domain
- mean and triangular: infinite in freq. and finite in spatial domain
Gaussian filter provides a good tradeoff between two domains
Other Filter Choices
Mean filter
Triangular filter
Gaussian filter
x
f(x)
u
|F(u)|sinc(u)
Sin2c(u)
2222 ue
-0.02
0.03
0.08
0.13
0.18
-0.02
0.03
0.08
0.13
0.18
2
2
2
2
1
x
e
2222 ue
Cannot have a filter with finite support in both spatial and frequency domain (box, mean, triangular)
- box filter: finite in freq. and infinite in spatial domain
- mean and triangular: infinite in freq. and finite in spatial domain
Gaussian filter provides a good tradeoff between two domains
Gaussian FilterFs(u)
u-fmax fmax
Gaussian FilterFs(u)
u-fmax fmax-0.02
0.03
0.08
0.13
0.18
Inverse Fourier transform
Gaussian FilterFs(u)
u-fmax fmax
Multiplying with gaussian function in frequency domain
Convolution with a gaussian function in spatial domain
-0.02
0.03
0.08
0.13
0.18
Inverse Fourier transform
Gaussian FilterFs(u)
u-fmax fmax
Multiplying with gaussian function in frequency domain
Convolution with a gaussian function in spatial domain
- Since the Gaussian function decays rapidly, it is reasonable to truncate the filter window
- Its standard deviation controls the smoothness of filtered signal: large σ in spatial domain = lower cutoff frequency
- Easy to implement it in the spatial domain
-0.02
0.03
0.08
0.13
0.18
Inverse Fourier transform
Filtering in Spatial Domain
=
Filter functionInput imageFiltered image
Gaussian Filtering in Spatial Domain
Discrete approximation to Gaussian function with σ =1.0
Filtering in Spatial Domain
Discrete approximation to Gaussian function with σ =1.0
Filtering in Spatial Domain
Discrete approximation to Gaussian function with σ =1.0
Filtering in Spatial Domain
Discrete approximation to Gaussian function with σ =1.0
Filtering in Spatial Domain
Discrete approximation to Gaussian function with σ =1.0
Filtered_I45 = Xwindownpixels
Gaussian Pre-filtering
G 1/4
G 1/8
Gaussian 1/2
Solution: filter the image, then subsample– Filter size should double for each ½ size reduction.
Why?
Gaussian Pre-filtering
G 1/4 G 1/8Gaussian 1/2
Solution: filter the image, then subsample– Filter size should double for each ½ size reduction.
Why?
Without Prefiltering
1/4 (2x zoom) 1/8 (4x zoom)1/2
Known as a Gaussian Pyramid– MipMap (Williams, 1983)
Image Pyramids
MipMap
Mip (latin phase): many things in a small place
Used for texture mapping
Small-scale texture used for rendering distant objects
Gaussian Pyramid
filter mask
Repeat– Filter– Subsample
Until minimum resolution reached – can specify desired number of levels (e.g., 3-level pyramid)
The whole pyramid is only 4/3 the size of the original image!
Laplacian Pyramid
Laplacian Pyramid (subband images)- Created from Gaussian pyramid by subtraction
- Encode fine to course structure in a different level
Laplacian Pyramid
What happens in frequency domain?
Where Are They Used?
Texture mapping - Mipmap
Image processing - Image composition/blending - Image compression
Computer vision - Image matching - Motion tracking - Optical flow estimation
Summary
2D Fourier Transform
Nyquist sampling theory - Sampling Rate ≥ 2 * max frequency in the signal
Antialiasing - prefiltering
Gaussian pyramid - Mipmap
Next Lecture
Image Processing
- Filtering noise
- Bilateral filter
- Feature extraction