Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
2011-04-19
Digital Image Processing
Achim J. Lilienthal
AASS Learning Systems Lab, Dep. Teknik
Room T1209 (Fr, 11-12 o'clock)
Digital Image ProcessingPart 3: Fourier Transform and
Filtering in the Frequency Domain
Course Book Chapter 4
Achim J. Lilienthal
Derivatives and their Fourier Transform
Laplacian in the Fourier Domain
[ ])()()( xfujdx
xfd nn
n
FF =
)(),( 22 vuvuH Laplacian +−=⇒
),()()],([ 222 vuFvuyxf +−=∇F
2 Relation Between Spatial and Frequency Filters
Achim J. Lilienthal
Derivatives and their Fourier Transform
Laplacian in the Fourier Domain
[ ])()()( xfujdx
xfd nn
n
FF =
( ) ( ) ( )vuFNvMuyxf ,22),(222
−+−−⇔∇
2 Relation Between Spatial and Frequency Filters
Achim J. Lilienthal
Laplacian in the Fourier Domain
2 Relation Between Spatial and Frequency Filters
Achim J. Lilienthal
Laplacian in the Fourier Domain
Laplacian in the Spatial Domain
( ) ( ) yxvuvu ++− −−+− 1]1)([ 221F
2
Achim J. Lilienthal
Deriving Spatial Filter Masks
General Idea select a filter in the frequency domain
transform this filter to the spatial domain
try to specify a small filter mask that captures the "essence" of the filter function
2
Achim J. Lilienthal
Filtering in the Frequency Domain
Other Filters bandpass
allows frequencies in a band between the two frequencies D0 and D1
bandstop: stops frequencies in a band
between the two frequencies D0 and D1
non-symmetric filters: allow different frequencies in the u and v direction
2
Achim J. Lilienthal
Recovering Intrinsic Images,Homomorphic Filtering
→ Contents
Achim J. Lilienthal
Reminder: Image Formation Model illumination i(x,y) from a source reflectivity r(x,y) = reflection / absorption in the scene
f(x,y) = r(x,y) i(x,y)
Image Formation3
Achim J. Lilienthal
Intrinsic Images "midlevel description" of scenes
proposed by Barrow and Tenebaum[H.G. Barrow and J.M. Tenenbaum. Recovering Intrinsic Scene Characteristics from Images". In Computer Vision Systems. Academic Press, 1978]
not a full 3D description of the scene
viewpoint dependent
physical causes of changes in illumination are not made explicit
Intrinsic Images3
Achim J. Lilienthal
Intrinsic Images "midlevel description" of scenes
proposed by Barrow and Tenebaum[H.G. Barrow and J.M. Tenenbaum. Recovering Intrinsic Scene Characteristics from Images". In Computer Vision Systems. Academic Press, 1978]
"The observed image is a product of two images: an illumination image and a reflectance image."
Intrinsic Images3
Achim J. Lilienthal
Intrinsic Images "midlevel description" of scenes
(input) image is decomposed into two images …
Intrinsic Images3
from "Deriving Intrinsic Images From Image Sequences",
Yair Weiss , Proc. ICCV 2001
Achim J. Lilienthal
Intrinsic Images "midlevel description" of scenes
(input) image is decomposed into two images … a reflectance image
Intrinsic Images3
from "Deriving Intrinsic Images From Image Sequences",
Yair Weiss , Proc. ICCV 2001
r(x,y)
Achim J. Lilienthal
Intrinsic Images "midlevel description" of scenes
(input) image is decomposed into two images … a reflectance image and
an illumination image
Intrinsic Images3
from "Deriving Intrinsic Images From Image Sequences",
Yair Weiss , Proc. ICCV 2001
r(x,y) i(x,y)
Achim J. Lilienthal
Intrinsic Images "midlevel description" of scenes
"The observed image is a product of two images: an illumination image and a reflectance image." segmentation on the intrinsic reflectance should be much simpler
than on the original image
3D information can be obtained from the illumination picture
Intrinsic Images3
from "Deriving Intrinsic Images From Image Sequences",
Yair Weiss , Proc. ICCV 2001
r(x,y) i(x,y)
Achim J. Lilienthal
Intrinsic Images "midlevel description" of scenes
(input) image is decomposed into two images … a reflectance image and
an illumination image
but: decomposition is an ill-posed problem number of unknowns is twice as high as the number of equations
(for example: set i(x,y) = 1 r(x,y) = f(x,y))
Intrinsic Images3
from "Deriving Intrinsic Images From Image Sequences",
Yair Weiss , Proc. ICCV 2001
),(),(),( yxryxiyxf =
Achim J. Lilienthal
Homomorphic Filtering
Idea: Separate Illumination and Reflectance
not separable directly …
… but the logarithm is separable
),(),(),( yxryxiyxf =
)],([)],([)],([ yxryxiyxf FFF ≠
[ ]),(ln),( yxfyxz ≡
[ ][ ] [ ][ ]),(),(),(),(ln),(ln)],([
vuFvuFvuZyxryxiyxz
ri +==+= FFF
3
Achim J. Lilienthal
Frequency Domain Approximation to Homomorphic Filtering – Assumption illumination component varies slowly
reflectance component tends to vary abruptly
⇒ use filter that affects low- and high-frequency components in a different way (decreases influence of illumination, increases influence of reflectance)
Homomorphic Filtering
),(),(),( yxryxiyxf =
γL<1
γH >1
3
Achim J. Lilienthal
Homomorphic Filtering
Idea: Separate Illumination and Reflectance
[ ][ ] [ ][ ]),(ln),(ln),(),(),( yxryxivuFvuFvuZ ri FF +=+=
),(),(),(),(),(),(),( vuFvuHvuFvuHvuZvuHvuS ri +==
[ ] [ ] [ ]),('),('
),(),(),(),(),(),( 111
yxryxivuFvuHvuFvuHvuSyxs ri
+=+== −−− FFF
),('),('),(),( zxrzxiyxs eeeyxg ==
3
Achim J. Lilienthal
Example consider non-uniform illumination
Homomorphic Filtering3
disturbance pattern
Achim J. Lilienthal
Example histogram equalization does not perform well
Homomorphic Filtering
histogram equalization
3
Achim J. Lilienthal
Example homomorphic filtering
Homomorphic Filtering3
homomorphic filtering
Achim J. Lilienthal
Homomorphic Filtering Assumption True?["Deriving Intrinsic Images From Image Sequences", Yair Weiss, Proc. ICCV 2001]
edges due to illumination often have as high a contrast as those due to reflectance changes possible solution: deriving intrinsic images from image sequences
Remarks3
Achim J. Lilienthal
Recovering Intrinsic Images from a Single Image["Recovering Intrinsic Images from a Single Image", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]
use colour information
Remarks3
Achim J. Lilienthal
Recovering Intrinsic Images from a Single Image[" Recovering Intrinsic Images from a Single Image ", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]
use colour information find chromaticity changes by classifying image derivatives by
thresholding scalar product of normalized RGB vector neighbours
Remarks3
Achim J. Lilienthal
Recovering Intrinsic Images from a Single Image[" Recovering Intrinsic Images from a Single Image ", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]
use colour information find chromaticity changes by classifying image derivatives by
thresholding scalar product of normalized RGB vector neighbours
Remarks3
Achim J. Lilienthal
Recovering Intrinsic Images from a Single Image[" Recovering Intrinsic Images from a Single Image ", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]
use colour information
learn appearance models of shading patterns classify gray-scale image
Remarks3
Achim J. Lilienthal
Recovering Intrinsic Images from a Single Image[" Recovering Intrinsic Images from a Single Image ", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]
use colour information
learn appearance models of shading patterns classify gray-scale image
Remarks3
Achim J. Lilienthal
Recovering Intrinsic Images from a Single Image[" Recovering Intrinsic Images from a Single Image ", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]
use colour information
learn appearance models of shading patterns classify gray-scale image
Remarks3
Achim J. Lilienthal
Recovering Intrinsic Images from a Single Image[" Recovering Intrinsic Images from a Single Image ", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]
use colour information
learn appearance models of shading patterns classify gray-scale image
propagate evidence (MRF model with learned parameters)
Remarks3
Achim J. Lilienthal
Recovering Intrinsic Images from a Single Image[" Recovering Intrinsic Images from a Single Image ", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]
use colour information +
learn appearance models of shading patterns +
propagate evidence (MRF model with learned parameters)
Remarks3
Achim J. Lilienthal
Recovering Intrinsic Images from a Single Image[" Recovering Intrinsic Images from a Single Image ", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]
use colour information +
learn appearance models of shading patterns +
propagate evidence (MRF model with learned parameters)
Remarks3
Achim J. Lilienthal
Recovering Intrinsic Images from a Single Image[" Recovering Intrinsic Images from a Single Image ", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]
use colour information +
learn appearance models of shading patterns +
propagate evidence (MRF model with learned parameters)
Remarks3
Achim J. Lilienthal
Recovering Intrinsic Images from a Single Image[" Recovering Intrinsic Images from a Single Image ", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]
Remarks3
Achim J. Lilienthal
Recovering Intrinsic Images from a Single Image[" Recovering Intrinsic Images from a Single Image ", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]
Remarks3
Achim J. Lilienthal
Recovering Intrinsic Images from a Single Image[" Recovering Intrinsic Images from a Single Image ", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]
Remarks3
Achim J. Lilienthal
Properties of the Fourier Transform
→ Contents
Achim J. Lilienthal
Properties of the Fourier Transform
a shift in f(x,y) does not affect |F(u,v)|
)//(200
00),(),( NyvMxujevuFyyxxf +−⇔−− π
Translation
4
Achim J. Lilienthal
Properties of the Fourier Transform
Translation a shift in f(x,y)
does not affect the spectrum |F(u,v)|
4
Achim J. Lilienthal
Distributive Over Addition
Not Distributive Over Multiplication
Scaling
Rotation
Properties of the Fourier Transform
)],([)],([)],(),([ 2121 yxfyxfyxfyxf FFF +=+
)],([)],([)],(),([ 2121 yxfyxfyxfyxf FFF ≠
),(),( vuaFyxaf ⇔ )/,/(1),( bvauFab
byaxf ⇔
),(),( 00 θϕωθθ +⇔+ Frf
4
Achim J. Lilienthal
Properties of the Fourier Transform
rotating f(x,y) rotates F(u,v) by the same angle
F(u,v)
f(x,y)
4
Rotation
Achim J. Lilienthal
Properties of the Fourier Transform
Periodicity
the discrete Fourier transform is periodic
also the inverse of the discrete Fourier transform is periodic
Conjugate Symmetry
the spectrum is symmetric about the origin
),(),(),(),( NvMuFNvuFvMuFvuF ++=+=+=
),(),(),(),( NyMxfNyxfyMxfyxf ++=+=+=
),(),( * vuFvuF −−= ),(),( * vuFvuF −−=⇒
4
Achim J. Lilienthal
Properties of the Fourier Transform
Separability
F(x,v) is the Fourier transform along one row
F(u,v) can be obtained by two successive applications of the simple 1D Fourier transform instead of by one application of the more complex 2D Fourier transform
4
[ ][ ]),(),(1
),(11
),(1),(
1
0
/2
1
0
/21
0
/2
1
0
1
0
)//(2
yxfvxFeM
eyxfN
eM
eyxfMN
vuF
vu
M
x
Muxj
N
y
NvyjM
x
Muxj
M
x
N
y
NvyMuxj
FF==
==
==
∑
∑∑
∑∑
−
=
−
−
=
−−
=
−
−
=
−
=
+−
π
ππ
π
Achim J. Lilienthal
Properties of the Fourier Transform
Fourier Transform
4
from "Computer Vision – A Modern Approach", Forsyth and Ponce,
Prentice Hall, 2002
( )1 ,box x ysin sinu v
u v( )( ),u vF f a b
ab
Achim J. Lilienthal
Correlation
Definition of Correlation
compare with convolution
also the need for padding
∑∑−
=
−
=
++=1
0
1
0
* ),(),(1),(),(M
m
N
nnymxhnmf
MNyxhyxf
∑∑−
=
−
=
−−=∗1
0
1
0),(),(),(),(
M
m
N
nnymxhnmfyxhyxf
4
Achim J. Lilienthal
Correlation
Template Matching
f(x,y) is the image
h(x,y) is a template
if h(x,y) matches somewhere in f(x,y) the correlation will be maximal there
∑∑−
=
−
=
++=1
0
1
0
* ),(),(1),(),(M
m
N
nnymxhnmf
MNyxhyxf
4
),(),(),(),( * vuHvuFyxhyxf ⇔
),(),(),(),(* vuHvuFyxhyxf ⇔
Achim J. Lilienthal
Correlation
Template Matching – Example 1
4
f1(x,y) – padded h(x,y) – padded
Achim J. Lilienthal
Correlation
Template Matching – Example 1
4
F -1[F1*(u,v)H(u,v)] – rescaledf1(x,y) – padded
Achim J. Lilienthal
Correlation
Template Matching – Example 1
4
F -1[F1*(u,v)H(u,v)] 4 – rescaled4f1(x,y) – padded
Achim J. Lilienthal
Correlation
Definition of Correlation (from previous slide)
Correlation Theorem
∑∑−
=
−
=
++=1
0
1
0
* ),(),(1),(),(M
m
N
nnymxhnmf
MNyxhyxf
),(),(),(),( * vuHvuFyxhyxf ⇔
),(),(),(),(* vuHvuFyxhyxf ⇔
4
Achim J. Lilienthal
Correlation
Template Matching – Example 2
4
f2(x,y) – padded h(x,y) – padded
Achim J. Lilienthal
Correlation
Template Matching – Example 2
4
f2(x,y) – padded F -1[F2*(u,v)H(u,v)] 4 – rescaled4
Achim J. Lilienthal
Correlation
Template Matching – Example 3
4
f3(x,y) – padded h(x,y) – padded
Achim J. Lilienthal
Correlation
Template Matching – Example 3
4
f3(x,y) – padded F -1[F3*(u,v)H(u,v)] 8 – rescaled8
Achim J. Lilienthal
Filters as Templates
Filters as Templates filters respond most strongly to patterns
that look like the filter
the kernel looks like the effect it is intended to detect
filtering has an analogy to computing a dot product measures similarity to the filter kernel
stronger response in brighter areas normalized correlation
filtering as changing the basis convolution can be seen as changing the base of an image
• base: vectors δ-functions base: shifted versions of the filter
• this process will typically loose information (coefficients on the new base can be redundant) but it might expose image structurein a useful way
4
Achim J. Lilienthal
Nyquist-Shannon Sampling Theorem
→ Contents
Achim J. Lilienthal
Nyquist-Shannon Sampling Theorem
Aliasing / Undersampling, Moiré Pattern
5
decrease resolution
Achim J. Lilienthal
Nyquist-Shannon Sampling Theorem
How to avoid Aliasing Problems? sampling
continuous function (irradiance in the camera) discrete grid
number of samples relative tothe function seems important a signal sampled too slowly is
misrepresented by the samples
5
from "Computer Vision – A Modern Approach", Forsyth and Ponce, Prentice Hall, 2002
Achim J. Lilienthal
Nyquist-Shannon Sampling Theorem
Sampling a Signal in 1D
Reconstruction of the Original Continuous Signal which sample rate?
how to derive the continuous signal from the samples?
how to model the sampling process?
f(x)
x
5
x… …
Achim J. Lilienthal
Nyquist-Shannon Sampling Theorem
Sampling a Signal in 1D
How to Model the Sampling Process? continuous model of a sampled signal needed
sum of delta functions (2D: "bed-of-nails function") sampling process = multiplication with a sampling function fIII(x)
f(x)
x x… …
5
… …
×
Achim J. Lilienthal
Nyquist-Shannon Sampling Theorem
Considering Band-Limited Signals
signal band-width: band/range of non-zero frequencies
a band-limited signal is constrained in terms of how fast it can change
f(x)
x
F(u)
w-w
u
F
→
5
Achim J. Lilienthal
Nyquist-Shannon Sampling Theorem
Fourier Transform of a Sampled Signal
sampling = multiplication with the sampling function in the spatial domain
f(x)⋅fIII(x)
x… …
5
Achim J. Lilienthal
Nyquist-Shannon Sampling Theorem
Fourier Transform of a Sampled Signal
sampling = multiplication with the sampling function in the spatial domain
equals convolution in the frequency domain
f(x)⋅fIII(x)
x… …
F(u)∗FIII(u)
5
F
→Fourier transform of a
"Dirac comb" is again a Dirac comb
( Poisson summation)
Achim J. Lilienthal
Nyquist-Shannon Sampling Theorem
Fourier Transform of a Sampled Signal
sampling = multiplication with the sampling function in the spatial domain
equals convolution in the frequency domain
F
→
…
f(x)⋅fIII(x)
x… …
F(u)∗FIII(u)
u2 w
5
Achim J. Lilienthal
Nyquist-Shannon Sampling Theorem
Fourier Transform of a Sampled Signal
non-overlapping support of the "shifted Fourier Transforms" we can reconstruct the signal from the sampled versions
F
→
…
f(x)⋅fIII(x)
x… …
F(u)∗FIII(u)
u2 w
5
Achim J. Lilienthal
Nyquist-Shannon Sampling Theorem
Reconstruction of the Signal
5
Cut out by multiplication with box filterInverse
Fourier Transform
from "Computer Vision – A Modern Approach", Forsyth and Ponce, Prentice Hall, 2002
Achim J. Lilienthal
Nyquist-Shannon Sampling Theorem
Reconstruction of the Signal but if support regions do overlap?
we can't reconstruct the signal
Fourier transform in the regions that overlap can't be determined
5
Achim J. Lilienthal
Nyquist-Shannon Sampling Theorem
Fourier Transform of a "Dirac comb"
reciprocal behaviour of Δx and Δu
F
→
x
fIII(x)
… …
Δxu
FIII(u)
… …
1/Δx
5
2 w
Achim J. Lilienthal
Nyquist-Shannon Sampling Theorem
Sampling Theorem there should be no overlap
between the repetitions of the FT of the signal
the sampling interval should be at least the double of the highest frequency (1/w) present in the signal
wxxw 2112 ≤∆⇒∆≤⇒
5
u
FIII(u)
… …
1/Δx2 w