Embed Size (px)
Citation preview
Short Time Fourier Transform
• Time/Frequency localization depends on window size. • Once you choose a particular window size, it will be the same
for all frequencies. • Many signals require a more flexible approach - vary the
window size to determine more accurately either time or frequency.
The Wavelet Transform
• Overcomes the preset resolution problem of the STFT by using a variable length window:
– Narrower windows are more appropriate at high frequencies
(better time localization)
– Wider windows are more appropriate at low frequencies (better frequency localization)
The Wavelet Transform (cont’d)
Wide windows do not provide good localization at high frequencies.
The Wavelet Transform (cont’d)
A narrower window is more appropriate at high frequencies.
The Wavelet Transform (cont’d)
Narrow windows do not provide good localization at low frequencies.
The Wavelet Transform (cont’d)
A wider window is more appropriate at low frequencies.
What is a wavelet? • It is a function that “waves” above and below the x-axis;
it has (1) varying frequency, (2) limited duration, and (3) an average value of zero.
• This is in contrast to sinusoids, used by FT, which have infinite energy.
Sinusoid Wavelet
Wavelets • Like sine and cosine functions in FT, wavelets can define
basis functions ψk(t):
• Span of ψk(t): vector space S containing all functions f(t) that can be represented by ψk(t).
( ) ( )k kk
f t a tψ=∑
Wavelets (cont’d)
• There are many different wavelets:
Morlet Haar Daubechies
Basis Construction - Mother Wavelet
( ) jk tψ=
Basis Construction - Mother Wavelet (cont’d)
scale =1/2j (1/frequency)
( )/2( ) 2 2 j jjk t t kψ ψ= −
j
k
Discrete Wavelet Transform (DWT)
( ) ( )jk jkk j
f t a tψ=∑∑
( )/2( ) 2 2 j jjk t t kψ ψ= −
(inverse DWT)
(forward DWT)
where
*( ) ( )jkjkt
a f t tψ=∑
DFT vs DWT
• FT expansion:
• WT expansion
or
one parameter basis
( ) ( )l ll
f t a tψ=∑
( ) ( )jk jkk j
f t a tψ=∑∑
two parameter basis
Multiresolution Representation using
high resolution (more details)
low resolution (less details)
…
( ) ( )jk jkk j
f t a tψ=∑ ∑
( )f t
1̂( )f t
2̂ ( )f t
ˆ ( )sf t
( )jk tψ
j
Prediction Residual Pyramid - Revisited
• In the absence of quantization errors, the approximation pyramid can be reconstructed from the prediction residual pyramid. • Prediction residual pyramid can be represented more efficiently.
(with sub-sampling)
Efficient Representation Using “Details”
details D2
L0
details D3
details D1
(without sub-sampling)
Efficient Representation Using Details (cont’d)
representation: L0 D1 D2 D3
A wavelet representation of a function consists of (1) a coarse overall approximation (2) detail coefficients that influence the function at various scales.
in general: L0 D1 D2 D3…DJ
Reconstruction (synthesis) H3=H2+D3
details D2
L0
details D3
H2=H1+D2
H1=L0+D1
details D1
(without sub-sampling)
Example - Haar Wavelets
• Suppose we are given a 1D "image" with a resolution of 4 pixels:
[9 7 3 5]
• The Haar wavelet transform is the following:
L0 D1 D2 D3 (with sub-sampling)
Example - Haar Wavelets (cont’d)
• Start by averaging the pixels together (pairwise) to get a new lower resolution image:
• To recover the original four pixels from the two
averaged pixels, store some detail coefficients.
1
[9 7 3 5]
Example - Haar Wavelets (cont’d)
• Repeating this process on the averages (i.e., low resolution image) gives the full decomposition:
1
Harr decomposition:
Example - Haar Wavelets (cont’d)
• We can reconstruct the original image by adding or subtracting the detail coefficients from the lower-resolution versions.
2 1 -1
Example - Haar Wavelets (cont’d)
Detail coefficients become smaller and smaller as j increases.
Dj
Dj-1
D1 L0
How to compute Di ?
Multiresolution Conditions • If a set of functions can be represented by a weighted
sum of ψ(2jt - k), then a larger set, including the original, can be represented by a weighted sum of ψ(2j
+1t - k):
low resolution
high resolution
j
Multiresolution Conditions (cont’d) • If a set of functions can be represented by a weighted
sum of ψ(2jt - k), then a larger set, including the original, can be represented by a weighted sum of ψ(2j
+1t - k):
Vj: span of ψ(2jt - k): ( ) ( )j k jkk
f t a tψ=∑
Vj+1: span of ψ(2j+1t - k): 1 ( 1)( ) ( )j k j kk
f t b tψ+ +=∑
1j jV V +⊆
Nested Spaces Vj
ψ(t - k)
ψ(2t - k)
ψ(2jt - k)
…
V0
V1
Vj
Vj : space spanned by ψ(2jt - k)
Multiresolution conditions à nested spanned spaces:
( ) ( )jk jkk j
f t a tψ=∑ ∑f(t) ϵ Vj
Basis functions:
i.e., if f(t) ϵ V j then f(t) ϵ V j+1
1j jV V +⊂
How to compute Di ? (cont’d)
( ) ( )jk jkk j
f t a tψ=∑ ∑f(t) ϵ Vj
IDEA: define a set of basis functions that span the difference between Vj+1 and Vj
Orthogonal Complement Wj • Let Wj be the orthogonal complement of Vj in Vj+1
Vj+1 = Vj + Wj
How to compute Di ? (cont’d) If f(t) ϵ Vj+1, then f(t) can be represented using basis functions φ(t) from Vj+1:
1( ) (2 )jk
kf t c t kϕ += −∑
( ) (2 ) (2 )j jk jk
k kf t c t k d t kϕ ψ= − + −∑ ∑
Vj+1 = Vj + Wj
Alternatively, f(t) can be represented using two sets of basis functions, φ(t) from Vj and ψ(t) from Wj:
Vj+1
Think of Wj as a means to represent the parts of a function in Vj+1 that cannot be represented in Vj
1( ) (2 )jk
kf t c t kϕ += −∑
( ) (2 ) (2 )j jk jk
k kf t c t k d t kϕ ψ= − + −∑ ∑
Vj Wj
How to compute Di ? (cont’d)
differences between Vj and Vj+1
How to compute Di ? (cont’d) • à using recursion on Vj:
( ) ( ) (2 )jk jk
k k jf t c t k d t kϕ ψ= − + −∑ ∑∑
V0 W0, W1, W2, … basis functions basis functions
Vj+1 = Vj-1+Wj-1+Wj = …= V0 + W0 + W1 + W2 + … + Wj
if f(t) ϵ Vj+1 , then:
Vj+1 = Vj + Wj
Summary: wavelet expansion (Section 7.2)
• Wavelet decompositions involve a pair of waveforms (mother wavelets):
φ(t) ψ(t) encode low resolution info
encode details or high resolution info
( ) ( ) (2 )jk jk
k k jf t c t k d t kϕ ψ= − + −∑ ∑∑
scaling function wavelet function
1D Haar Wavelets
• Haar scaling and wavelet functions:
computes average computes details
φ(t) ψ(t)
1D Haar Wavelets (cont’d)
• V0 represents the space of one pixel images
• Think of a one-pixel image as a function that is constant over [0,1)
Example: 0 1
width: 1
1D Haar Wavelets (cont’d)
• V1 represents the space of all two-pixel images • Think of a two-pixel image as a function having 21
equal-sized constant pieces over the interval [0, 1).
• Note that
Examples: 0 ½ 1
0 1V V⊂
= +
width: 1/2
e.g.,
1D Haar Wavelets (cont’d) • V j represents all the 2j-pixel images • Functions having 2j equal-sized constant pieces over
interval [0,1).
• Note that
Examples: width: 1/2j
ϵ Vj ϵ Vj
1j jV V− ⊂
e.g.,
1D Haar Wavelets (cont’d)
V0, V1, ..., V j are nested
i.e.,
VJ … V1 V0 coarse details
fine details
1j jV V +⊂
Define a basis for Vj
• Mother scaling function:
• Let’s define a basis for V j :
0 1
note alternative notation: ( ) ( )ji jix xϕ ϕ≡
1
Define a basis for Vj (cont’d)
Define a basis for Wj (cont’d)
• Mother wavelet function:
• Let’s define a basis ψ ji for Wj :
1 -1
0 1/2 1
( ) ( )ji jix xψ ψ≡note alternative notation:
Define a basis for Wj
Note that the dot product between basis functions in Vj and Wj is zero!
Define a basis for Wj (cont’d)
Basis functions ψ ji of W j Basis functions φ ji of V j
form a basis in V j+1
Example - Revisited
f(x)=
V2
Example (cont’d)
V1and W1
V2=V1+W1
φ1,0(x)
φ1,1(x)
ψ1,0(x)
ψ1,1(x)
(divide by 2 for normalization)
Example (cont’d)
V2=V1+W1=V0+W0+W1
V0 ,W0 and W1
φ0,0(x) ψ0,0(x)
ψ1,0 (x)
ψ1,1(x)
(divide by 2 for normalization)
Example
Example (cont’d)
Filter banks • The lower resolution coefficients can be calculated
from the higher resolution coefficients by a tree-structured algorithm (i.e., filter bank).
h0(-n) is a lowpass filter and h1(-n) is a highpass filter
Subband encoding (analysis)
Example (revisited)
[9 7 3 5] low-pass, down-sampling
high-pass, down-sampling
(9+7)/2 (3+5)/2 (9-7)/2 (3-5)/2 V1 basis functions
Filter banks (cont’d) Next level:
Example (revisited)
[9 7 3 5]
high-pass, down-sampling
low-pass, down-sampling
(8+4)/2 (8-4)/2
V1 basis functions
Convention for illustrating 1D Haar wavelet decomposition
x x x x x x … x x
detail average
…
re-arrange:
re-arrange:
V1 basis functions
Examples of lowpass/highpass (analysis) filters
Haar h0
h1
Filter banks (cont’d)
• The higher resolution coefficients can be calculated from the lower resolution coefficients using a similar structure.
Subband encoding (synthesis)
Filter banks (cont’d) Next level:
Examples of lowpass/highpass (synthesis) filters
Haar (same as for analysis):
+
g0
g1
2D Haar Wavelet Transform
• The 2D Haar wavelet decomposition can be computed using 1D Haar wavelet decompositions. – i.e., 2D Haar wavelet basis is separable
• Two decompositions (i.e., correspond to different basis functions): – Standard decomposition – Non-standard decomposition
Standard Haar wavelet decomposition
• Steps
(1) Compute 1D Haar wavelet decomposition of each row of the original pixel values.
(2) Compute 1D Haar wavelet decomposition of each column of the row-transformed pixels.
Standard Haar wavelet decomposition (cont’d)
x x x … x x x x … x … … . x x x ... x
(1) row-wise Haar decomposition:
…
detail average
…
… … .
…
…
… … .
re-arrange terms
Standard Haar wavelet decomposition (cont’d)
(1) row-wise Haar decomposition:
…
detail average
…
… … … .
…
… … . …
row-transformed result
Standard Haar wavelet decomposition (cont’d)
(2) column-wise Haar decomposition:
…
detail average
…
… … … .
… …
… … … .
…
row-transformed result column-transformed result
Example
… …
… … … .
row-transformed result
…
… … .
re-arrange terms
Example (cont’d)
… …
… … … .
column-transformed result
Non-standard Haar wavelet decomposition
• Alternates between operations on rows and columns.
(1) Perform one level decomposition in each row (i.e., one step of horizontal pairwise averaging and differencing).
(2) Perform one level decomposition in each column from step 1 (i.e., one step of vertical pairwise averaging and differencing).
(3) Repeat the process on the quadrant containing averages
only (i.e., in both directions).
Non-standard Haar wavelet decomposition (cont’d)
x x x … x x x x … x … … . x x x . . . x
one level, horizontal Haar decomposition:
… …
… … .
…
… …
… … .
one level, vertical Haar decomposition:
…
Non-standard Haar wavelet decomposition (cont’d)
one level, horizontal Haar decomposition on “green” quadrant
one level, vertical Haar decomposition on “green” quadrant
… …
… … .
… …
re-arrange terms
… …
… … … .
…
Example
… …
… … .
… …
re-arrange terms
Example (cont’d)
… …
… … … .
2D DWT using filter banks (analysis) LL LH
HL HH
2D DWT using filter banks (analysis)
LL Hà LH Và HL Dà HH
The wavelet transform is applied again on the LL sub-image
LL LL LH
HL HH
2D DWT using filter banks (analysis)
Fast Multiresolution Image Querying painted low resolution target
queries
Charles E. Jacobs Adam Finkelstein David H. Salesin, "Fast Multiresolution Image Querying", SIGRAPH, 1995.
