Wavelets and Multiresolution Processing

Thinh Nguyen

Multiresolution Multiresolution Analysis (MRA)Analysis (MRA)

A scaling function scaling function is used to create a series of approximations of a function or image, each differing by a factor of 2 from its neighboring approximations.

Additional functions called wavelets wavelets are then used to encode the difference in information between adjacent approximations.

Express a signal as

If the expansion is unique, the are called basis functionsbasis functions, and the expansion set

is called a basisbasis

Series ExpansionsSeries Expansions

( )f x( ) ( )k k

kf x xα ϕ=∑

expansion coefficientsexpansion functions

( )k xϕ

{ }( )k xϕ

All the functions expressible with this basis form a function spacefunction space which is referred to as the closed spanclosed span of the expansion set

If , then is in the closed span of and can be expressed as

Series ExpansionsSeries Expansions

{ }( )kk

V Span xϕ=

( )f x V∈ ( )f x{ }( )k xϕ

( ) ( )k kk

f x xα ϕ=∑

The expansion functions form an orthonormal basis for V

The basis and its dual are equivalent, i.e.,and

OrthonormalOrthonormal BasisBasis

0( ), ( )

1j k jkj k

x xj k

ϕ ϕ δ≠⎧

= = ⎨ =⎩

( ) ( )k kx xϕ ϕ= %

( ), ( ) ( ) ( )k k kx f x x f x dxα ϕ ϕ∗= = ∫

Consider the set of expansion functions composed

of integer translations and binary scalings of the

real square-integrable function defined by

for all and

By choosing the scaling function wisely,

can be made to span

Scaling FunctionsScaling Functions

( )xϕ

{ } { }/ 2, ( ) 2 (2 )j j

j k x x kϕ ϕ= −

,j k∈ 2( ) ( )x Lϕ ∈

{ }, ( )j k xϕ2( )L

( )xϕ

Index k determines the position of

along the x-axis, index j determines its width;

controls its height or amplitude.

By restricting j to a specific value the

resulting expansion set is a subset of

One can write

, ( )j k xϕ

/ 22 j

{ } { }/ 2, ( ) 2 (2 )j j

j k x x kϕ ϕ= −

oj j=

{ }, ( )j k xϕ{ }, ( )

oj k xϕ

{ }, ( )o oj j k

kV Span xϕ=

Example: The Example: The Haar Haar Scaling FunctionScaling Function

1,0 1,1 1,4( ) 0.5 ( ) ( ) 0.25 ( )f x x x xϕ ϕ ϕ= + −0, 1,2 1,2 1

1 1( ) ( ) ( )2 2k k kx x xϕ ϕ ϕ += +

0 1V V⊂

1 0 1( )

0x

xotherwise

ϕ≤ <⎧

= ⎨⎩

1. The scaling function is orthogonal to its integer translates

2. The subspaces spanned by the scaling function at low scales are nested within those spanned at higher scales:

MRA RequirementsMRA Requirements

1 0 1 2V V V V V V−∞ − ∞⊂ ⊂ ⊂ ⊂ ⊂ ⊂ ⊂L L

Given a scaling function which satisfies the MRA requirements, one can define a wavelet wavelet functionfunction which, together with its integer translates and binary scalings, spans the difference between any two adjacent scaling subspaces and

Wavelet FunctionsWavelet Functions

( )xψ

jV 1jV +

Define the wavelet set

for all that spans the spacesWe write

and, if

{ } { }/ 2, ( ) 2 (2 )j j

j k x x kψ ψ= −

Wavelet FunctionsWavelet Functions

k∈ jW

{ }, ( )j j kk

W Span xψ=

( ) jf x W∈

,( ) ( )k j kk

f x xα ψ=∑

This implies that

for all appropriateWe can write

and also

OrthogonalityOrthogonality:: 1j j jV V W+ = ⊕1j j jV V W+ = ⊕

, ,( ), ( ) 0j k j lx xϕ ψ =, ,j k l∈

20 0 1( )L V W W= ⊕ ⊕ ⊕L

22 1 0 1 2( )L W W W W W− −= ⊕ ⊕ ⊕ ⊕ ⊕ ⊕L L

(no need for scaling functions, only wavelets!)

Example: Example: Haar Haar Wavelet Functions in Wavelet Functions in WW00 and and WW11

( ) ( ) ( )a df x f x f x= +

low frequencies

high frequencies

A function can be expressed as

Wavelet Series ExpansionsWavelet Series Expansions

0 0

0

,,( ) ( ( )) ( ) ( )j jj j kk k

kj j

f x c kx xk d ψϕ∞

== +∑ ∑ ∑

2( ) ( )f x L∈

approximation or scaling coefficientsapproximation or scaling coefficientsdetail or wavelet coefficientsdetail or wavelet coefficients

00 ,( ) ( ), ( )jj k xc k f x ϕ=,( ) ( ), ( )j kjd x xk f ψ=

0 00

21( ) j jj WV WL += ⊕ ⊕ ⊕L

ConsiderIf , the expansion coefficients are

Example: The Example: The Haar Haar Wavelet Series Wavelet Series Expansion of Expansion of yy==xx22

2 0 10x xy

otherwise

⎧ ≤ <= ⎨⎩

1 12 2

0 00 01 1

2 21 1

0,0

1,0

,

0

0 0

01,1

1 1(0) (0)3 4

2 3 2(0) (1)32

( )

( (

)

) )

(

32

c x xdx d x dx

xd x dx d x xx d

x ψ

ψ ψ

ϕ= = = = −

= = − = = −

∫ ∫

∫ ∫

0 0 1

1 0 0

2 1 1 0 0 1

0,0 1,0 10 0 ,1, ( )1 1 2 3 23 4

( ) ( ) (2 32

)3

V W WV V W

V V W V W W

x xx xy ψ ψ ψϕ

= ⊕

= ⊕ = ⊕ ⊕

⎡ ⎤⎡ ⎤= + − + − − +⎢ ⎥⎢ ⎥⎣ ⎦ ⎣ ⎦L

14243 1442443 144444244444314444244443

14444444444244444444443

0 0j =

Example: The Example: The Haar Haar Wavelet Series Wavelet Series Expansion of Expansion of yy==xx22

Let denote a discrete function Its DWT is defined as

Let and so that

The Discrete Wavelet Transform (DWT)The Discrete Wavelet Transform (DWT)( ), 0,1, , 1f x x M= −K

00 ,

,

1( , ) ( )

1( , ) (

(

) ( )

)x

j

k

xk

jW j k f xM

W j k f x xM

xϕ

ψ ψ

ϕ=

=

∑

∑ 0j j≥

0

00 ,, ( )1 1( )( ) ( , ) ( , ) j kk j j k

j kf x W j k W j kM M

xxϕ ψ ψϕ∞

== +∑ ∑ ∑

approximation approximation coefficientscoefficients

detail detail coefficientscoefficients

2JM =0oj =0,1, , 1,0,1, , 1,0,1, ,2 1j

x Mj Jk

= −⎧⎪ = −⎨⎪ = −⎩

K

K

K

Consider the discrete function

It isThe summations are performed over

and for andfor

Use the Haar scaling and wavelet functions

Example: Computing the DWTExample: Computing the DWT

(0) 1, (1) 4, (2) 3, (3) 0f f f f= = = − =24 2 2M J= = ⎯⎯→ =

0,1,2,3x = 0k = 0j =0,1k = 1j =

Example: Computing the DWTExample: Computing the DWT

[ ]

[ ]

3

03

03

0

0,0

1

0,

3

0

,

1,

0

0

1

1(0,0) ( ) 1 4 3 0 12

1(0,0) ( ) 1 4 3 ( ) 0 ( ) 42

1(1,0) ( ) 1

1( ) 1 1 1 12

1( ) 2 2 0 0

1( ) 1 1 1

4 ( ) 3 0 1.5 22

1(1,1) ( ) 1

2

1( ) 0 0 2

12

32

4 0 (2

x

x

x

x

x

x

W f x

W f x

W f x

W f x x

xϕ

ψ

ψ

ψ

ψ

ψ

ψ

ϕ=

=

=

=

= = ⋅ + ⋅ − ⋅ + ⋅ =

= = ⋅ + ⋅ − ⋅ + ⋅ =

⎡ ⎤= = ⋅ + ⋅ − ⋅ + ⋅ = −⎣ ⎦

= = ⋅ + ⋅ − ⋅ + ⋅

− −

−

−

∑

∑

∑

∑ ) 12 .5 2⎡ ⎤ = −⎣ ⎦

The DWT of the 4-sample function relative to the Haar wavelet and scaling functions thus is

The original function can be reconstructed as

for

Example: Computing the DWTExample: Computing the DWT

{ }1,4, 1.5 2, 1.5 2− −

0,0

1,0 1

0,

,1

0 ( )1( ) (0,0) (0,0)2

(1 (,0) (1,1 (

)

) )

(

)

f x W W

W W

x

x x

xϕ ψ

ψ ψ

ψϕ

ψ ψ

⎡= + +⎣

⎤+ ⎦0,1,2,3x =

In 2-D, one needs one scaling function

and three wavelets

is a 1-D scaling function and is its corresponding wavelet

Wavelet Transform in 2Wavelet Transform in 2--DD

( , ) ( ) ( )x y x yϕ ϕ ϕ=

( , ) ( ) ( )( , ) ( ) ( )( , ) ( ) ( )

H

V

D

x y x yx y x yx y x y

ψ ψ ϕψ ϕ ψψ ψ ψ

⎧ =⎪ =⎨⎪ =⎩

•detects horizontal details

•detects vertical details

•detects diagonal details

(.)ϕ(.)ψ

Define the scaled and translated basis functions

Then

22--D DWT: DefinitionD DWT: Definition

{ }

/ 2

/

,

2

,

, ,

2 (2 ,2 )

2 (2 ,2 ),( ,

( )

,

,

) ,ij m n

j j j

j i j

j m n

j

x m y n

x m yx n i H V D

x y

y

ϕϕ

ψψ

= − −

= − − =

{ }

0

1 1

00 0

1 1

0 0,

,

,

,

( ,

( ,

)

1( , , ) ( , )

1( , , ) ( , , ,

)

, )

M N

x yj m

M Ni

x

ij n

ym

nW j m n f x yMN

W j x ym n f

x y

x y i H V DMN

ϕ

ψ ψ

ϕ− −

= =

− −

= =

=

= =

∑ ∑

∑ ∑

0

0

0

,,

,

, ,

,

1( , ) ( ,

(

( , )

, )

, )

1 ( , , )

m n

i

i

j m

H V D j j m nm

n

ij n

f x y W j m nMN

W j m nM

x y

xN

y

ϕ

ψ ψ

ϕ

∞

= =

=

+

∑∑

∑ ∑ ∑∑

Filter bank implementation of 2Filter bank implementation of 2--D D waveletwavelet

analysis FBanalysis FB

synthesis FBsynthesis FB

resulting resulting decompositiondecomposition

( , )f x y

LL

LHHL

HH

HHHL

LHLL

Example: A ThreeExample: A Three--Scale FWTScale FWT

Analysis and Synthesis FiltersAnalysis and Synthesis Filters

scaling scaling functionfunction

wavelet wavelet functionfunction

analysis analysis filtersfilters

synthesis synthesis filtersfilters

“An Introduction to Wavelets,” by Amara Graps

Amara’s Wavelet Page (with many links to other resources) http://www.amara.com/current/wavelet.html

“Wavelets for Kids,” (A Tutorial Introduction), by B. Vidakovic and P. Mueller

Gilbert Strang’s tutorial papers from his MIT webpage

http://www-math.mit.edu/~gs/

Wavelets and Subband Coding, by Jelena Kovacevic and Martin Vetterli, Prentice Hall, 2000.

Want to Learn More About Wavelets?Want to Learn More About Wavelets?