24
Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

Embed Size (px)

Citation preview

Page 1: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

Multilevel approach for signal restoration problems with Toeplitz matrices

Malena Español, Tufts UniversityMisha Kilmer, Tufts University

Page 2: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

2

Outline Background Multilevel Method Algorithm Implementation Numerical Example Conclusion and Future Work

Page 3: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

3

Ill-posed Problem

A problem is ill-posed if its solution is not unique, or its solution does not depend continuously

on the data

)()( ,

kindfirst ofequation integral Fredholm :Example

sgdttft)K(s

Page 4: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

4

Discrete Ill-Posed problem

holds condition Picard Discrete

yoscillator more become ectorssingular v

noise (white)unknown is

gap without aluessingular v Decaying

:Properties

e

matrix dconditione-ill large, a is where

,

model theand ,given , Find

nm

truetrue

true

RA

ebbAx

bAx

Page 5: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

5

Need for regularization

)(

1

)(

1ii

)(

1i

)(

1i

)(

)(

:are solutions squareleast Then the

. of SVD theLet

Arank

i

Arank

itrue

i

T

i

trueT

Arank

i i

trueT

true

Arank

i i

trueT

truetrue

errorxeubu

ebuebAx

bubAx

AVUA

ii

i

i

Page 6: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

6

Regularization

small is if 0

large is if 1

where

solution dRegularize

1i

i

ii

n

i i

T

ireg

bux i

Page 7: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

7

Regularization methods

CGLS.or LSQR, eg.

methods, Krylov of iterationsk :Methods Iterative

min

:tionRegulariza Tikhonov

: (TSVD) SVD Truncated

2

2

22

2

1i22

2

1i

LxbAx

bux

bux

x

n

i i

T

i

iTik

k

i i

T

TSVD

i

i

Page 8: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

8

Two-Level Method

cii

ic

i

iiii

iiii

iii

iiii

iii

iiii

iii

xxx

rxA

xAbr

xPxx

rxA

PARA

rRr

xAbr

bxA

Solve""

Solve""

Solve""

1

111

1

1

Pre-Smoothing

Post-Smoothing

Coarse-Grid Correction

Restriction

Prolongation

Page 9: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

9

Multilevel Method

If End

Solve

,

Solve

Else

Solve

gridcoarsest If

function

1

111

11

cii

ic

i

iiii

iiii

iii

iiiiiii

iiii

iii

iii

iii

xxx

rxA

xAbr

xPxx

),bMGM(Ax

PARArRr

xAbr

bxA

bxA

),bMGM(Ax

Page 10: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

10

Restriction and Prolongation Operators

11000000

00110000

00001100

00000011

11000000

00110000

00001100

00000011

2

2TW

Haar wavelet transform

W2W1

Page 11: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

11

Wavelet Domain

2

1

2

1

43

21

ˆ

ˆ

ˆ

ˆˆˆ

ˆˆ

.ˆ and ˆ ,ˆ where

ˆˆˆ

becomes domain In wavelet

b

b

x

x

AA

AA

bWbxWxAWWA

bxA

bAx

TTT

Page 12: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

12

Coarse-scale equation

x

1x̂111

22111

2

1

2

1

43

21

ˆˆˆ

ˆˆˆˆˆ

ˆ

ˆ

ˆ

ˆˆˆ

ˆˆ

bxA

xAbxA

b

bx

x

AA

AA

Page 13: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

13

.ˆˆˆ

Then,

ˆˆ

ˆˆˆ

ˆ

ˆˆ

ˆ

ˆˆ

ˆˆˆˆˆˆˆˆˆ

111

)ˆ(

1i

2211

221122111

1

bxA

xAueubu

xAebxAbxA

Arank

i i

T

i

T

i

trueT

true

iii

Coarse-scale equation

Page 14: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

14

p

p

p

xxLbxA

bAx

bxA

1

2

2111ˆ

111

111

ˆˆˆˆmin

3)or 2 ,ˆ,ˆLSQR(ˆ

solve we

ˆˆˆ

ofsolution dregularize aget To

1

Coarse-Grid Correction

Pre-smoothing

Pre-smoothing

Coarse-Grid Correction

Page 15: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

15

1

3

1

2

12

4

2 ˆˆ

ˆ

ˆ

ˆˆ

ˆ

ˆx

A

A

b

bx

A

A

Post-Smoothing

p

pprep

newx

xWxWxLrxA

A)ˆˆ(ˆ

ˆ

ˆmin 2211

2

2

2

4

2

ˆ2

newr

Page 16: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

16

Multilevel Method

If End

ˆ

),,,(Newtonˆ

ˆ

ˆ

;

If End

,3),(

gridfinest If

Else

)(Newtonor solvedirect

gridcoarsest If

function

122

21

2

111

1111

111

11

iinew

i

inew

inew

Tii

inew

iiinew

iiinew

iii

iTii

Ti

iiii

iii

iii

iii

xWxx

xLWrWWAx

xAbr

xWxx

),bMGM(Ax

WAWArWb

xAbr

bALSQRx

non

,bAx

),bMGM(Ax

Page 17: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

17

Toeplitz Matrices

),,,,,(: 1101)1(

0321

)3(012

)2(101

)1(210

mm

mmm

m

m

m

ttttttvectorToeplitz

tttt

tttt

tttt

tttt

A

Page 18: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

18

Toeplitz structure inheritance

A

43

21

ˆˆ

ˆˆˆ

AA

AAA

.ˆeach ofvector -Toeplitz theknow weMoreover,

Toeplitz. are ˆ and ˆ,ˆ,ˆ then Toeplitz, is If

:Theorem

4321

iA

AAAAA

Page 19: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

19

Pre-smoothing

productvector -matrixFast structure Toeplitz

LSQR. of iterations 3or 2 applyingby

ˆˆˆ

solve We

111

bxA

Page 20: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

20

Coarse-Grid Correction

.ˆon depend and where,ˆ

ˆmin

Method sNewton'by ˆˆˆˆmin

or

solvedirect aby ˆˆˆ

solve We

11

2

21

11

1

2

2111ˆ

111

1

xrDxDL

rd

DL

A

xLbxA

bxA

d

p

p

p

x

Page 21: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

21

Post-Smoothing

.ˆon depend ~ and ~, where,~

ˆ

min

Method sNewton'by

)ˆˆ(ˆˆ

ˆmin

solve We

221

2

2

2

1

2

4

2

2211

2

2

2

4

2

ˆ2

xrrDrcD

rd

DLW

A

A

xWxWxLrxA

A

d

p

pprep

newx

Page 22: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

22

Numerical Example

1.1 operator, derivative

)01.0,0( with

solution edgy :

matrix symmetric Toeplitz, Gaussian, :

pL

bNeeAxb

x

A

true

true

Page 23: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

23

Numerical Example

truex b

MGMxLSQRx

Page 24: Multilevel approach for signal restoration problems with Toeplitz matrices Malena Español, Tufts University Misha Kilmer, Tufts University

10th Copper Mountain Conference on Iterative Methods

24

Conclusions and Future work

General Cases – Non Structured Matrices

Parameter Selection Adaptive p-norm Extension to 2D, 3D