Sparse Approximation by Wavelet Frames and Applications

Bin Dong

Department of Mathematics

The University of Arizona

2012 International Workshop on

Signal Processing , Optimization, and ControlJune 30- July 3, 2012

USTC, Hefei, Anhui, China

OutlinesI. Wavelet Frame Based Models for Linear

Inverse Problems (Image Restoration)

II. Applications in CT Reconstruction

1-norm based models

Connections to variational model

0-norm based model

Comparisons: 1-norm v.s. 0-norm

Quick Intro of Conventional CT Reconstruction

CT Reconstruction with Radon Domain Inpainting

Tight Frames in Orthonormal basis

Riesz basis

Tight frame: Mercedes-Benz frame

Expansions:Unique

Not unique

Tight Frames General tight frame systems

Tight wavelet frames

Construction of tight frame: unitary extension principles [Ron and Shen, 1997]

• They are redundant systems satisfying Parseval’s identity

• Or equivalently

where and

Tight Frames Example:

Fast transforms

Lecture notes: [Dong and Shen, MRA-Based Wavelet Frames and Applications, IAS Lecture Notes Series,2011]

Decomposition

Reconstruction

Perfect Reconstruction

Redundancy

Image Restoration Model Image Restoration Problems

Challenges: large-scale & ill-posed

• Denoising, when is identity operator

• Deblurring, when is some blurring operator

• Inpainting, when is some restriction operator

• CT/MR Imaging, when is partial Radon/Fourier

transform

Frame Based Models Image restoration model:

Balanced model for image restoration [Chan, Chan, Shen and Shen, 2003], [Cai, Chan and Shen, 2008]

When , we have synthesis based model [Daubechies, Defrise and De Mol, 2004; Daubechies, Teschke and Vese, 2007]

When , we have analysis based model [Stark, Elad and Donoho, 2005; Cai, Osher and Shen, 2009]

Resembles Variational Models

Connections: Wavelet Transform and Differential Operators Nonlinear diffusion and iterative wavelet and wavelet

frame shrinkageo 2nd-order diffusion and Haar wavelet: [Mrazek,

Weickert and Steidl, 2003&2005]o High-order diffusion and tight wavelet frames in 1D:

[Jiang, 2011] Difference operators in wavelet frame transform:

True for general wavelet frames with various vanishing moments [Weickert et al., 2006; Shen and Xu, 2011]

Filters

Transform

Approximation

Connections: Analysis Based Model and Variational Model [Cai, Dong, Osher and Shen, Journal of the AMS,

2012]:

The connections give us

Leads to new applications of wavelet frames:

Converges

• Geometric interpretations of the wavelet frame transform (WFT)

• WFT provides flexible and good discretization for differential operators

• Different discretizations affect reconstruction results

• Good regularization should contain differential operators with varied

orders (e.g., total generalized variation [Bredies, Kunisch, and Pock,

2010])

Image segmentation: [Dong, Chien and Shen, 2010] Surface reconstruction from point clouds: [Dong and Shen, 2011]

For any differential operator when proper parameter is chosen.

Standard Discretization Piecewise Linear WFT

Frame Based Models: 0-Norm Nonconvex analysis based model [Zhang, Dong and

Lu, 2011]

Motivations:

Related work:

• Restricted isometry property (RIP) is not

satisfied for many applications

• Penalizing “norm” of frame coefficients

better balances sparsity and smoothnes

• “norm” with : [Blumensath and Davies,

2008&2009]

• quasi-norm with : [Chartrand,

2007&2008]

Fast Algorithm: 0-Norm Penalty decomposition (PD) method [Lu and Zhang, 2010]

Algorithm:

Change of variables

Quadratic penalty

Fast Algorithm: 0-Norm Step 1:

Subproblem 1a): quadratic

Subproblem 1b): hard-thresholding

Convergence Analysis [Zhang, Dong and Lu, 2011] :

Numerical Results Comparisons (Deblurring)

Balanced

Analysis

0-Norm

PFBS/FPC: [Combettes and Wajs, 2006] /[Hale, Yin and Zhang, 2010]

Split Bregman: [Goldstein and Osher, 2008] & [Cai, Osher and Shen, 2009]PD Method: [Zhang, Dong and Lu, 2011]

Numerical Results Comparisons

Portrait

Couple

Balanced Analysis

Faster Algorithm: 0-Norm Start with some fast optimization method for nonsmooth

and convex optimizations: doubly augmented Lagrangian (DAL) method [Rockafellar, 1976].

Given the problem:

The DAL method:

where

We solve the joint optimization problem of the DAL method using an

inexact alternative optimization scheme

Faster Algorithm: 0-Norm Start with some fast optimization method for nonsmooth

and convex optimizations: doubly augmented Lagrangian (DAL) method [Rockafellar, 1976].

The inexact DAL method:

Given the problem:

The DAL method:

where

Hard thresholding

Faster Algorithm: 0-Norm However, the inexact DAL method does not seem to

converge!! Nonetheless, the sequence oscillates and is bounded.

The mean doubly augmented Lagrangian method (MDAL) [Dong and Zhang, 2011] solve the convergence issue by using arithmetic means of the solution sequence as outputs instead:MDAL:

Comparisons: Deblurring Comparisons of best PSNR values v.s. various noise level

Comparisons: Deblurring Comparisons of computation time v.s. various noise level

Comparisons: Deblurring What makes “lena” so special?

Decay of the magnitudes of the wavelet frame coefficients is very fast, which is what 0-norm prefers.

Similar observation was made earlier by [Wang and Yin, 2010].

1-norm 0-norm: PD 0-norm: MDAL

APPLICATIONS IN CT RECONSTRUCTION

With the Center for Advanced Radiotherapy and Technology (CART), UCSD

Cone Beam CT

3D Cone Beam CT

xy

z

u

v

0

g(u)

f(x)

n0

xS

x

u*

Discrete

3D Cone Beam CT

=

Animation created by Dr. Xun Jia

Goal: solve

Difficulties:

Related work:

Cone Beam CT Image Reconstruction

Unknown Image

Projected Image

• In order to reduce dose, the system is highly underdetermined. Hence the solution is not unique.•Projected image is noisy.

Total Variation (TV): [Sidkey, Kao and Pan 2006], [Sidkey

and Pan, 2008], [Cho et al. 2009], [Jia et al. 2010];

EM-TV: [Yan et al. 2011]; [Chen et al. 2011];

Wavelet Frames: [Jia, Dong, Lou and Jiang, 2011];

Dynamical CT/4D CT: [Chen, Tang and Leng, 2008],

[Jia et al. 2010], [Tian et al., 2011]; [Gao et al. 2011];

CT Image Reconstruction with Radon Domain Inpainting Idea: start with

Benefits:

Instead of solving

We find both and such that:

• is close to but with better quality

•

• Prior knowledge of them should be used

• Safely increase imaging dose• Utilizing prior knowledge we have for both CT images and the projected images

CT Image Reconstruction with Radon Domain Inpainting Model [Dong, Li and Shen, 2011]

Algorithm: alternative optimization & split Bregman.

where

• p=1, anisotropic• p=2, isotropic

CT Image Reconstruction with Radon Domain Inpainting Algorithm [Dong, Li and Shen, 2011]: block

coordinate descend method [Tseng, 2001]

Convergence Analysis

Problem:

Algorithm:

Note: If each subproblem is solved exactly, then the convergence analysis was given by [Tseng, 2001], even for nonconvex problems.

CT Image Reconstruction with Radon Domain Inpainting Results: N denoting number of projections

N=15 N=20

CT Image Reconstruction with Radon Domain Inpainting Results: N denoting number of projections

N=15

N=20

W/O Inpainting With Inpainting

Thank You

Collaborators:

Mathematics Stanley Osher, UCLA Zuowei Shen, NUS Jia Li, NUS Jianfeng Cai, University of Iowa Yifei Lou, UCLA/UCSD Yong Zhang, Simon Fraser University, Canada Zhaosong Lu, Simon Fraser University, Canada

Medical School Steve B. Jiang, Radiation Oncology, UCSD Xun Jia, Radiation Oncology, UCSD Aichi Chien, Radiology, UCLA