Department of Mathematics and Statistics University of ...€¦ · Eero Saksman,UniversityofHelsinki. Outline Sparse sampling and tomography Total variation regularization Discretization-invariance

Sparse tomography

Samuli Siltanen

Department of Mathematics and StatisticsUniversity of Helsinki, Finland

Minisymposium:Fourier analytic methods in tomographic image reconstruction

Applied Inverse Problems ConferenceDaejeon, Korea, July 1, 2013

Finland

•PPPP

•��

��

•��

••��

��•H

HHHHH

H•

http://wiki.helsinki.fi/display/inverse/Home

This is a joint work with

Keijo Hämäläinen, University of Helsinki

Aki Kallonen, University of Helsinki

Ville Kolehmainen, University of Eastern Finland

Matti Lassas, University of Helsinki

Esa Niemi, University of Helsinki

Kati Niinimäki, University of Eastern Finland

Eero Saksman, University of Helsinki

Outline

Sparse sampling and tomography

Total variation regularization

Discretization-invariance

Besov space regularization

Parameter choice: the S-curve method

Let us study a simple two-dimensional example oftomographic imaging

4 4 5

1 3 4

1 0 2

Tomography is based on measuring densities ofmatter using X-ray attenuation data

13 (=4+4+5)4 4 5

1 3 4

1 0 2

X-ray source• -

Detector

A projection image is produced by parallel X-raysand several detector pixels (here three pixels)

13 (=4+4+5)

8 (=1+3+4)

3 (=1+0+2)

4 4 5

1 3 4

1 0 2

• -

• -

• -

Detector

For tomographic imaging it is essential to recordprojection images from different directions

4 4 5

1 3 4

1 0 2

6 7 11

•

?

•

?

•

?

The length of X-rays traveling inside each pixel isimportant, thus here the square roots

4 4 5

1 3 4

1 0 2

√2

√2

9 √2

8 √2

5 √2

•••••

@@

@@@

@@@

@@@@I

@@

@@

@@

@@@

@@@I

@@

@@@

@@@

@@

@@I

@@@

@@@

@@@

@@@I

@@

@@@

@@@

@@

@@I

��

��

��

The direct problem of tomography is to find theprojection images from known tissue

4 4 5

1 3 4

1 0 2

√2

√2

9 √2

8 √2

5 √2

6 7 11

13

8

3

The inverse problem of tomography is toreconstruct the interior from X-ray data

? ? ?

? ? ?

? ? ?

√2

√2

9 √2

8 √2

5 √2

6 7 11

13

8

3

We write the reconstruction problemin matrix form and assume Gaussian noise

f1 f4 f7

f2 f5 f8

f3 f6 f9@

@@

@@

@@@

@@@

@@

@@

@@@

@@@@

@@

@@@

@@@

@@@

Our measurement model is m = Af + ε with independently distributedGaussian noise (white noise) with standard deviation σ > 0.

f =

f1f2f3f4f5f6f7f8f9

, m =

m1m2m3m4m5m6

,

m = Af

m1

m2

m3

m4

m5

m6

Outline






We reconstruct the unknown pixel values usingtotal variation regularization and non-negativity

We consider the minimizer of the anisotropic TV functional

‖Af −m‖22 + α {‖LHf ‖1 + ‖LVf ‖1}

where LH and LH are horizontal and vertical first-order differencematrices. [Rudin, Osher and Fatemi 1992]

Primal-dual algorithms Chambolle, Chan, Chen, Esser, Golub, Mulet,Nesterov, ZhangThresholding Candès, Chambolle, Chaux, Combettes, Daubechies,Defrise, DeMol, Donoho, Pesquet, Starck, Teschke, Vese, WajsBregman iteration Cai, Burger, Darbon, Dong, Goldfarb, Mao, Osher,Shen, Xu, Yin, ZhangSplitting approaches Chan, Esser, Fornasier, Goldstein, Langer,Osher, Schönlieb, Setzer, WajsNonlocal TV Bertozzi, Bresson, Burger, Chan, Lou, Osher, Zhang

We use quadratic programming (QP) to computethe non-negative minimizer of the TV functional

The minimizer of the functional

argminf ∈Rn+

{‖Af −m‖22 + α‖LHf ‖1 + α‖LVf ‖1

}can be transformed into the standard form

argminz∈R5n

{12zTQz + cT z

}, z ≥ 0, Ez = b,

where Q is symmetric and E implements equality constraints.

Large-scale primal-dual interior point QP method was developed inKolehmainen, Lassas, Niinimäki & S (2012) andHämäläinen, Kallonen, Kolehmainen, Lassas, Niinimäki & S (2013).

Reduction to argminz∈R5n

{12z

TQz + cT z}

Denote horizontal and vertical differences by

LHf = u+H − u−H and LVf = u+V − u−V ,

where u±H , u±V ≥ 0. TV minimization is now

argminf ∈Rn+

{f TATAf − 2f TATm + α1T (u+H + u−H + u+V + u−V )

},

where 1 ∈ Rn is vector of all ones. Further, we denote

z =

f

u+Hu−Hu+Vu−V

, Q =

1σ2

ATA 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 00 0 0 0 0

, c =−2ATmα1α1α1α1

.

We collected X-ray projection data of a walnutfrom 1200 directions

The data was collected by Keijo Hämäläinen andAki Kallonen at University of Helsinki.

This is the reconstruction using all 1200projections and filtered back-projection

When only few projection angles are available,TV regularization performs better than FBP

23 angles 15 angles 10 angles

FBP

TV

These images were computed by Kati Niinimäki.

Outline






X-ray tomography: Continuum model

X-ray tomography: Practical measurement model

m ∈ Rk

k = 4

X-ray tomography: Computational model

m = Afm ∈ Rk

f ∈ Rn

k = 4n = 48

The numbers k and n are independent

m = Afm ∈ Rk

f ∈ Rn

k = 8n = 48


m = Afm ∈ Rk

f ∈ Rn

k = 8n = 156


m = Afm ∈ Rk

f ∈ Rn

k = 8n = 440


m = Afm ∈ Rk

f ∈ Rn

k = 16n = 440


m = Afm ∈ Rk

f ∈ Rn

k = 24n = 440

Bayesian inversion using total variation priordoes not have a well-defined continuous limit

Theorem (Lassas and S 2004)Total variation prior is not discretization-invariant.

Sketch of proof: Apply a variant of the central limit theorem tothe independent, identically distributed random consecutivedifferences.

New numerical experiments are reported inKolehmainen, Lassas, Niinimäki and S (2012) andLucka (2012).

Bayesian inversion using Besov space priorshas a well-defined continuous limit

Theorem (Lassas, Saksman and S 2009)Besov space priors are discretization-invariant.

Sketch of proof: Construction of well-defined Bayesian inversiontheory in infinite-dimensional Besov spaces that allow wavelet bases.Discretizations are achieved by truncating the wavelet expansion.

Numerical experiments are reported inKolehmainen, Lassas, Niinimäki and S (2012).

Deterministic Besov space regularization was first introduced inDaubechies, Defrise and De Mol (2004); this applies toBayesian MAP estimates.

Outline






The wavelet transform divides an image into threetypes of details at different scales

This is how the wavelet decomposition is definedfor two-dimensional periodic images

We can represent periodic functions by wavelet expansion

f (x , y) =2J0−1∑k1=0

2J0−1∑k2=0

cJ0~k φJ0,~k(x , y)+J−1∑j=J0

3∑`=1

2j−1∑k1=0

2j−1∑k2=0

wj~k` ψ`j ,~k(x , y),

where ~k = (k1, k2) and the coefficients cJ0~k and wj~k` are defined by

cJ0~k = 〈f , φJ0~k〉 =∫T2

f (x , y)φj~k(x , y)dxdy ,

wj~k` = 〈f , ψ`j~k〉 =

∫T2

f (x , y)ψ`j~k(x , y)dxdy .

Besov space norm can be definedin terms of the wavelet coefficients

A function f belongs to Bspq(T2) if and only if the followingexpression is finite:

2J0−1∑k1=0

2J0−1∑k2=0

|cJ0~k |p

1p+ ∞∑

j=J0

2jq(s+1−2p )

3∑`=1

2j−1∑k1=0

2j−1∑k2=0

|wj~k`|p

qp

1q

We focus on the case p = 1 = q and s = 1:

‖f ‖B111(T2) =∑~k

|cJ0~k |+∑j ,`,~k

|wj~k`|.

We promote sparsity by minimizinga sum of `2 and `1 norms

In the case of Besov space penalty we minimize

f MAP = arg minf ∈Rn+

{1

2σ2‖Af −m‖2`2 + α‖f ‖B111(T2)

},

where ψj are the wavelet basis functions.

The above kind of minimization task has been studied inChambolle, DeVore, Lee & Lucier 1998,Daubechies, Defrise & De Mol 2004,Candès, Romberg & Tao 2006,Klann, Maaß & Ramlau 2006,Grasmair, Haltmeier & Scherzer 2008,Kolehmainen, Lassas, Niinimäki & S 2013.

We use constrained quadratic programming for the minimization.

History of multiresolution tomography

1994 Olson and DeStefano1994 Sahiner and Yagle1995 Delaney and Bresler1996 Berenstein and Walnut1996 Bhatia, Karl and Willsky1997 Rashid-Farrokhi, Liu,Berenstein and Walnut1997 Zhao, Welland and Wang1998 Louis, Maaß and Rieder1999 Candès and Donoho1999 Madych2000 Smith and Adhami2000 Bonnet, Peyrin, Turjmanand Prost

2002 Das and Sastry2002 Frese, Bouman and Sauer2004 Soleski and Walter2004 Zhong, Ning and Conover2006 Sastry and Das2006 Soleski and Walter2006 Lee and Lucier2006 Rantala, Vänskä, Järvenpää,Kalke, Lassas, Moberg and S2007 Niinimäki, S and Kolehmainen2008 Soussen and Idier2009 Vänskä, Lassas and S2011 Klann, Ramlau and Reichel2011 Terzija and McCann2011 Frikel

Besov regularized sparse-data reconstructionscompared to filtered back-projection

Outline






We took photographs of walnuts cut in half

These photos are used for estimating the expected number of nonzerowavelet coefficients in a two-dimensional tomographic reconstruction.Special thanks go to Esa Niemi for his careful job in sawing the walnuts.

The S-curve method determines a regularizationparameter value giving the right sparsity level

��

XXXX

�

Kolehmainen, Lassas, Niinimäki & S 2012

Num

berof

nonzerowavelet

coeffi

cients

Regularization parameter α

Something needs to be done to deal with theerratic behaviour of the “raw” S-curve

Reconstruction from 30 projections usinginterpolated S-curve

α = 0.019

Interpolated S-curve method for TV

These unpublished images were computed by Kati Niinimäki.

All Matlab codes freelyavailable on a website!

Part I: Linear Inverse Problems1 Introduction2 Naïve reconstructions and inverse crimes3 Ill-Posedness in Inverse Problems4 Truncated singular value decomposition5 Tikhonov regularization6 Total variation regularization7 Besov space regularization using wavelets8 Discretization-invariance9 Practical X-ray tomography with limited data10 Projects

Part II: Nonlinear Inverse Problems11 Nonlinear inversion12 Electrical impedance tomography13 Simulation of noisy EIT data14 Complex geometrical optics solutions15 A regularized D-bar method for direct EIT16 Other direct solution methods for EIT17 Projects

Inverse Days, Dec 11–13, 2013, Inari, Finlandhttp://inverse-problems.org/id2013/

Organizers:Maarten de HoopMatti LassasMarkku LehtinenLassi RoininenS. S.Gunther Uhlmann

��

��

��

•

Thank you for your attention!

Preprints available at www.siltanen-research.net.

Sparse sampling and tomographyTotal variation regularizationDiscretization-invarianceBesov space regularizationParameter choice: the S-curve method

Documents

Department of Mathematics and Statistics University of ...€¦ · Eero Saksman,UniversityofHelsinki. Outline Sparse sampling and tomography Total variation regularization Discretization-invariance