Bayesian Experimental Design for Large Scale Signal

Bayesian Experimental Design forLarge Scale Signal Acquisition Optimization

Matthias Seeger

Laboratory for Probabilistic Machine LearningEcole Polytechnique Fédérale de Lausanne

http://lapmal.epfl.ch/

9/12/2013

Seeger (EPFL) Large Scale Bayesian Inference 9/12/2013 1 / 20

Motivation

Magnetic Resonance Imaging

⊕ Extremely versatile⊕ Noninvasive, no ionizing radiation Very expensive Long scan times: Major limiting factor


Motivation


Faster scans by undersampledreconstructionWhich fast designs give best images?


Motivation


Faster scans by undersampledreconstructionWhich fast designs give best images?


Motivation

Image Reconstruction

Ideal Image u

Reconstruction

Measurement

Design

y X u

Data y


Motivation

Sampling Optimization

Reconstruction

y X u


Bayesian Experimental Design


Posterior: Uncertainty inreconstructionExperimental design:Find poorly determineddirectionsSequential search withinterjacent partialmeasurements




Posterior: Uncertainty inreconstructionExperimental design:Find poorly determineddirectionsSequential search withinterjacent partialmeasurements



Maximizing Information Gain

Score design extension X ∗ by information gain:

I(X ∗) = I(y∗,u |y ) = H[P(u |y )]︸︷︷︸Before

−H[P(u |y∗,y )]︸︷︷︸After

Most work: Combinatorial aspectsAssume: I(X ∗) tractable to computeAssume: I(X ∗) cheap to compute (many X ∗)Simple greedy forward works well in practice . . .















So is it . . . ?



Challenges

I(X ∗) = H[P(u |y )]− H[P(u |y∗,y )

Assume: I(X ∗) tractable to compute.Only if P(u |y ) Gaussian . . .



Image Statistics

Whatever images are . . .

they are not Gaussian!



Challenges

I(X ∗) = H[P(u |y )]− H[P(u |y∗,y )

Assume: I(X ∗) tractable to compute?No: Needs approximate inferenceAssume: I(X ∗) cheap to compute (many X ∗).



Size Does Matter

1 Global covariancesScores I(X ∗) need full CovP [u |y ]

2 Massive scaleR131072 (just one slice).

3 Many timesPosterior after each design extension



Challenges

I(X ∗) = H[P(u |y )]− H[P(u |y∗,y )

Assume: I(X ∗) tractable to compute?No: Needs approximate inferenceAssume: I(X ∗) cheap to compute (many X ∗)?No: Needs new algorithms and high performance computing


Variational Bayesian Inference


Approximate inference for non-Gaussian modelsComputations driven by Gaussian inference




P(u |y ) =P(y |u)× P(u)

P(y )

Variational Inference ApproximationWrite intractable integration as optimizationRelax to tractable optimization problem




P(u |y ) =P(y |u)× P(u)

P(y )

Variational Relaxation: Bound the master function

− log P(y ) = − log∫

P(u ,y )du ≤ 12

minγ

minu∗

φ(u∗,γ)

Approximate posterior P(u |y ) by GaussianIntegration⇒ Convex optimization




P(u |y ) =P(y |u)× P(u)

P(y )

Variational Relaxation: Bound the master function

− log P(y ) = − log∫

P(u ,y )du ≤ 12

minγ

minu∗

φ(u∗,γ)

Approximate posterior P(u |y ) by GaussianIntegration⇒ Convex optimization



No Inference Without . . .



Double Loop Algorithm

Double loop algorithmInner loop optimization:Standard MAP EstimationOuter loop update:Gaussian Variances



Double Loop Algorithm

Double loop algorithmInner loop optimization:Standard MAP EstimationOuter loop update:Gaussian Variances



Tricks of the Trade

Monte Carlo Gaussian variances: Perturb&MAP Papandreou, Yuille, NIPS 2010

Inner loop: Fast first-order MAP solversWarmstarting variational optimization:Small changes after each (X ∗,y∗)



Tricks of the Trade

Monte Carlo Gaussian variances: Perturb&MAP Papandreou, Yuille, NIPS 2010

Inner loop: Fast first-order MAP solversWarmstarting variational optimization:Small changes after each (X ∗,y∗)


Experimental Results

Optimizing Cartesian MRI

Bayes Optim. VD Random Low Pass

Seeger et.al., MRM 63(1), 2010 Lustig, Donoho, Pauli, MRM 58(6), 2007 Common MRI practice


Experimental Results

Experimental Results: Test Set Errors

80 100 120 140 1601

2

3

4

5

6

7

Number phase encodes

L 2 rec

onst

ruct

ion

erro

r

axial short TE

80 100 120 140 160Number phase encodes

axial long TE

1

2

3

4

5

6

7

L 2 rec

onst

ruct

ion

erro

r

sagittal short TE

Low PassVD RandomBayes Optim

sagittal long TE


Outlook

Large Scale Bayesian Inference

Advanced Bayesian experimental designSignal acquisition optimization

Computer VisionHierarchically structured image priors Ko, Seeger, ICML 2012

Learning Image Models (fields of experts, . . . )Bayesian dictionary learningIntelligent user interfaces (Bayesian active learning)

Advanced variational inferenceSpeeding up expectation propagation Seeger, Nickisch, AISTATS 2011

Generic frameworkYou can do MAP estimation efficiently?You can do variational Bayesian inference!


Outlook

Large Scale Bayesian Inference

Advanced Bayesian experimental designSignal acquisition optimization

Computer VisionHierarchically structured image priors Ko, Seeger, ICML 2012

Learning Image Models (fields of experts, . . . )Bayesian dictionary learningIntelligent user interfaces (Bayesian active learning)

Advanced variational inferenceSpeeding up expectation propagation Seeger, Nickisch, AISTATS 2011

Generic frameworkYou can do MAP estimation efficiently?You can do variational Bayesian inference!


Outlook

People& Code

glm-ie: Toolbox by Hannes Nickisch

mloss.org/software/view/269/

Generalized sparse linear modelsMAP reconstruction and variational Bayesianinference (double loop algorithm forsuper-Gaussian bounding)Matlab 7.x, GNU Octave 3.2.x

Hannes Nickisch (now Philips Research, Hamburg)Rolf Pohmann, Bernhard Schölkopf (MPI Tübingen)Young Jun KoEmtiyaz Khan


Documents

Bayesian Experimental Design for Large Scale Signal