Old and New Algorithms for BlindDeconvolution

Yair Weiss

Hebrew University of Jerusalemjoint work with Anat Levin, WIS

(thanks to Meir Feder, EE, TAU)

Blind Deblurring

y = x * k

Much Recent Progress

(Fergus et al. 06, Cho et al. 07, Jia 07, Joshi et al. 08, Whyte et al.10, Shan et al. 07, Harmeling et al. 10, Sroubek and Milanfar 11 · · · )

MAP using sparse derivative prior

−100 −50 0 50 10010

0

105

dx

log

coun

ts

log P(x , k |y) = C +∑

i

|xi |α + λ‖k ∗ x − y‖2

• MAPxk : (x∗, k∗) = arg max P(x , k |y). Guaranteed to failwith global optimization. Often works well in practice.

• MAPk : (k∗) = arg maxk∫

x P(x , k |y). Guaranteed tosucceed if images sampled from prior. Can be difficult tooptimize.

(Levin et al. 09)

This talk:

• Old algorithms for blind deconvolution from communicationsystems. 6= MAP. Rigorous correctness proofs.

• Can explain success of some MAPx ,k algorithms.

Blind Deconvolution in Communication

• x(t) the transmitted signal.• y(t) =

∑τ kτx(t − τ) received signal.

• y=x * k

Blur makes signals Gaussians (Central Limit Theorem)Orig. Signal (IID) Histogram (κ=1)

−1 −0.5 0 0.5 10

100

200

300

400

500

600

Blurred Histogram (κ = 2.683)

0 1 2 3 4 50

10

20

30

40

50

κ(y) =1N

∑i

y4i , yi = yi/std(y)

(Shalvi and Weinstein 1990)

Blur makes signals Gaussians (Central Limit Theorem)

Orig. Signal (IID) Histogram (κ=26.43)

−4 −2 0 2 40

100

200

300

400

Blurred Histogram (κ = 5.45)

−4 −2 0 2 40

20

40

60

80

100

Simple Blind Deconvolution Algorithm (Shalvi andWeinstein 1980)

Assume xi IID. y = x ∗ k . Solve for the “inverse filter” e suchthat e ∗ y = x .

• xi sub-Gaussian (κ < 3):

e∗ = arg mineκ(e ∗ y)

(Godard 1970)• xi super-Gaussian (κ > 3):

e∗ = arg maxe

κ(e ∗ y)

Claim: Guaranteed to find the correct e: e∗ ∗ y = x . No localminima.

Simple Proof

Assume xi IID. xi super-Gaussian (κ > 3): y = x ∗ k .

e∗ = arg maxe

κ(e ∗ y)

Proof: x = e ∗ y = e ∗ k ∗ x = (e ∗ k) ∗ x⇒ Unless e ∗ k = δ, x is more Gaussian than x so κ(x) < κ(x)

Proof by pictures:Orig. Signal (IID) Histogram (κ=26.43)

−4 −2 0 2 40

100

200

300

400

Blurred Histogram (κ = 5.45)

−4 −2 0 2 40

20

40

60

80

100

deblurred (wrong e) Histogram (κ = 14.14)

−3 −2 −1 0 1 2 30

20

40

60

80

100

120

140

160

180

Blind Deconvolution by Maximizing non Gaussianity

Assume xi IID. xi super-Gaussian. (κ > 3): y = x ∗ k .

e∗ = arg maxe

κ(e ∗ y)

• Universal proof of correcteness (don’t need to know Pr(xi),just IID and sub/super Gaussianity).

• Proofs of global convergence of iterative algorithms.• Used in microwave radio transmissions (1980s), cable

set-top boxes (mid 1990s) and wireless communication(late 1990s-today) (Johnson et al. 1998)

• Can we use this for blind image deblurring?

Blur makes derivatives more Gaussian

Sharp image dx Histogram (κ=20.02)

−100 −50 0 50 10010

0

105

Blurred image dx Histogram (κ = 17.99)

−100 −50 0 50 10010

0

105

blurredsharp

New class of image blind deblurring alglrithms

y : blurred (deriv) image Histogram (κ = 17.99)

−40 −20 0 20 4010

0

105

x sharp (deriv) image Histogram (κ=20.02)

−100 −50 0 50 10010

0

105

Find (x∗, k∗) such that:• y = x∗ ∗ k∗

• x∗ as non Gaussian as possible.If derivatives are IID, this algorithm provably finds the correctblur kernel.

Measuring non Gaussianity using normalizedmoments

1N

∑i

xαi , xi = xi/std(x)

−5 0 5−2.5

−2

−1.5

−1

−0.5

0

x

log

prob

−5 0 5−5

−4

−3

−2

−1

0

x

log

prob

−5 0 5−25

−20

−15

−10

−5

0

x

log

prob

∑i x4

i = 25.73∑

i x4i = 5.8

∑i x4

i = 3∑i x1/2

i = 0.60∑

i x1/2i = 0.74

∑i x1/2

i = 0.81

A new(?) algorithm

y : Blurred image Histogram (κ = 17.99)

−40 −20 0 20 4010

0

105

(x∗, k∗) = arg minx ,k

∑i

|xi |α + λ‖k ∗ x − y‖2

xi = xi/std(x)

Guaranteed to succeed.

MAPx ,k algorithm

y : Blurred image Histogram (κ = 17.99)

−40 −20 0 20 4010

0

105

(x∗, k∗) = arg minx ,k

∑i

|xi |α + λ‖k ∗ x − y‖2

Guaranteed to fail.

Normalized Sparsity (Krishnan et al. 11)

minx ,k

λ‖x ∗ k − y‖2 +‖x‖1‖x‖2

+ Ψ‖k‖1

Can be seen as special case of “new” algorithm.

MAPxk with bilateral filtering (Cho and Lee 09, Hirschet al. 11)

x ← BilateralFilter(x)

k ← arg mink‖y − k ∗ x‖

x ← arg minxλ‖y − k ∗ x‖2 + sparsity(x)

Can be seen as approximating special case of “new” algorithm.

A new(?) algorithm

y : Blurred image Histogram (κ = 17.99)

−40 −20 0 20 4010

0

105

(x∗, k∗) = arg minx ,k

∑i

|xi |α + λ‖k ∗ x − y‖2

xi = xi/std(x)

Guaranteed to succeed.

So what does this buy us?

⇔• Understanding recent algorithms. Proof of success.• Improving recent algorithms. Filters with better

independence properties. Iterative algorithms with globalconvergence.

Conclusions

• MAP algorithms for blind image deconvolution. May workeven when not supposed to.

• Old algorithms for blind deconvolution in communications.Universal guarantees, global convergence, used in millionsof devices.

• Can help us understand and improve image deblurringalgorithms.