Super-resolution, Extremal Functions and the Condition ...people.csail.mit.edu/moitra/docs/Beurling2.pdfSuper-resolution, Extremal Functions and the Condition Number of Vandermonde

Super-resolution, Extremal Functions and the Condition Number of Vandermonde Matrices

Ankur Moitra

Massachusetts Institute of Technology

Limits to Resolution

Lord Rayleigh (1842-1919)

Ernst Abbe (1840-1905)

λ d = 2n sinθ

numerical aperture

Limits to Resolution

Lord Rayleigh (1842-1919)

Ernst Abbe (1840-1905)

λ d = 2n sinθ

numerical aperture

In microscopy, it is difficult to observe sub-wavelength structures (Rayleigh Criterion, Abbe Limit, …)

Many devices are inherently low-pass


Super-resolution: Can we recover fine-grained structure from coarse-grained measurements?



Applications in medical imaging, microscopy, astronomy, radar detection, geophysics, …



Applications in medical imaging, microscopy, astronomy, radar detection, geophysics, …

Super-resolution Cameras 2014 Nobel Prize in Chemistry!

Eric Betzig, Stefan Hell, William Moerner

A Mathematical Framework [Donoho, ‘91]:

Super-position of k spikes, each fj in [0,1):

f1 f2 f3 f4



x(t) = uj δf (t) j j = 1

k

coefficient delta function at fj



x(t) = uj δf (t) j j = 1

k

Measurement at frequency ω:

f1 f2 f3 f4

ei2πωt



x(t) = uj δf (t) j j = 1

k


vω = 0

1

ei2πωt x(t) dt



x(t) = uj δf (t) j j = 1

k


uj ei2πf ω j

j = 1

k

vω =



x(t) = uj δf (t) j j = 1

k

Measurement at frequency ω, |ω| ≤ m

uj ei2πf ω j

j = 1

k

vω =

cut-off frequency



x(t) = uj δf (t) j j = 1

k

Measurement at frequency ω, |ω| ≤ m

uj ei2πf ω + ηω j

j = 1

k

vω =

cut-off frequency

noise

When can we recover the coeffs (uj’s) and locations (fj’s) from low frequency measurements?


Proposition 1: When there is no noise (ηω=0), there is a polynomial time algorithm to recover the uj’s and fj’s exactly with m = 2k +1 – i.e. measurements at

ω = -k, -k+1, …, k-1, k

[Prony (1795), Pisarenko (1973), Matrix Pencil (1990),…]�


Proposition 1: When there is no noise (ηω=0), there is a polynomial time algorithm to recover the uj’s and fj’s exactly with m = 2k +1 – i.e. measurements at

ω = -k, -k+1, …, k-1, k

[Prony (1795), Pisarenko (1973), Matrix Pencil (1990),…]�

What is possible in the noise-free vs. the noisy setting will turn out to be fundamentally different…

What if there is noise?

Under what conditions is there an estimator

and

which converges at a polynomial-rate (in |ηω|)?


uj uj fj fj

And is there an algorithm?


Under what conditions is there an estimator

and

which converges at a polynomial-rate (in |ηω|)?

uj uj fj fj

Theorem: There is a polynomial time algorithm for noisy super-resolution if m > 1/Δ+1

separation condition



…where dw is the “wrap-around” distance: 0

1/8 7/8

i.e. dw(1/8,7/8) = 1/4



…where dw is the “wrap-around” distance: 0

1/8 7/8

i.e. dw(1/8,7/8) = 1/4 and Δ = mini≠jdw(fi,fj)

…where dw is the “wrap-around” distance:

Theorem: There is a polynomial time algorithm to recover estimates where

min matchings σ

max j

fσ(j) uσ(j) - fj - uj + ≤ ε provided |ηω| ≤ poly(ε, 1/m, 1/k), and m > 1/Δ + 1

0 1/8 7/8

i.e. dw(1/8,7/8) = 1/4 and Δ = mini≠jdw(fi,fj)

Theorem: For any m ≤ (1-ε)/Δ and k, there is a pair of Δ-separated signals x and x where

uj ei2πf ω j

j = 1

k

uj ei2πf ω j

j = 1

k

- ≤ e-εk

for any |ω| ≤ m

Theorem: For any m ≤ (1-ε)/Δ and k, there is a pair of Δ-separated signals x and x where

uj ei2πf ω j

j = 1

k

uj ei2πf ω j

j = 1

k

- ≤ e-εk

for any |ω| ≤ m

ei2πωt

[Donoho, ’91]: Asymptotic bounds for m = 1/Δ, on a grid


[Candes, Fernandez-Granda, ’12]: Convex program for m ≥ 2/Δ, no noise



[Fernandez-Granda, ’13]: Convex program for m ≥ 2/Δ, with noise



[Fernandez-Granda, ’13]: Convex program for m ≥ 2/Δ, with noise

[Liao, Fannjiang, ’14]: (concurrent) Algorithm for m = (1+C(Δ))/Δ, with noise

The Noise-free Case

Vandermonde Matrices

1 α1 α1 2

…

α1 m-1

1 α2 α2 2

… α2 m-1

… 1 αk αk 2

…

αk m-1 …

= αj = ei2πf j def

Vm k


1 α1 α1 2

…

α1 m-1

1 α2 α2 2

… α2 m-1

… 1 αk αk 2

…

αk m-1 …


Vm k

This matrix plays a key role in many exact inverse problems


1 α1 α1 2

…

α1 m-1

1 α2 α2 2

… α2 m-1

… 1 αk αk 2

…

αk m-1 …


Vm k

This matrix plays a key role in many exact inverse problems

e.g. polynomial interpolation, sparse recovery, inverse moment problems, …

Matrix Pencil Method

V Du

diagonal of uj’s

VH A =


Claim 1: The entries of A correspond to measurements with -m+1 ≤ ω ≤ m

V Du

diagonal of uj’s

VH A =



V Du

diagonal of uj’s

VH V Du VH Dα

diagonal of αj’s

A = B =



V Du

diagonal of uj’s

VH V Du VH Dα

diagonal of αj’s

A = B =

and B



V Du

diagonal of uj’s

VH V Du VH Dα

diagonal of αj’s

A = B =

and B

Claim 2: If αj’s are distinct and m ≥ k and uj’s are non-zero, the unique solns to Ax = λBx are λ = 1/αj

Noise Stability?

Robust Recovery?

Robust Recovery? Fact: The Vandermonde has full (column) rank iff αj’s are distinct, and this is enough for noise-free recovery


When is the Matrix Pencil Method robust to noise?



Generalized Eigenvalue Problem




[Stewart, Sun]: Various stability bounds for generalized eigenvalues/vectors based on the condition number




[Stewart, Sun]: Various stability bounds for generalized eigenvalues/vectors based on the condition number

We show a sharp phase-transition for the condition number of the Vandermonde matrix

We use extremal functions to bound the condition number of the Vandermonde matrix


Such functions are used to prove sharp inequalities (on exponential sums) in analytic number theory

Theorem: ||Vmu||2 = (m-1 ±1/Δ) ||u||2 k



Theorem: ||Vmu||2 = (m-1 ±1/Δ) ||u||2 k

Moreover a direct construction based on the Fejer kernel shows this is tight…



Theorem: ||Vmu||2 = (m-1 ±1/Δ) ||u||2 k

Moreover a direct construction based on the Fejer kernel shows this is tight…



Theorem: If m = (1-ε)/Δ, there is a choice of αj’s, uj’s s.t. ||Vmu||2 ≤ e-εk ||u||2 k

An Interlude The Beurling-Selberg majorant:

sgn(ω)

B(ω)


sgn(ω)

B(ω)

(1) sgn(ω) ≤ B(ω) Properties:


sgn(ω)

B(ω)

(1) sgn(ω) ≤ B(ω)

(2) B(x) supported in [-1,1]

Properties:


sgn(ω)

B(ω)

(1) sgn(ω) ≤ B(ω)


(3) B(ω) – sgn(ω) dω = 1 -∞

∞

Properties:


Properties: (1) sgn(ω) ≤ B(ω)


(3) B(ω) – sgn(ω) dω = 1 -∞

∞

( sign(πω) π ) 2 (

j = 1

∞ (ω - j)-2 -

j = -

-1

(ω - j)-2 + ∞

2 ω )

An Interlude The Beurling-Selberg minorant:

sgn(ω)

b(ω)

(1) b(ω) ≤ sgn(ω)

(2) b(x) supported in [-1,1]

(3) sgn(ω) – b(ω) dω = 1 -∞

∞

Properties:

Proof Omitted

Taking a Step Back Many inverse problems are well-studied in the exact case


When is the solution robust to noise?



Example #1: Polynomial Interpolation

Lagrange Interpolation: Can recover a polynomial p(x) from its evaluations at deg(p)+1 points

p0 … pm-1

coeffs of poly. p(x)


1 α1 α1 2 …

α1 m-1

… 1 αk αk 2

…

αk m-1 …

p0 … pm-1 …

coeffs of poly. p(x)


1 α1 α1 2 …

α1 m-1

… 1 αk αk 2

…

αk m-1 …

p0 … pm-1 p(α1) … p(αk) … = coeffs of poly. p(x)

evals of p(x)


1 α1 α1 2 …

α1 m-1

… 1 αk αk 2

…

αk m-1 …


evals of p(x)

Often highly unstable


1 α1 α1 2 …

α1 m-1

… 1 αk αk 2

…

αk m-1 …


evals of p(x)

Often highly unstable (over the reals), but not if the αj’s are complex roots of unity (DFT matrix)








Example #2: Sums of Exponentials (i.e. super-resolution)




Example #2: Sums of Exponentials (i.e. super-resolution)

Example #3: Extrapolation with Boundary Conditions (lossy population recovery [Moitra, Saks])

Promise: |f(x)| ≤ 1

Known: f(x) ± noise

Goal: approximate value of f(0)

Promise: |f(x)| ≤ 1

Known: f(x) ± noise

Goal: approximate value of f(0)

Hadamard Three Circle Theorem: Can extrapolate f(0) from evaluations on inner circle, if f is bounded on the outter circle

Theme: noisy inverse problems are better posed over the complex plane


Are there other examples of this phenomenon?


We also give other connections between test functions in harmonic analysis and preconditioners



We also give other connections between test functions in harmonic analysis and preconditioners


These functions give a way to obliviously rescale rows of an unknown Vandermonde to make it nearly orthogonal

Thanks!

Any Questions?

Documents

Super-resolution, Extremal Functions and the Condition ...people.csail.mit.edu/moitra/docs/Beurling2.pdfSuper-resolution, Extremal Functions and the Condition Number of Vandermonde