Yonina Eldar

Technion – Israel Institute of Technology

http://www.ee.technion.ac.il/people/YoninaEldar yonina@ee.technion.ac.il

Beyond Nyquist: Compressed Sensing of

Analog Signals

Dagstuhl SeminarDecember, 2008

2

Sampling: “Analog Girl in a Digital World…” Judy Gorman 99

Digital worldAnalog world

ReconstructionD2A

SamplingA2D

Signal processing Denoising Image analysis…

(Interpolation)

3

Compression

Original 2500 KB100%

Compressed 950 KB38%

Compressed 392 KB15%

Compressed 148 KB6%

“Can we not just directly measure the part that will not

end up being thrown away ?” Donoho

4

Outline

Compressed sensing – background

From discrete to analogGoalsPart I : Blind multi-band reconstructionPart II : Analog CS framework

ImplementationsUncertainty relations

Can break the Shannon-Nyquist barrier by exploiting signal structure

5

K non-zero entries at least 2K measurements

CS Setup

y A x

Recovery: brute-force, convex optimization, greedy algorithms, …

6

is uniquely determined by

is random with high probability

Donoho, 2006 and Candès et. al., 2006

NP-hard

Convex and tractable

Greedy algorithms: OMP, FOCUSS, etc.

Donoho, 2006 and Candès et. al., 2006

Tropp, Elad, Cotter et. al,. Chen et. al., and many others

Brief Introduction to CS

Donoho and Elad, 2003

Uniqueness:

Recovery:

7

Naïve Extension to Analog Domain

Standard CSDiscrete Framework

Analog Domain

Sparsity prior what is a sparse analog signal ?

Generalized sampling

Finite dimensional elements Infinitesequence

Continuoussignal

Operator

Random is stable w.h.p Stability Randomness Infinitely manyNeed structure for efficient implementation

Finite program, well-studied Undefined program over a continuous signal

Reconstruction

8

Random is stable w.h.p

Naïve Extension to Analog Domain

Sparsity prior what is a sparse analog signal ?

Generalized sampling

Finite dimensional elements Infinitesequence

Continuoussignal

Operator

Stability Randomness Infinitely manyNeed structure for efficient implementation

Finite program, well-studied Undefined program over a continuous signal

Reconstruction

Questions:

1. What is the definition of analog sparsity ?

2. How to select a sampling operator ?

3. Can we introduce stucture in sampling and still preserve stability ?

4. How to solve infinite dimensional recovery problems ?

Standard CSDiscrete Framework

Analog Domain

9

Goals

1. Concrete analog sparsity model

2. Reduce sampling rate (to minimal)

3. Simple recovery algorithms

4. Practical implementation in hardware

10

Analog Compressed Sensing

A signal with a multiband structure in some basis

no more than N bands, max width B, bandlimited to

(Mishali and Eldar 2007)

1. Each band has an uncountable number of non-zero elements

2. Band locations lie on an infinite grid

3. Band locations are unknown in advance

What is the definition of analog sparsity ?

11

Multiband “Sensing”

bands

Sampling Reconstruction

Goal: Perfect reconstruction

Next steps:1. What is the minimal rate requirement ?2. A fully-blind system design

Analog Infinite Analog

Known band locations (subspace prior):Minimal-rate sampling and reconstruction (NB) with known band locations (Lin and Vaidyanathan 98) Half blind system (Herley and Wong 99, Venkataramani and Bresler 00)

We are interested in unknown spectral support (a union of subspace prior)

(Mishali and Eldar 2007)

12

Rate Requirements

Average sampling rate

Theorem (non-blind recovery)

Landau (1967)

1. The minimal rate is doubled2. For , the rate requirement is samples/sec (on average)

13

Sampling

Analog signal

In each block of samples, only are kept, as described by

Point-wise samples

02

30

0

2

2

3

3

Multi-Coset: Periodic Non-uniform on the Nyquist grid

14

The Sampler

DTFTof sampling sequences

Constant

matrixknown

in vector form

unknowns

Length .knownProblems:

1. Undetermined system – non unique solution

2. Continuous set of linear systems

is jointly sparse and unique under appropriate parameter selection ( )

is sparse

Observation:

15

Paradigm

Solve finiteproblem

Reconstruct

0

1

2

3

4

5

6

S = non-zero rows

16

CTF block

Solve finiteproblem

Reconstruct

MMV

Continuous to Finite

Continuous

Finite

span a finite spaceAny basis preserves the sparsity

17

2-Words on Solving MMV

Variety of methods based on optimizing mixed column-row

norms

We prove equivalence results by extending RIP and coherence to

allow for structured sparsity (Mishali and Eldar, Eldar and

Bolcskei)

New approach: ReMBo – Reduce MMV and Boost

Main idea: Merge columns of V to obtain a single vector problem

y=Aa

Sparsity pattern of a is equal to that of U

Can boost performance by repeating the merging with different

coeff.

Find a matrix U that has as few non-zero rows as

possible

18

Algorithm

CTF

Continuous-to-finite block: Compressed sensing for analog signals Perfect reconstruction at minimal rate

Blind system: band locations are unknown

Can be applied to CS of general analog signals

Works with other sampling techniques

19

Framework: Analog Compressed Sensing

Sampling signals from a union of shift-invariant spaces (SI)

generators

Subspace

(Eldar 2008)

20

Framework: Analog Compressed Sensing

What happen if only K<<N sequences are not zero ?

There is no prior knowledge on the exact indices in the sum

Not a subspace !

Only k sequences are non-zero

21

Framework: Analog Compressed Sensing

Only k sequences are non-zero

CTF

Step 1: Compress the sampling sequencesStep 2: “Push” all operators to analog domain

System AHigh sampling rate = m/T

Post-compression

22

Framework: Analog Compressed Sensing

CTF

Eldar (2008)

Theorem

System BLow sampling rate = p/T

Pre-compression

23

Simulations

0 5 10 15 20 25

1

2

3

4

5

x 104

Time (nano secs)

-50 0 50

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

n

0 5 10 15 20 25

1

2

3

4

5

x 104

Time (nano secs)

Signal Reconstruction filter

Output

Time (nSecs)

Time (nSecs)

Am

plitu

de

Am

plitu

de

24

Simulations

Brute-Force

M-OMP

5 10 150

0.2

0.4

0.6

0.8

1

r

Em

pir

ica

l s

uc

ce

ss

ra

te

SBR4SBR2

5 10 150

0.2

0.4

0.6

0.8

1

r

Em

pir

ica

l s

uc

ce

ss

ra

te

SBR4SBR2

Sampling rate Sampling rate

Minimal rate Minimal rate

25

Simulations

SBR4 SBR2

r

SN

R

5 10 15

10

15

20

0.2

0.4

0.6

0.8

1

rS

NR

5 10 15

10

15

20

0.2

0.4

0.6

0.8

1

r

SN

R

5 10 15

10

15

20

50 100 150 200 250

0.2 0.4 0.6 0.8 1

0.2 0.4 0.6 0.8 1

Empirical recovery rate

Sampling rate Sampling rate

0% Recovery 100% Recovery 0% Recovery 100% Recovery

Noise-free

26

Multi-Coset Limitations

02

30

0

2

2

3

3

Analog signalPoint-wise samples

ADC@ rate

Delay

1. Impossible to match rate for wideband RF signals(Nyquist rate > 200 MHz)

2. Resource waste for IF signals

3. Requires accuratetime delays

27

Efficient Sampling

Use CTF

Efficientimplementation

(Mishali, Eldar, Tropp 2008)

28

Hardware Implementation

A few first steps…

29

Pairs Of Bases

Until now: sparsity in a single basisCan we have a sparse representation in two bases?Motivation: A combination of bases can sometimes better represent the signal

Both and are small!

30

Uncertainty RelationsHow sparse can be in each basis?

Finite setting: vector in

Elad and

Brukstein 2002

Uncertainty relation

Different bases

31

Analog Uncertainty Principle

Eldar (2008)

Theorem

32

Bases With Minimal Coherence

In the DFT domain

Spikes Fourier

What are the analog counterparts ?

Constant magnitude Modulation

“Single” component Shifts

33

Analog Setting: Bandlimited SignalsMinimal coherence:

Tightness:

34

Finding Sparse Representations

Given a dictionary , expand using as few elements as possible:

minimize

Solution is possible using CTF if is small enough

Basic idea:

Sample with basis Obtain an IMV model:

maximal value

Apply CTF to recover Can establish equivalence with as long as is

small enough

35

Conclusion

Extend the basic results of CS to the analog setting - CTF

Sample analog signals at rates much lower than Nyquist

Can find a sparse analog representation

Can be implemented efficiently in hardware

Questions:

Other models of analog sparsity?

Other sampling devices?

Compressed Sensing of Analog Signals

36

Some Things Should Remain At The Nyquist Rate

Thank you

Thank you

High-rate

37

References

M. Mishali and Y. C. Eldar, "Blind Multi-Band Signal Reconstruction: Compressed Sensing for Analog Signals,“ to appear in IEEE Trans. on Signal Processing.M. Mishali and Y. C. Eldar, "Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors", IEEE Trans. on Signal Processing, vol. 56, no. 10, pp. 4692-4702, Oct. 2008.Y. C. Eldar , "Compressed Sensing of Analog Signals", submitted to IEEE Trans. on Signal Processing.Y. C. Eldar and M. Mishali, "Robust Recovery of Signals from a Union of Subspaces’’, submitted to IEEE Trans. on Inform. Theory.Y. C. Eldar, "Uncertainty Relations for Analog Signals",  submitted to IEEE Trans. Inform. Theory. Y. C. Eldar and T. Michaeli, "Beyond Bandlimited Sampling: Nonlinearities, Smoothness and Sparsity", to appear in IEEE Signal Proc. Magazine.