Compressive Covariance Sensing - EURASIP · Compressive Covariance Sensing Covariance Estimation and Detection o Covariance Estimation Least Squares Maximum Likelihood Modal Analysis

1

Acknowledgements: Dyonisius Dony Ariananda (TU Delft)

Siavash Shakeri (TU Delft)

Roberto López-Valcarce (UVigo)

EUSIPCO 2014

Lisbon, Portugal

Compressive Covariance Sensing A New Flavor of Compressive Sensing

Geert Leus Delft University of Technology

[email protected]

Zhi Tian Michigan Technological

University

[email protected]

Daniel Romero University of Vigo

[email protected]

2

Roadmap

Introduction

Compressive Covariance Sensing

Covariance Estimation and Detection

Sampler Design

Advanced Techniques

Open Issues

Conclusions

3

Roadmap

Introduction



Sampler Design

Advanced Techniques

Open Issues

Conclusions

4

Emerging Challenges

Very large arrays

Sampling rate issue Need for compressive techniques

Impulse radio

Cognitive radio (CR)

(Ultra-)wideband signals

Massive MIMO

Large Arrays

5

Compressed Sensing

Large bandwidths require high sampling rates

Popular alternative is compressive sensing (CS)

o Random linear projections of Nyquist rate samples

o Multiple sparse reconstruction techniques

[Donoho, 2006] [Candès et al, 2006] [Tropp, 2004]

6

Spectrum Estimation

Most compressive spectrum estimation methods estimate the

spectrum (or signal itself) using CS methods

Underdetermined problems sparsity constraint

o High computational complexity

o Difficult performance analysis

Observation: many applications just require second-order

statistics (power spectrum)

Overdetermined problems

o Low computational complexity

o Easy performance analysis

o Sparsity or positivity constraints can also be included

7

Covariance and Spectrum Estimation

Cognitive radio (CR)

frequency spectrum Radar

Doppler + angular spectra

Radio astronomy

spatial spectrum

Medical Imaging

resonance spectrum

Seismic

seismic design response spectrum

8

Acquisition of Wideband Signals

RF circuit choices: multiple NB or single WB ?

Multiple, fixed RF chains

Preset LO filter range

Simple detection per BPF

Single, flexible RF chain

burden on A/D: fs ~ GHz

complex wideband sensing

wideband (WB) circuit

A/D LNA AGC

Fixed LO

Wideband

Sensing

WB filter

SNReff

A/D LNA

LO1

A/D LNA AGC

A/D LNA AGC

LO2

LON

Band 1

Band 2

Band N

multiple narrowband (NB) circuits

NB filter SNReff

AGC

Challenge: reduce the sampling rate without sacrificing bandwidth

9

Angular Spectrum Estimation

Array processing

o Imaging

Optical/radar/ultrasound/acoustic

Radio Astronomy

Seismology

o DoA estimation localization

Source: IAI Inc.

Acquisition and processing hardware

prop. to #antenna elements

Challenge: reduce number of antennas

without sacrificing resolution

10

Roadmap

Introduction


o Problem definition

o Covariance structures

o Compression schemes

o Modal Analysis


Sampler Design

Advanced Techniques

Open Issues

Conclusions

11


Problem definition:

Structure:

Compression operation:

Remarks:

NO SPARSITY NEEDED

compression

uncompressed

signal

compressed

signal

SOS:

Estimate from , multiple ’s, or

12

Second-Order Statistics

Stationary input:

Auto-correlation:

Power Spectral Density (PSD):

compression

uncompressed

signal

compressed

signal

Frequency domain PSD

time frequency

Angular domain PSD

space angle

13

Covariance Structure

All covariance matrices are Hermitian and positive semi-definite

Typical structures

o Toeplitz:

Constant along diagonals

Stationary time-signals

Uniform linear arrays (ULA)

Modal analysis

o Circulant:

Toeplitz + property

diagonal in the freq. domain

o -Banded

Toeplitz + property

Encompassing model: Basis Expansion Model (BEM)

basis matrices

14


Toeplitz Circulant

real unknowns real unknowns

d-Banded

real unknowns

BEM representation:

15

Compression Schemes

We consider throughout linear compression schemes

Focus on:

o Frequency PSD Time-domain autocorrelation Time compression

o Angular PSD Space-domain autocorrelation Spatial compression

compression

uncompressed

signal

compressed

signal

16

Periodic Compression

with

Given and estimate

# blocks

Uncompressed

domain

Compressed

domain

Kronecker notation

17

Compression in the Time Domain

Nyquist-rate

sampling

MUX

Compressive ADC (conceptual model)

Periodic Acquisition

18

Compression in the Time Domain

Implementations

o Multi-coset sampling [Herley-Wong,1999][Venkataramani-

Bresler,2000]

o Random demodulator [Tropp et al, 2010]

o Modulated wideband converter [Mishali-Eldar,2010]

o Random modulator pre-integrator [Becker, 2011][Yoo et al, 2012]

[Mishali-Eldar,2010]

Modulated wideband converter

19

Compression in the Spatial Domain Uniform linear array (ULA)

Periodic Acquisition

20

Compression in the Spatial Domain

Implementation:

o Sparse some antennas are active subarrays [Hoctor-Kassam, 1990], [Moffet, 1968]

o Dense all antennas are active analog beamforming [Wang-Leus-Pandharipande, 2009],[Wang-Leus, 2010],[Venkateswaran-Van der

Veen, 2010]

[Wang-Leus, 2010]

21

PSD Estimation from

Dense PSD estimation

o Fourier transform

o Application:

Frequency domain PSD estimation of time stationary signals

Angular domain incoherent imaging (continuous source distribution)

Sparse PSD estimation

o Modal analysis

o Application:

Frequency domain frequency estimation of a sum of sinusoids in noise

Angular domain direction of arrival (DoA) estimation (discrete source

distribution)

Estimate from

22

Roadmap

Introduction



o Covariance Estimation

Least Squares

Maximum Likelihood

Modal Analysis

o Covariance Detection

o Sample Statistics Pre-Processing

Sampler Design

Advanced Techniques

Open Issues

Conclusions

23

Roadmap

Introduction




Least Squares

Maximum Likelihood

Modal Analysis



Sampler Design

Advanced Techniques

Open Issues

Conclusions

24

Major steps:

1. Identify relation between and

2. Identify relation between and

3. Estimate using (pre-processed) sample estimates

4. Invert the relation with least squares (LS)

5. Reconstruct

Least Squares Estimation [Ariananda-Leus, 2012]

[Leus-Ariananda, 2011]

25

Least Squares Estimation

Sample

Estimate

Least

Squares

Covariance

Reconstruction

Compressed

domain Uncompressed

domain

• Overdetermined system

• Unique reconstruction if

full (column) rank

Design of is critical!!

[Ariananda-Leus, 2012]

[Leus-Ariananda, 2011]

26

LS: Toeplitz/Circulant/Banded Matrices

Toeplitz Circulant

real unknowns real unknowns

d-Banded

real unknowns

27

LS Estimation: PSD

Power Spectrum:

LS estimate:

Improvements:

PSD is non-negative

OR

PSD is sparse (can be relaxed)

OR

Covariance is pos. semidefinite

28

Simulations: Frequency PSD Estimation

Least Squares reconstruction

space of Hermitian Toeplitz d-banded matrices

Complex baseband representation of OFDM signal:

o 16-QAM data symbols

o 8192 tones in band

o 3072 active tones in

o Cyclic prefix length of 1024

o SNR of 10 dB

and

Start with length-42 minimal sparse ruler,

Larger cases by randomly adding extra rows

29


MSE of the estimated PSD

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

10 -2

10 -1

Compression rate [M/N]

MS

E

Sparse Ruler (I=549)




Nyquist (I=549)

Nyquist (I=1646)

Nyquist (I=3291)

Nyquist (I=5485)

30


Reconstructed PSD

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 -15

-10

-5

0

5

10

Normalized Frequency ( p rad/sample)

Po

wer/

Fre

qu

en

cy (

dB

/rad

/sam

ple

)

Theoretical Noisy PSD

Sparse Ruler (460746 samples, M/N=0.5)

31

Maximum Likelihood

If is well designed:

Maximum Likelihood (ML) estimate:

independent independent It suffices to estimate

32

Maximum Likelihood

[Burg et al, 1982]

Sample estimate

Gaussian Gaussian

Numerical solution Covariance matching

Inverse Iteration Algorithm (IIA)

Trading performance with computation

Pre-processed sample estimates [Romero-Leus,2013a]

Asymptotic approximations: COMET [Ottersten et al, 1998],

SPICE [Stoica et al, 2011], etc.

33

Simulations: Wideband Spectrum Sensing

primary transmitters

-th transmitter normalized PSD

received power:

Estimators:

o LS estimators: Weighted LS and constrained & weighted LS

o Approx. ML estimation: SIIA

bandpass Gaussian signals with disjoint support

white Gaussian noise

C-ADC:

[Romero-Leus, 2013a]

34

Simulations: Wideband Spectrum Sensing

Strong compression ratios may result in a small performance loss

35

Modal Analysis: Observation Model

# sources/sinusoids

Space domain Time domain

36

What kind of sources do we have?

o Uncorrelated sources: is diagonal is Toeplitz

o Correlated sources? [Ariananda-Leus, 2012, 2013]

Modal Analysis

[Pillai et al,1985] [Abramovich et al,1998,1999]

[Pal-Vaidyanathan, 2010, 2011] [Shakeri-Ariananda-Leus, 2012]

[Yen-Tsai-Wang, 2013], [Krieger-Kochman-Wornell, 2013]

Uncompressed domain Compressed domain

37

Modal Analysis: Standard Methods

Correlation matrix of :

More measurements than sources

MUSIC:

MVDR:

[Bresler, 2008], [Mishali-Eldar, 2009]

[Wang-Pandharipande-Leus, 2010]

38

Modal Analysis: Gridding-Based Methods Grid of frequencies/angles and virtual

sources/sinusoids on this grid source vector

is overcomplete basis for but is sparse

Sparse reconstruction [Malioutov et al, 2005] [Mishali-Eldar, 2009] [Tropp et al, 2010]

More measurements than sources!!

39

Modal Analysis: Virtual Sampler/Array

Uncorrelated sources:

represents a virtual sampler/array of virtual

samples/antennas receiving virtual sources/sinusoids

equations

unknowns

40

Modal Analysis: Virtual Sampler/Array

Problem: virtual sources are constant or fully coherent

o Gridding

: LS : sparsity/positivity

Sparse sampling (antenna selection):

o MUSIC or MVDR with spatial smoothing

Virtual sampler/array should be uniform!

sparse ruler

o Translate problem into circulant covariance matrix

Uniform gridding required with grid points

circular sparse ruler

[Shakeri-Ariananda-Leus, 2012]

41

Simulations: DoA Estimation

Least Squares and MUSIC reconstruction

space of Hermitian Toeplitz matrices

ULA of and available antenna positions (aperture)

o Minimal sparse ruler array Virtual ULA of 2x36-1 antennas

o Two-level nested array

Inner array of 5 and outer array of 6 antennas

Virtual ULA of 2x36-1 antennas

o Co-prime array

9 antennas spacing 2 and 3 antennas spacing 9

Virtual ULA of only 2x20-1 antennas

1600 time samples

SNR of 0 dB

42


-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-40

-35

-30

-25

-20

-15

-10

-5

0

Direction of Arrival [sin()]

No

rma

lize

d S

pe

ctr

um

[d

B]

Least Squares Method vs. MUSIC Method

LS method

MUSIC method

Reconstructed spectrum using LS and MUSIC for the minimal sparse ruler array

(S=17 sources with 10 degrees of separation; for LS S=71)

43


-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-50

-45

-40

-35

-30

-25

-20

-15

-10

-5

0


No

rma

lize

d S

pe

ctr

um

[d

B]

Least Squares Method vs. MUSIC Method

LS method

MUSIC method

Reconstructed spectrum using LS and MUSIC for the minimal sparse ruler array

(continuous source from 30 to 40 degrees; for LS S=71)

44


-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-60

-50

-40

-30

-20

-10

0


No

rma

liz

ed

Sp

ec

tru

m [

dB

]

Minimal Sparse Ruler Array

Two-Level Nested Array

Coprime Array

Reconstructed spectrum using MUSIC (S=21 sources with 7 degrees separation)

45

Roadmap

Introduction




Least Squares

Maximum Likelihood

Modal Analysis



Sampler Design

Advanced Techniques

Open Issues

Conclusions

46

Covariance Detection

Binary hypothesis test

Problem statement:

Depending on prior information: Neyman-Pearson, generalized

likelihood ratio test (GLRT), Bayesian, etc

Noise covariance

Signal covariance

Given decide or

GLRT:

[Kay, 1998]

47

Covariance Detection

GLRT: alternative formulation

Exact computation of ML estimates expensive

Alleviate computational cost

o Replace ML estimates by approximations

o Smoothing/cropping sample statistics

BEM formulation:

[Romero-Leus,2013a]

[VázquezVilar-LópezValcarce, 2011]

given

48

Covariance Detection: Spectrum Sensing

Wideband spectrum sensing formulation:

GLRT:

Efficiency approximate ML estimators

#signals

Power -th signal

Is the -th primary

user transmitting

ML estimator under

ML estimator under

[Romero-Leus,2013a]

[VázquezVilar-LópezValcarce, 2011]

49

Simulation: Wideband Spectrum Sensing

Test on the user with

primary transmitters

-th transmitter normalized PSD

received power:

Estimators:

o LS estimators: WLS, CWLS

o ML estimator: LIKES

o Approx. ML estimation: SSPICE, SIIA, SLIKES

bandpass Gaussian signals with disjoint support

white Gaussian noise

C-ADC:

[Romero-Leus,2013a]

50

Simulation: Wideband Spectrum Sensing

Approximate ML estimators achieve a similar performance at a much reduced cost

51

Roadmap

Introduction




Least Squares

Maximum Likelihood

Modal Analysis


o Sample Statistics Pre-processing

Sampler Design

Advanced Techniques

Open Issues

Conclusions

52

Sample Statistics Pre-Processing

LS and ML work on

Sometimes several observations of are available

• Spatial auto-correlation average along time

• Temporal auto-correlation average along space Dual domain averaging

53


Dual domain averaging

54


MUX

MUX

MUX

Dual domain averaging

55


If a single observation of is given

o Raw sample estimate may result in poor performance

o Smoothing exploiting periodic structure:

Controlling the bias/variance trade-off windowing/cropping

o Given blocks make be with

o May also help to control complexity

stationary

Rows of

stationary

Smoothed estimate

56

Roadmap

Introduction



Sampler Design

o Design criteria

o Sparse samplers

o Dense samplers

Advanced Techniques

Open Issues

Conclusions

57

Sampler Design: Compression Model

General structure for

o Temporal compression

o Spatial compression

o …

Model can represent

o Periodic sampling

o Non-periodic sampling

o Sparse sampling

o Dense sampling

58

Design Problem

# blocks

Uncompressed

domain

Compressed

domain Compression Ratio

Find conditions for to allow estimation of from

Maximize the compression ratio among the admissible samplers

Goals

59


Modal analysis

Multi-band signal

Toeplitz subspace

Banded subspace

Circulant subspace

Uncompressed

domain

Compressed

domain

Dimension:

60

Design Criteria

The identifiability of is preserved

linearly independent linearly independent

A matrix defines an -covariance sampler

if the associated function is invertible.

Focus on

Toeplitz subspaces

banded subspaces

Circulant subspaces

Universal cov. samplers

61

Sparse Samplers

Architectures:

o Space domain sampling Sub-array

o Time domain sampling C-ADC

Set representation:

62

(Linear) Sparse Rulers

Difference set:

Sparse ruler:

Minimal sparse ruler

Suboptimal designs: nested, co-prime, …

[Rédei-Rényi, 1949] [Leech, 1956] [Wichmann, 1963] [Moffet, 1968] [Miller, 1971] [Wild,

1987] [Pearson et al, 1990] [Linebarger et al, 1993] [Ariananda-Leus, 2012]

[Wichmann, 1963] [Pearson et al, 1990] [Linebarger et al, 1993] [Pumphrey, 1993]

[Pal-Vaidyanathan, 2010] [Pal-Vaidyanathan, 2011]

is a length- sparse ruler

w/o repetition

63

Circular Sparse Rulers

Modular difference set:

Circular sparse ruler:

Minimal circular sparse ruler

is a length- circular sparse ruler

w/o repetition

[Singer, 1938] [Miller, 1971] [Ariananda-Leus, 2012] [Romero-Leus, 2013b]

[Krieger-Kochman-Wornell, 2013] [Romero-LópezValcarce-Leus, 2014]

64

Sparse Samplers: Toeplitz Subspace

Sparse sampler Toeplitz subspace

covariance sampler

linear sparse ruler

Optimum Sampler

minimal linear sparse ruler

[Rédei-Rényi, 1949] [Leech, 1956] [Pearson et al, 1990][Romero-LópezValcarce-Leus, 2014]

65

Sparse Samplers: Circulant Subspace

Sparse sampler Circulant subspace

covariance sampler


Optimum Sampler

minimal circular sparse ruler

[Romero-LópezValcarce-Leus, 2014]

66

Sparse Samplers: Banded Subspace

Sparse sampler -banded subspace

covariance sampler

linear sparse ruler


incomp. sparse ruler

[Ariananda-Leus, 2012][Romero-LópezValcarce-Leus, 2014]

67

Dense Samplers

Architectures:

o Space domain sampling

o Time domain sampling C-ADC

Design:

o Similar to CS random designs

o Existing random designs

Continuous distributions

Cov. samplers with probability one

Attain compression limits

analog

beamforming


68

-covariance sampler a.s. iff

Random Sampling: Compression Limits

Toeplitz subspace:

Banded subspace:

Circulant subspace:

drawn from a cont. distrib.


69

Optimal Samplers: Summary

(transpose the table)

70

Sampler Design: Summary

71

Roadmap

Introduction



Sampler Design

Advanced Techniques

o Cyclostationarity

o Multiband spectrum estimation

o Distributed Compressive Covariance Sensing

o Dynamic Sampling

Open Issues

Conclusions

72

Roadmap

Introduction



Sampler Design

Advanced Techniques

o Cyclostationarity

Estimation

Sampler Design



o Dynamic Sampling

Open Issues

Conclusions

73

Cyclostationary modulated signals

Cyclic features reveal critical signal parameters:

o carrier frequency

o symbol rate

o modulation type

o timing, phase etc.

Non-cyclic signals (e.g. noise) do not possess cycle frequencies

Periodic autocorrelation Cyclic spectrum

2x Fourier Transform

74

Noise Suppression in the Cyclic Domain

Power spectrum density (PSD)

(a = 0)

Spectral correlation density (SCD)

Hig

h S

NR

L

ow

SN

R

Multi-harmonics

peaks at

Energy detection vs. cyclic feature detection e.g., [Sahai-Cabric, 2005]

✔ no noise components when

75

BPSK signal alone

Source Separation in Cyclic Domain

Cyclostationarity-based approach to detection

o resilient against Gaussian noise

o robust to multipath

o can differentiate modulation types and separate interference

o insensitive to unknown signal parameters

Overlapping in PSD, separable in SCD

Spectral correlation density (SCD) e.g., [Gardner, 1988]

White noise plus

five AM interferences BPSK in noise plus

five AM interferers

76

Second-order Cyclic Statistics

Suppose is periodic in t with period T

Periodic autocorrelation and cyclic spectrum

Cyclic frequency: Frequency: f

Stack samples over a block of multiple ( ) cyclic periods

o # samples per block:

compression

uncompressed

signal

compressed

signal

T=1: x[t] stationary T>1: x[t] cyclostationary

77


Cyclostationary signals with period T

Covariance structure

o Within one cyclic period: dense, with DoF = T2

o Over a block of N cyclic periods

block Toeplitz

DoF

e.g.

78

Compressive samples

Compressive measurements: M samples per block

Corss-correlation of compressive samples

It can be shown that

v.s.

79

Vector-form Relationships: Covariances

Assuming limited support we obtain

is a block-circulant matrix, which allows for block-diagonalization

80

Vector-form Relationships: Cyclic Spectrum

Since , delays in

Hence, uniquely determined by samples in f

Since is related to by DFT and DTFT operations:

is a invertible matrix, so

81

Cyclic Spectrum Reconstruction

If then we can use simple least squares (LS)

When this is only possible if (no compression)

When (e.g., when ) this is possible even for

Advantages

Computational simplicity

Performance guarantees (e.g., probability of false alarm)

When the period of the sampler is larger than the period of the

cyclostationarity, we can reconstruct the cyclic spectrum by LS

82

C-ADC

(sub-Nyquist)

Sparse cyclic

spectrum

reconstruction

Cyclic feature

detection

Cyclic feature

classification

Wideband aspect:

multiple signal sources

frequency pow

er

Cyclic Spectrum Estimation for CR

83

Example: Robustness to Rate Reduction

Probability of detection vs. compression ratio (PFA= 0.1, #blocks=32, 200)

Monitored band |fmax| < 300 MHz

2 sources (noise-free): PU1 - BPSK at 150MHz;

PU2 - QPSK at 225MHz; Ts=0.02667μs

Cisco 802.11 DSSS

Spread spectrum

50%

compression

50%

compression

84

Example: Robustness to Noise Uncertainty

Outperforms energy detection and insensitive to noise uncertainty

Receiver Operating Characteristic (ROC): PD vs PFA (SNR=5dB, 50% compression)

Energy detection

(noise uncertainty = 0, 1, 2, 3dB)

cyclic

85

Roadmap

Introduction



Sampler Design

Advanced Techniques

o Cyclostationarity

Estimation

Sampler Design



o Dynamic Sampling

Open Issues

Conclusions

86

Sampler Design for Cyclostationary Signals

Sampler design for reconstructing cyclic spectrum

o Multiple samplers used periodically across N blocks

Proposition: it is possible to reconstruct non-

sparse 2D cyclic spectrum in closed form when

Original signal x[n] Structure of Rx Compressed signal y[n]

Stationary Toeplitz Cyclostationary

Cyclostationary

(period T)

Block Toeplitz

(block size NT )

Cyclostationary

(period M)

Circular

sparse ruler

[Leus-Tian, 2011]

87

Cyclostationary Signals: Intuition

Stationary signal

Input: stationary with

cross-correlations

Output: cyclostationary

with period

cross-correlations

Increase of degrees of freedom!

88


Cyclostationary signal

Input: cyclostationary

with period

cross-correlations


with period Q

cross-correlations

No increase of degrees of freedom!

89


Cyclostationary signal

Input: cyclostationary

with period

cross-correlations


with period

cross-correlations

To increase degrees of freedom the period of the sampler needs to be

larger than the period of the cyclostationarity!

[Leus-Tian, 2011]

90

Our goals:

reconstruct from using least squares (LS)

System of Equations

must have full column rank.

selection matrix

collection of

correlations of

collection of

correlations of

at blocks-lag at blocks-lag

91

Minimal Sparse Ruler Sampling

One possible sampling pattern that leads to a full

column rank minimal sparse ruler based design

Example:

Other are set to

identity matrix empty matrix

Minimal sparse ruler based design obtain the minimum

compression for the case when each is equal to either [] or

92

Example

Consider based on length-5 minimal sparse ruler, we

have: and

All

have full

column rank

93

Exploiting Correlations Between -blocks

94

System of equations:

correlations of

When is not constrained to either [] or

greedy algorithm [Ariananda-Leus, 2014] to design

sub-optimal compression rate

at blocks-lag


95

full

column

rank!

full

column

rank!

Example: , and

as well as

full column

rank!

full column

rank!


96

constrained to or for each

Maximum compression minimal circular sparse ruler

Example:

Other are set to



97

8 10 12 14 16 18 20 220.2

0.25

0.3

0.35

0.4

0.45

The value of N

Ac

hie

ve

d C

om

pre

ss

ion

Ra

te

Minimal Circular SparseRuler (all T)

Greedy Algorithm T=18




Compression Rate

98

Roadmap

Introduction



Sampler Design

Advanced Techniques

o Cyclostationarity



o Dynamic Sampling

Open Issues

Conclusions

99

Temporal Compression

Nyquist-

rate

samples

MUX

...

...

analog-to-information converter (AIC)

100

Spatial Compression

uniform linear

array (ULA)

...

...

...

...

[Krieger et al, 2013]

101

With compression vector:

Data Model

Compute the DTFT:

and

Nyquist-rate case vector

We then have:

102

Multi-bin Approach

Write and collect in

We have an IDFT matrix

Consider the multiband model and divide into bins

...

[Mishali-Eldar, 2009]

103

Diagonal/Circulant Correlation Matrix

If the bin size is equal to the largest band

is diagonal [Yen-Tsai-Wang, 2013]

The correlation matrix of :

is circulant [Ariananda-Romero-Leus, 2013, 2014]

104

full column rank

universal and M/N > 1/2

full column rank


Diagonal/Circulant Correlation Matrix Given

Multicoset sampling/array

...

...

If has full column rank, we can perform LS ( )

ML methods can also be employed [Romero-Leus, 2013]

[Yen-Tsai-Wang, 2013]

[Ariananda-Romero-Leus, 2013, 2014]

105

Sample Estimates

Q: Over which domain do we average?

A: The dual domain!

...

...

...

...

Spatial-domain

Compression

106

Sample Estimates Time-domain

Compression

...

MUX ... MUX ...

...

MUX ...

...

107

Simulations

Number of samples: block size:

Number of blocks:

Sensing based on length- minimal circular sparse ruler

=> active cosets:

Averaged over 200 independent realizations

Six bands (user signals) => details can be found in the paper.

White Gaussian noise for each realization; variance:

108

Simulations

-0.4 -0.2 0 0.2 0.4

0

50

100

150

200

2p ( 2p) (radian)

No

rma

liz

ed

Ma

gn

itu

de

/Fre

qu

en

cy

(mW

/ra

dia

n/s

am

ple

)

M/N=0.278

Nyquist rate

109

Roadmap

Introduction



Sampler Design

Advanced Techniques

o Cyclostationarity



o Dynamic Sampling

Open Issues

Conclusions

110

Distributed Sensing: Partial Observation

Each sensor cluster

may estimate a subset

of the total lags

Sampling rate reduction

[Ariananda-Romero-Leus, 2014]

LS estimation

111

Distributed Sensing: Sampler Design

Necessary condition

Number

of sensor

clusters

Modular dif. set

Non-overlapping circular

Golumb rulers

[Ariananda-Romero-Leus, 2014]

112

Simulation

113

Roadmap

Introduction



Sampler Design

Advanced Techniques

o Cyclostationarity



o Dynamic Sampling

Open Issues

Conclusions

114

Dynamic Sampling

Changing sub-array configurations allows detecting more correlated

sources than antennas

Uncorrelated sources:

represents a virtual sampler/array of virtual

samples/antennas receiving virtual sources/sinusoids

equations

unknowns

115

Dynamic Sampling

Assume the sources are correlated

Problem: degrees of freedom are not increased

Solution: periodic compression with slots per period

o Periodic (in time) antenna selection

o Periodic (in space) sample selection


116

Correlated Sources

Combining the correlations from slots

represents a virtual sampler/array of virtual samples/antennas

receiving virtual sources

Spatial smoothing not directly possible, so gridding left:

Over-determined: LS

Under-determined: sparsity or positivity constraints

Universality: every pair of samples appears in a period

117

Roadmap

Introduction



Sampler Design

Advanced Techniques

Open Issues

Conclusions

118

Open Issues

Super resolution

Big data

Combining spatial and

temporal domains

Extensions to Doppler

spectrum, imaging, …

Applications to radar,

MRI, seismic, radio

astronomy, …

119

Open Issues

Large-scale networks

Sample size issue sparsity-enforcing regularization

Internet backbone network (Abilene) Disease gene network

120

Sparse event detection using large-scale wireless sensor nets

o Local phenomena induce spatially sparse signals

o Desiderata: energy efficiency, scalability, robustness

Centralized

FUSION

CENTER

Decentralized

FUSION

CENTER Scalability Robustness

Infrastructureless

Structural Health Monitoring

Floor 1

Floor 2

Floor 3

Floor 4

Floor 5

Floor 6

Floor 7

Floor 8

Floor 9

Floor 10

Floor 11

Floor 12

Bay 1 Bay 2 Bay 3 Bay 4 Bay 5 Bay 6 Bay 7 Bay 8 Bay 9 Bay 10

(2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (2,7) (2,8) (2,9) (2,10)

(3,1) (3,2) (3,3) (3,4) (3,5) (3,6) (3,7) (3,8) (3,9) (3,10)

(4,1) (4,2) (4,3) (4,4) (4,5) (4,6) (4,7) (4,8) (4,9) (4,10)

(5,1) (5,2) (5,3) (5,4) (5,5) (5,6) (5,7) (5,8) (5,9) (5,10)

(6,1) (6,2) (6,3) (6,4) (6,5) (6,6) (6,7) (6,8) (6,9) (6,10)

(7,1) (7,2) (7,3) (7,4) (7,5) (7,6) (7,7) (7,8) (7,9) (7,10)

(8,1) (8,2) (8,3) (8,4) (8,5) (8,6) (8,7) (8,8) (8,9) (8,10)

(9,1) (9,2) (9,3) (9,4) (9,5) (9,6) (9,7) (9,8) (9,9) (9,10)

(10,1) (10,2) (10,3) (10,4) (10,5) (10,6) (10,7) (10,8) (10,9) (10,10)

(11,1) (11,2) (11,3) (11,4) (11,5) (11,6) (11,7) (11,8) (11,9) (11,10)

(12,1) (12,2) (12,3) (12,4) (12,5) (12,6) (12,7) (12,8) (12,9) (12,10)

(13,1) (13,2) (13,3) (13,4) (13,5) (13,6) (13,7) (13,8) (13,9) (13,10)

[Ling-Tian et al, 2009]

Distributed optimization

[Ling-Tian, 2010, 2011]

Open Issues

121

Roadmap

Introduction



Sampler Design

Advanced Techniques

Open Issues

Conclusions

122

Conclusions: CS vs. CCS

Compressed Covariance Sensing Compressed Sensing

Aims at recovering statistics Aims at recovering the signal

Lossy Lossless

Use sparse sampling

Use random sampling

No sparsity is required Sparsity is required

Linear/non-linear reconstruction Non-linear reconstruction

Overdetermined Underdetermined

123

Conclusions

Aiming at reconstructing the second-order statistics

o Compressive samples

o Linear compression

o Linear/non-linear covariance structures

Estimation/detection:

o LS

o ML

o GLRT

Sampler design

o Sparse samplers

o Dense samplers

Advanced techniques

124

Thank you!

125

References

D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory, vol. 52,

no. 4, pp. 1289–1306, Apr. 2006.

E. J. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact

signal reconstruction from highly incomplete frequency information,”

IEEE Trans. Inf. Theory, vol. 52, no. 2, pp. 489–509, feb. 2006.

J. A. Tropp, “Greed is good: algorithmic results for sparse approximation,”

IEEE Trans. Inf. Theory, vol. 50, no. 10, pp. 2231–2242, Oct 2004.

M. Mishali and Y. C. Eldar, “From theory to practice: Sub-Nyquist

sampling of sparse wideband analog signals,” IEEE J. Sel. Topics Signal

Process., vol. 4, no. 2, pp. 375–391, Apr. 2010.

J. Yoo, S. Becker, M. Monge, M. Loh, E. Candes, and A. Emami-

Neyestanak, “Design and implementation of a fully integrated compressed-

sensing signal acquisition system,” in Inf. Theory Appl. Workshop, Mar.

2012, pp. 5325–5328.

126

References

W. C. Black and D. Hodges, “Time interleaved converter arrays,” IEEE J.

Solid-State Circuits, vol. 15, no. 6, pp. 1022–1029, Dec 1980.

J. A. Tropp, J. Laska, M. Duarte, J. Romberg, and R. Baraniuk, “Beyond

nyquist: Efficient sampling of sparse bandlimited signals,” IEEE Trans. Inf.

Theory, vol. 56, no. 1, pp. 520–544, Jan 2010.

C. Herley and P. W. Wong, “Minimum rate sampling and reconstruction of

signals with arbitrary frequency support,” IEEE Trans. Inf. Theory, vol. 45,

no. 5, pp. 1555–1564, Jul 1999.

R. Venkataramani and Y. Bresler, “Perfect reconstruction formulas and

bounds on aliasing error in sub-Nyquist nonuniform sampling of multiband

signals,” IEEE Trans. Inf. Theory, vol. 46, no. 6, pp. 2173–2183, Sep 2000.

S. Becker, “Practical compressed sensing: modern data acquisition and

signal processing,” Ph.D. dissertation, California Institute of Technology,

2011.

R. T. Hoctor and S. A. Kassam, “The unifying role of the coarray in

aperture synthesis for coherent and incoherent imaging,” Proc. IEEE, vol.

78, no. 4, pp. 735–752, Apr. 1990.

127

References

A. Moffet, “Minimum-redundancy linear arrays,” IEEE Trans. Antennas

Propag., vol. 16, no. 2, pp. 172–175, Mar. 1968.

Y. Wang, G. Leus, and A. Pandharipande, “Direction estimation using

compressive sampling array processing,” in IEEE/SP 15th Workshop on

Statistical Signal Process., 2009, pp. 626–629.

Y. Wang and G. Leus, “Space-time compressive sampling array,” in

Sensor Array and Multichannel Signal Process. Workshop (SAM), 2010,

pp. 33–36.

V. Venkateswaran and A. J. van der Veen, “Analog beamforming in

MIMO communications with phase shift networks and online channel

estimation,” IEEE Trans. Signal Process., vol. 58, no. 8, pp. 4131–4143,

2010.

S. U. Pillai, Y. Bar-Ness, and F. Haber, “A new approach to array

geometry for improved spatial spectrum estimation,” Proc. IEEE, vol. 73,

no. 10, pp. 1522–1524, 1985.

128

References

Y. I. Abramovich, D. A. Gray, A. Y. Gorokhov, and N. K. Spencer,

“Positive-definite Toeplitz completion in DOA estimation for nonuniform

linear antenna arrays. I. Fully augmentable arrays,” IEEE Trans. Signal

Process., vol. 46, no. 9, pp. 2458–2471, 1998.

Y. I. Abramovich, N. K. Spencer, and A. Y. Gorokhov, “Positive-definite

Toeplitz completion in DOA estimation for nonuniform linear antenna

arrays. II. Partially augmentable arrays,” IEEE Trans. Signal Process., vol.

47, no. 6, pp. 1502–1521, 1999.

P. Pal and P. P. Vaidyanathan, “Nested arrays: A novel approach to array

processing with enhanced degrees of freedom,” IEEE Trans. Signal

Process., vol. 58, no. 8, pp. 4167–4181, Aug. 2010.

——, “Coprime sampling and the MUSIC algorithm,” in Digital Signal

Process. Workshop and Signal Process. Educ. Workshop (DSP/SPE), 2011

IEEE, Jan. 2011, pp. 289–294.

S. Shakeri, D. D. Ariananda, and G. Leus, “Direction of arrival estimation

using sparse ruler array design,” in IEEE Int. Workshop Signal Process.

Advances Wireless Commun. (SPAWC), Jun. 2012, pp. 525–529.

129

References

C. P. Yen, Y. Tsai, and X. Wang, “Wideband spectrum sensing based on

sub-Nyquist sampling,” IEEE Trans. Signal Process., vol. 61, no. 12, pp.

3028–3040, 2013.

J. D. Krieger, Y. Kochman, and G. W. Wornell, “Design and analysis of

multi-coset arrays,” in Inf. Theory Appl. Workshop, 2013.

D. D. Ariananda and G. Leus, “Direction of arrival estimation for more

correlated sources than active sensors,” Signal Processing, vol. 93, no. 12,

pp. 3435–3448, 2013.

Y. Wang, A. Pandharipande, and G. Leus, “Compressive sampling based

mvdr spectrum sensing,” in Cognitive Inform. Process. (CIP), June 2010,

pp. 333–337.

Y. Bresler, “Spectrum-blind sampling and compressive sensing for

continuous-index signals,” 27 2008-feb. 1 2008, pp. 547–554.

D. Malioutov, M. Cetin, and A. S. Willsky, “A sparse signal reconstruction

perspective for source localization with sensor arrays,” IEEE Trans. Signal

Process., vol. 53, no. 8, pp. 3010–3022, 2005.

130

References

M. Mishali and Y. C. Eldar, “Blind multiband signal reconstruction:

Compressed sensing for analog signals,” IEEE Trans. Signal Process., vol.

57, no. 3, pp. 993–1009, Mar. 2009.

D. D. Ariananda and G. Leus, “Compressive wideband power spectrum

estimation,”IEEE Trans. Signal Process., vol. 60, no. 9, pp. 4775–4789,

2012.

G. Leus and D. D. Ariananda, “Power spectrum blind sampling,” IEEE

Signal Process. Lett., vol. 18, no. 8, pp. 443–446, 2011.

J. P. Burg, D. G. Luenberger, and D. L. Wenger, “Estimation of structured

covariance matrices,” Proc. IEEE, vol. 70, no. 9, pp. 963–974, Sep. 1982.

D. Romero and G. Leus, “Wideband spectrum sensing from compressed

measurements using spectral prior information,” IEEE Trans. Signal

Process., vol. 61, no. 24, pp. 6232–6246, 2013.

S. M. Kay, Fundamentals of Statistical Signal Process., Vol. II: Detection

Theory. Prentice-Hall, 1998.

131

References G. Vazquez-Vilar and R. Lopez-Valcarce, “Spectrum sensing exploiting

guard bands and weak channels,” IEEE Trans. Signal Process., vol. 59, no.

12, pp. 6045–6057, 2011.

L. Redei and A. Renyi, “On the representation of the numbers 1,2,. . . ,n by

means of differences (Russian),” Matematicheskii sbornik, vol. 66, no. 3,

pp. 385–389, 1949.

J. Leech, “On the representation of 1, 2,..., n by differences,” J. London

Mathematical Society, vol. 1, no. 2, pp. 160–169, 1956.

B. Wichmann, “A note on restricted difference bases,” J. London

Mathematical Society, vol. 1, no. 1, pp. 465–466, 1963.

J. C. P. Miller, “Difference bases, three problems in additive number

theory,” Comput. in Number Theory, pp. 299–322, 1971.

P. Wild, “Difference basis systems,” Discrete mathematics, vol. 63, no. 1,

pp. 81–90, 1987.

D. Pearson, S. U. Pillai, and Y. Lee, “An algorithm for near-optimal

placement ofsensor elements,” IEEE Trans. Inf. Theory, vol. 36, no. 6, pp.

1280–1284, 1990.

132

References

D. A. Linebarger, I. H. Sudborough, and I. G. Tollis, “Difference bases and

sparse sensor arrays,” IEEE Trans. Inf. Theory, vol. 39, no. 2, pp. 716–721,

1993.

H. C. Pumphrey, “Design of sparse arrays in one, two, and three

dimensions,” J. Acoust. Society America, vol. 93, p. 1620, 1993.

J. Singer, “A theorem in finite projective geometry and some applications

to number theory,” Trans. American Math. Soc., vol. 43, no. 3, pp. 377–

385, 1938.

D. Romero and G. Leus, “Compressive covariance sampling,” in Inform.

Theory Appl. Workshop, San Diego, CA, Feb. 2013, pp. 1–8.

D. Romero, R. Lopez-Valcarce, and G. Leus, “Compression limits for

random vectors with linearly parameterized second-order statistics,”

Submitted to IEEE Trans. Inf. Theory, 2014, Online: http://arxiv.org/abs/

1311.0737.

A. Sahai and D. Cabric, “Spectrum sensing: fundamental limits and

practical challenges,” in IEEE Int. Symp. New Frontiers Dynamic

Spectrum Access Netw.,Baltimore, MD, Nov. 2005.

133

References

P. Stoica, P. Babu, and J. Li, “SPICE: A sparse covariance-based

estimation method for array processing,” IEEE Trans. Signal Process., vol.

59, no. 2, pp. 629–638, Feb. 2011.

B. Ottersten, P. Stoica, and R. Roy, “Covariance matching estimation

techniques for array signal processing applications,” Digital Signal

Process., vol. 8, no. 3, pp.185–210, 1998.

Z. Tian, Y. Tafesse, and B. M. Sadler, “Cyclic feature detection with sub-

Nyquist sampling for wideband spectrum sensing,” IEEE J. Sel. Topics

Signal Process., vol. 6, no. 1, pp. 58–69, Feb 2012.

G. Leus and Z. Tian, “Recovering second-order statistics from compressive

measurements,” in Comput. Advances in Multi-Sensor Adaptive Process.

(CAMSAP), 2011 4th IEEE Int. Workshop on, Dec. 2011, pp. 337–340.

D. D. Ariananda and G. Leus, “Non-uniform sampling for compressive

cyclic spectrum reconstruction,” in Inf. Theory Appl. Workshop, May

2014, pp. 41–45.

134

References

D. D. Ariananda, D. Romero, and G. Leus, “Compressive angular and

frequency periodogram reconstruction for multiband signals,” in IEEE Int.

Workshop Comput.Advances Multi-Sensor Adaptive Process (CAMSAP),

San Martin, Dec. 2013.

——, “Cooperative compressive power spectrum estimation,” in IEEE

Sensor Array Multichannel Signal Process. Workshop (SAM), A Corunha,

Spain, Jun. 2014.

——, “Compressive periodogram reconstruction using a multibin

approach,” Submitted to IEEE Trans. Signal Process., 2014, Online:

http://arxiv.org/abs/1407.4017.

Q. Ling and Z. Tian, “Decentralized sparse signal recovery for compressive

sleeping wireless sensor networks,” IEEE Trans. Signal Process., vol. 58,

no. 7, pp. 3816–3827, July 2010.

——, “Decentralized support detection of multiple measurement vectors

with joint sparsity,” in Inf. Theory Appl. Workshop, May 2011, pp. 2996–

2999.

Documents

Compressive Covariance Sensing - EURASIP · Compressive Covariance Sensing Covariance Estimation and Detection o Covariance Estimation Least Squares Maximum Likelihood Modal Analysis