Object Specific Compressed Sensing by minimizing a weighted L2-norm

A. Mahalanobis

Background

Lockheed Martin has been working on the DARPA ISP program Team includes Duke, JHU, Yale and NAVAIR

An adaptive sensing scheme has been developed that allocates sensor resources (spectral and spatial) based on relevant information content Algorithms are currently working in a coded aperture hyperspectral imager

hardware

Compressed Sensing is a natural extension of this ISP concept

Motivation

Can we create an efficient sensing process where objects of interest are well resolved, but other parts of the scene are heavily compressed? Economize on number of data measurements required and the

computations needed to reconstruct the image

Currently, Compressed Sensing is focused on the general reconstruction problem We are not interested in the perfect reconstruction of the whole scene

Our approach embeds pattern recognition objectives (detection, discrimination) and compression in the sensing process, while producing visually meaningful images.

Approach

It has been shown that under certain conditions, minimizing the L-1 Norm yields the optimum solution for perfect reconstruction, but the optimization requires iterative (potentially cumbersome) techniques

L-2 norm techniques are well known, analytical closed form solutions that are easy to implement Computationally attractive for the formation of large images However, the minimum L2 norm solution does not yeild good

reconstruction

Can a weighted L2-norm arrive “close” to the optimum solution when we are interested in specific objects ? How can we incorporate prior knowledge about the objects ?

The general solution Assume that the image vector y can be represented as linear combination of basis

vectors (columns of the matrix A) such that h is the coefficient vector we seek to estimate from a small number of measurements, and

hence re-construct y

In compressed sensing, we measure a smaller vector u, (i.e. the projection of the image y through a “random” mask W)

The most general family of solution for the estimate h that satisfies the above linear constraints is

All solutions (including those which minimize the L-0, L-1 or L-2 norm) belong to this family The particular solution is the “minimum L-2 norm” solution The homogeneous solution can be viewed as a correction to the L-2 norm that results in other solutions

with different properties

WAXuhX

uAhW

uyW

TT

T

T

where

Ahy

zIXXXXuXXXh

TTT 11

Particular solution Homogeneous Solution

A random vector

Weighted L-2 norm Minimizing the L-2 norm does not relate to a well-defined “information”

metric for reconstruction It minimizes the variation in the estimate when white noise is present in the

measurement

Rather, we seek a weighting that minimizes the L-2 norm of the coefficient vector while maximizing information about the objects of interest This results in attenuation of those weights which do not bear useful information

for reconstruction

Or maximize

This implies that the best choice for the weights is

We envision that can be calculated “apriori” from a set of representative images of the class of objects of interest, or a suitable statistical model may be used.

2

2 maximizing while minimize ii

idiidi

2

2

i

i

idi

idi

idJ

iid

1

id

Solution using the methods of Lagrange multipliers Problem is stated as

Minimize the quadratic subject to the linear constraints D is a diagonal matrix whose diagonal elements are calculated apriori

from a set of representative images or a statistical model

The well known solution for the estimate of the coefficient vector is now

h is estimate of the coefficients based on the measurements u A is a matrix that can be used as a basis to represent the image W is a random matrix on which the image is projected to obtain u D is a weight vector that maximizes information for the objects of interest

DhhT uhX T

2id

uWAADWWAD

uXDXXDh111

111

TTT

T

Reconstruction Equation The Reconstructed Image is given by

where depends on the basis functions and the weights

Without weights, R = I, and the solution does not depends on the underlying basis set Minimum L-2 norm solution is then simply

We will use i) DCT and ii) KL basis sets to demonstrate performance For the KL basis set, D is the same as the eigen-values

uRWWRW

uWAADWWAAD

Ahy

1

111

T

TTT

TAADR 1

uWWWy1

T

Example using ideal weights

Original image is 32 x 32 (1024 elements) DCT is used as a basis set

Any other basis set that allows compact representation can be used

ideal coefficients are used as a “place-holder” for weights In practice, these will be estimated representative images of the

class of objects of interest, or statistically modeled. Weighted L2 norm produces recognizable results using

1/4th the data (256 measurements) Conventional L2 norm does not perform well

K=256mse=0.19

K=192mse=0.25

K=64mse=0.5

K=256mse=0.86

Conventional L2 norm

WEIGTED L2 norm

Original 32 x 32 image

DCT Basis Set and Weights

The DCT of the image shows good compaction properties. Indicates it should be possible to achieve nearly zero mse with only 50% of the coefficients Other basis sets should yield much greater compactness

A

(as a 2D image)

Example 2: weights estimated for a “class”

The goal is to sense all objects that belong to a “class” Exact weights for any one image is not known, but an average estimate for the class is used

The average DCT is estimated using 1600 representative views and the inverse of the DCT coefficients is used weights in the reconstruction process

Object

DCT of Object Average DCT

Weighted vs. Conventional approach using DCT basis

Comparison of conventional and weighted minimum L2 norm reconstruction using the DCT basis functions. Weighting the reconstruction process makes a significant difference in the reconstruction error

Reconstruction based on DCT with and without weights

Reconstructions using 512 projections and the DCT basis set with weighting estimated over the class shows better performance than without weighting, i.e. the conventional minimum L2 norm solution

(a) Weighted(b) Unweighted

Using the K-L Basis set

The weights are the reciprocal of the square-root of the eigen-values of the auto-correlation matrix estimated using representative images of the class of vehicles of interest. Only M=450 basis functions are necessary for accurately representing the images, which reduces the

size of the matrix R and hence the overall computations

Weighted vs. Conventional approach using KL basis

Reconstruction using the KL basis far out-performs DCT when weights are used Performance of unweighted scheme is comparable to the unweighted DCT (not

surprising)

Other Computational advantages of the KL set

KL transform offers computational advantages in Two ways: Fewer measurements are necessary (reduces the number of rows of R) Fewer basis functions as required to represent the image (reduces the number of columns of R)

Image on the left was reconstructed using the first 450 eigen-vectors of the KL decomposition, whereas all 1024 were used on the right. The two images are almost identical, although the image in (a) requires considerably less computations.

(a) Esimated using 256 measurements and450 eigen-vectors

(b) Esimated using 256 measurements andAll 1024 eigen-vectors

Example of full scene reconstruction (back to DCT)

L2-norm approach easily reconstructs large scene Computationally straightforward

Weighted optimization clearly demonstrates ability to heavily compress uninteresting regions of the scene, while achieving reasonable reconstruction where true objects are present

Original Image

Summary Minimizing the L-2 norm is a viable way of reconstructing objects of

interest in a compressed sensing scheme Requires prior knowledge of the weights that are representative of the

class of objects

Embeds attributes of pattern recognition in the sensing process to preserve visual detail for the human user, while effectively achieving detection, discrimination and compression

Selection of basis set is important Good basis sets require fewer measurements and fewer terms in the

representation which speeds up the computations.

The selection of basis sets and criterion for choosing weights both require further research