Upload
dylon-hayley
View
224
Download
0
Tags:
Embed Size (px)
Citation preview
Introduction to Subspace Tracking
IEEE AES Dallas Chapter MeetingRichardson, TX
Feb 28, 2006
Scott BakerL-3 Communications
Greenville, TX [email protected]
Outline
Introduction Time Series Data Model Subspaces & the Subspace Tracking Idea Subspace Tracking Categories A Secular Equation Subspace Tracker A Plane Rotation Subspace Tracker Simulations
Data Model
The time-series model for a sum of complex sinusoids in noise may be written
Q
qne
nqjeqsnx1
)()(
This time series may be placed in the vector form
)()(
)(
)2(
)1(
)( n
nx
x
x
n esωAx
And in a matrix form
)()(2)(1
)1()1(2)1(1
)(nQjenjenje
Qjejeje
ωA
Type?Singular?
Type?
NMNMN xxxX ,,,1 Size?Looks like?
Data Model
Two Typical Goals in Time Series Estimation 1)Parameter estimation
Number of signals Q The frequencies themselves The signal amplitudes
2)Noise reduction Find a cleaner version of x(n)
Typical assumptions Signal model is stationary over a block Noise is pre-whitened as needed N >= M (more snapshots than sensors)
Data Model
The data matrix is often place in form of a correlation matrix Data matrix forms lead to an SVD Correlation matrix forms lead to an EVD
HVΣUX HH VΛVXXR
where
MHH
NHH
IVVVV
IUUUU
What is U?
What is V?
Data Model
The singular value decomposition can be seen in more detail as
HM
H
H
M
NH
v
v
v
0
uuuVΣUX 2
1
2
1
21
N x N
N x M M x M
ui are unitary left singular vectorsvi are unitary right singular vectorsi are singular values
Data Model
The eigenvalue decomposition can be seen in more detail as
M x M
M x MM x Mvi are unitary right eigenvectors
i are eigenvalues
HM
H
H
M
MH
v
v
v
vvvVΛVR 2
1
2
1
21
How is the SVD and EVD related? How are the singular values and eigenvalues related?How are the singular vectors and eigenvectors related?How can we interpret an eigenvalue and an eigenvector?
Data Model
0 10 20 30 40 50 60 70 80 90 100-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
0 5 10 15 20 25 30 35 40-0.25
-0.2
-0.15
-0.1
-0.05
0
0.05
0.1
0.15
0.2
0.25
M = 40;x = sin(2*pi*1 * [1:100]/22); % 1 Hz sinewave, figure; plot(x)X = datamtrx(x, M);R = X'*X;[V, L] = svd(R);figure; plot(V(:,1));hold on; plot(V(:,2))
>> disp(L(1:3,1:3)) 693.4222 0 0 0 531.3492 0 0 0 0.0000
Figure 1 - x Figure 2 – V (1st two)
Box 1
Box 2N=61Period = 22 samples
0 5 10 15 20 25 30 35 40-1
-0.5
0
0.5
Figure 3 – V (last 38)
Why not one?
Subspaces & Tracking
From this example, consider a partition of the EVD
HN
H
N
SN
H
V
VΛ
ΛVVVΛVR S S
Signal Eigenvectors
NoiseEigenvectors
SignalEigenvalues “large”
NoiseEigenvalues “small”Span?
VS and A ?
Subspaces & Tracking
Extend our previous example, by adding noise
0 10 20 30 40 50 60 70 80 90 100-3
-2
-1
0
1
2
3
M = 40;x = sin(2*pi*1 * [1:100]/22); % 1 Hz sinewave, x = x + randn(1,100);figure; plot(x)X = datamtrx(x, M);R = X'*X;[V, L] = svd(R);figure; plot(V(:,1));hold on; plot(V(:,2))
Box 1
Only onechange
0 5 10 15 20 25 30 35 40-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0 5 10 15 20 25 30 35 400
100
200
300
400
500
600
700
Figure 1 - x Figure 2 – V (1st two) Figure 3 – Eigenvalues
Effect?
Meaning?
Knowledge of the signal eigenvectors VS gives us knowledge of the frequency matrix A
Knowing the span of VS is equivalent to knowing the span of VN
One can show that VS VSH + VNVN
H = I
MUSIC The MUSIC algorithm uses VN to estimate the
frequencies It is possible to show VN
H A = 0
Which the same as VNH [a1,a2,…,aQ] = 0
Subspaces & Tracking
Recall that the frequency matrix had the form
)()(2)(1
)1()1(2)1(1
)(nQjenjenje
Qjejeje
ωA
Subspaces & Tracking
So, we know the structure of a column and if we had the qth frequency, we could construct the qth column
We do know that VNH aq = 0 and we know VN from the
decomposition of the data To find q, what can we do?
Yup, good guess! We only need to build a sweep vector a()
for from DC to ½ the sample frequency We can plot the inverse instead so that
peaks are at the Q frequencies S() = 1 / || VN
H a()||
S() is then the frequency spectrum with peaks at the Q frequencies
Way cool!
Subspaces & Tracking
Subspaces & Tracking
So how do we do this again? Step 1: Build a data matrix
NMNMN xxxX ,,,1 Step 2: Build a correlation matrix and find its EVD
HH VΛVXXR Step 3: Sort the eigenvalues largest to smallest and partition the
EVD into signal portion & noise portion
NVVV S
S() = 1 / || VNH a()||
Step 4: Compute the MUSIC spectrum & locate Q peaks
Subspaces & Tracking
randn('state',1);
n=[0:99];
x=exp(i*pi/2*n)+2*exp(i*pi/4*n)+exp(i*pi/3*n)+randn(1,100);
R=corrmtx(x,12,'mod');
pmusic(R,3,'whole')
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8-30
-20
-10
0
10
20
30
40
50
60
Normalized Frequency ( rad/sample)
Pow
er (dB
)
Pseudospectrum Estimate via MUSIC
S()
MUSIC Spectrum
(type ‘help pmusic’)
So are we done yet? In stationary cases, yes In non-stationary cases, no
The Q frequencies are often time-varying Hopping sigals Chirping signals Any kind of frequency modulated signal
It is too expensive to compute an EVD at every instant of time
Instead, use a subspace tracker to track the changes in VS (or equivalently VN) as new data comes in
Subspaces & Tracking
Let’s slightly modify our data model so that it is recursive
Subspaces & Tracking
HN
H
N
SN
V
VΛ
ΛVVR S S
HxxRR 1~
The factor is called a fading factor and is slightly less than unity
The effective window size is B = 1 / (1-) The update assumes that we know the old EVD
What type of update is this called?
HN
H
N
SN
V
V
Λ
ΛVVR ~
S~
~
~
~S
~~
And we wish to know the new EVD Do we needthe wholeEVD?
Subspace Tracking Categories
There are about 250 published papers on subspace tracking It sky-rocketed after the invention of the MUSIC algorithm There are several broad categories
Secular equation (e.g., Bunch) Plane rotation (e.g., Businger) Optimization Lanczos Perturbation Power Method & Subspace Iteration Eigenvector computation with a priori eigenvalues Analog or Continuous Time Hybrid Techniques
The Secular Equation Subspace Tracker
Golub discovered the secular equation in 1971
Can be considered the first eigen- based subspace tracker (Businger wrote the first overall ST in 1970)
Reveals a nice relation between original and updated eigenvalues
Seven years later, Bunch, Nielsen and Sorensen introduced the classic secular equation subspace tracker
A keen observation is that the eigenvalues of the new correlation are larger assuming the data has non-zero energy
MM ~
22~
11~
Gene Golub (Stanford)
updated eigenvalue, largest
original eigenvalue, largest
The Secular Equation Subspace Tracker
This observation is important because it was noted that the rank-1 modification of the correlation can be written as the rank-1 modification of a diagonal matrix
HxxRR 1~
HH xxVΛVR 1 ~
Stick in the EVD
HHH VVxxVΛVR ]1 [~
Because V is?
HH VzzΛVR ]1 [~
And z is?
What is left in the brackets then?Recall the monotonicity relation
The Secular Equation Subspace Tracker
HVDVR ~
HHVQΒVQR ~
HzzΛD 1 where
Is D a diagonal matrix?Note that we can form the EVD
HQΒQD And so the resultant EVD of the updated correlation is
Note we have the form
V~ HVΛVR
~
~
~~
Remember the EVD…
Λ~
related? ~
and of seigenvalue theare how So RD
The Secular Equation Subspace Tracker
HzzΛD 1
Answer: They are equal
Hence, to find the updated eigenvalues, we can find the eigenvalue of D.
Recall from basic linear algebra, we can write
bDb
These two expressions can be used to find a formula for the updated eigenvalues
0)det( bDb
Combining these three equations leads to the secular equation
The Secular Equation Subspace Tracker
M
i i
iz
1 )(
2)1(1)(
1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6-5
-4
-3
-2
-1
0
1
2
3
4
5
Original eigenvalue (p)
Updated eigenvalue (z)
The secular equation has zeros & poles at the eigenvalue locations
)(
Efficient to curve fithere between poles
EigenvectorComputation(i.i. not needed)IDD
zD
zVDv
ii
i
ii
~
21
1~
A Plane Rot. Subspace Tracker
The first subspace tracker developed in 1970 P. Businger of Bell Labs Used plane rotations Impetus was updating a least squares
problem SVD based solution Businger acknowledged the problem was
posed by G. Dahlquist and related by G. Golub
How might we proceed to update an SVD?
A Plane Rot. Subspace Tracker
HH
HHH
H
HH
H
H
H
H
VZQI0
0U
VZZVx
ΣQQ
I0
0U
VVx
Σ
I0
0U
x
VΣU
x
XX
VΣUX
~
)1(
)1(
)1(
)1(
~
What is Q? Z?
Is this in SVD form?
What structure does thisfactor have?
Adding a row
Similar trickto what?
The Q and Z matrices are called plane rotation matrices
The specific type of plane rotation used here is called a Given’s rotation
A Plane Rot. Subspace Tracker
0
* r
b
a
cs
sc
222 |||||| bar
0 cbsa 122 sc
22
2
ba
ac
c
a
bs
Real Given’s Rotation
So how do we use Given’s rotations to diagonalize the matrix? We have to be careful Eliminating one element in a column / row may perturb
another column / row Need to apply Given’s rotations to both rows and columns
A Plane Rot. Subspace Tracker
***b
a 0 x0
xrx
xQ =
Given’sMatrix
OriginalMatrix Transformed
Matrix
*
****
**
*
Q1=
*
**0 **
*
*
x
*
**
*x
0 0 0 0
xx=
*
**
*
*Q3
0 0 *
A Plane Rot. Subspace Tracker
Left(Row)Given’sRotationsFirst
What is theresultantform?
A Plane Rot. Subspace Tracker The bidiagonal form is reduced to a diagonal form The process is iterative
Start with column rotations (right) Then a set of row rotations (left) Repeat until the energy is minimal off the main diagonal
HHVZQI0
0UX
~~
So, going back to our SVD update
Q and Z are series of Given’s rotation matrices Q1Q2...QL Z1Z2….ZP
And so our final result then is in the proper SVD form
HVΣUX~
~
~~
A Plane Rot. Subspace Tracker
Example Simulation
Single signal SNR is 15 dB Sample Rate = 640MHz Linear FM signal
140 to 160 MHz 20MHz / 1000 samples 12.8 GHz / second
= 0.975 (80 samples) TQR + root-MUSIC Spline Interpolation
Polynomial = feature
500 1000 1500 2000 2500 3000 3500 4000 4500 5000
1.42
1.44
1.46
1.48
1.5
1.52
1.54
1.56
1.58
1.6
1.62x 10
8 Freq Track w/ MTH=5, SNR=15, EBS = 40
freq
uenc
y
time sample
100 200 300 400 500 600 700 800 900
1.42
1.44
1.46
1.48
1.5
1.52
1.54
1.56
1.58
1.6
x 108
This is a splinefit to the first ramp.
0 100 200 300 400 500 600 700 800 900 1000
1.4
1.45
1.5
1.55
1.6
x 108 Freq Track w/ MTH=5, SNR=15, EBS = 40
frequency
time sample
0 200 400 600 800 1000 1200 1400 1600 1800 2000
1
1.1
1.2
1.3
1.4
1.5
1.6
x 108 Freq Track w/ MTH=5, SNR=15, EBS = 40
frequency
time sample
Other Examples