Embed Size (px)
Citation preview
Some words on Stochastic Eigen-Analysis
Alan Edelman Raj Rao
Dept of MathematicsComputer Science & AI Laboratories
MITJuly 10, 2006
Some words on Stochastic Eigen-Analysis
Alan Edelman Raj Rao
Dept of MathematicsComputer Science & AI Laboratories
MITJuly 10, 2006
Just when you thought mathematics just about
wrapped up …
1. Orthogonal Polynomials & Special Functions
2. Convolutions3. Stochastic Differential Operators
Just when you thought mathematics just about
wrapped up …
1. Orthogonal Polynomials & Special Functions
2. Convolutions3. Stochastic Differential Operators
Pre & Early Computer Days
The Bateman Manuscript
Project
The web era
The SEA era
Ahead of its time Orthogonal Polynomials & Random Matrices:
A Riemann-Hilbert Approach MOPS: Dumitriu
Koev
Anshelevich (Free Meixner poly.)
Chikuse (Statistics on manifolds)
Just when you thought mathematics just about
wrapped up …
1. Orthogonal Polynomials & Special Functions
2. Convolutions3. Stochastic Differential Operators
Classical Convolutions
Plus
-3 -2 -1 0 1 2 30
0.05
0.1
0.15
0.2
0.25
0.3
0.35
x
Prob
abili
ty
0 0.5 1 1.5 2 2.5 30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
Pro
babi
lity
Y=randn(n,2n)B=Y*Y’
zm2+(2z-1)m+2=0
+
X =randn(n,n)A=X+X’
m2+zm+1=0
-2 -1 0 1 2 3 40
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
x
Pro
babi
lity
A+Bm3+(z+2)m2+(2z-1)m+2=0
Times
-3 -2 -1 0 1 2 30
0.05
0.1
0.15
0.2
0.25
0.3
0.35
x
Prob
abili
ty
0 0.5 1 1.5 2 2.5 30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
Pro
babi
lity
X =randn(n,n)A=X+X’
m2+zm+1=0
Y=randn(n,2n)B=Y*Y’
zm2+(2z-1)m+2=0
*
-2 -1 0 1 2 3 40
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
x
Pro
babi
lity
A*Bm4z2-2m3z+m2+4mz+4=0
-3 -2 -1 0 1 2 30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
x
Pro
babi
lity
The convolutions (Free Prob)
Spectrum of Sample Covariance Matrix
-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 40
0.2
0.4
0.6
0.8
1
1.2
1.4
x
Pro
babi
lity
c = 0.5c = 0.1
- Convolution is highly non-linear
- Density is function of Sensors/Snapshots, eig(R)
- Symbolic package (RMTool) to compute density
. Moments (canonically) in closed form!
1 3
0.4
0.6
eig(R) Marcenko-Pastur SCM spectrum
0 1 2 3 4 5 6 7 80
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
x
Pro
babi
lity
c = 0.5c = 0.1
Eigenvalues of true covariance matrix
Eigenvalues blurred (free convolution)
R^ = R1/2 W(c) R1/2
Two distinct subspaces
Blurring of eigenvaluesbecause of insufficient sample support
Free “Deconvolving” of a singleSample Covariance Matrix
There is no structure visible to the eye, but the subspace structure can be deduced by free deconvolution
Eigenvalues blur because of limited data
“Convolution” <-> “Deconvolution”
• Model based– moment matching + “second” order freeness
(with Speicher + Mingo)– parametric
• Non-model based– Stieltjes transform-to-resolvent matching– Connection to Lanczos, GMRES– (with Per-Olof Persson)– non-parametric
Free probability in SEA’06
• Speicher (Survey)• Chatterjee (Rate of convergence)• Speicher + Mingo (Fluctuations & 2nd order
freeness)• Anshelevich (Free Meixner polynomials)• Demni (Processes)• Burda (Free Levy matrices)• Kargin (Large deviations)• Rashidi Far (Operator values free probability)
Just when you thought mathematics just about
wrapped up …
1. Orthogonal Polynomials & Special Functions
2. Convolutions3. Stochastic Differential Operators
Stochastic Operators
• Ito & Stratonovich• Many recent methods• Whole Field of stochastic differential
equations–MATLAB SPEAK:
• rand + “\” well studied• rand + “eig” missing
Stochastic Operator Limit
,
N(0,2)χχN(0,2)χ
χN(0,2)χχN(0,2)
nβ21~H
β
β2β
2)β(n1)β(n
1)β(n
βn
⎟⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜⎜
⎝
⎛
−−
−
,dWβ
2x dxd
2
2
+−
,Gβ
2HH nnβn +≈ ∞
… … …
21
Those betas• Real Numbers: x β=1• Complex Numbers: x+iy β=2• Quaternions: x+iy+jz+kw β=4• β=2½? x+iy+jz
Other (Math) Talks in SEA’06
• Appearance of “universal” distributions– Kuijlaars, Baik, Johnstone, El Karoui, Dieng,
Sutton, Rider, Sasamoto, Seba
• Causal sets, airplane boarding– Bachmat
• Complex systems– Ergün, Sethna, Timme
• Principal component analysis– Paul, Onatski
• Applications & more!
