Upload
vidal
View
48
Download
0
Tags:
Embed Size (px)
DESCRIPTION
The second-round discussion* on. Geometric diffusions as a tool for harmonic analysis and structure definition of data. By R. R. Coifman et al. * The first-round discussion was led by Xuejun; * The third-round discussion is to be led by Nilanjan. Diffusion Maps. Purpose - PowerPoint PPT Presentation
Citation preview
Geometric diffusions as a tool for harmonic analysis and structure definition of dataBy R. R. Coifman et al.
The second-round discussion* on
* The first-round discussion was led by Xuejun;
* The third-round discussion is to be led by Nilanjan.
Diffusion Maps
• Purpose - finding meaningful structures and geometric descriptions of a data set X.
- dimensionality reduction
• Why?
The high dimensional data is often subject to
a large quantity of constraints (e.g. physical laws)
that reduce the number of degrees of freedom.
• Markov Random Walk
KDxxKxxK
xxpaA
jji
jiijij
1
),(),(
)]|([][
• Symmetric Kernel
ji
jji
iji
jiij a
xxKxxK
xxKa ~
),(),(
),(~
Diffusion Maps
Many works propose to use first few eigenvectors of A as a low representation of data (without rigorous justification).
2/12/1~ ADDA
• Relationship
• Diffusion maps
)(
)(
)( 11
00
x
x
x m
m
m
• Spectral Decomposition of A
Diffusion Maps
n
jninnij xxa )()(~ 2
where 01 210
• Spectral Decomposition of Am
n
jninmn
mij xxa )()(~ 2)(
• Diffusion distance of m-step
2
)()()(,
2
)()(
~2~~)(
jmim
mij
mjj
miijim
xx
aaaxxD
• Interpretation
dzzxazxaxxD jm
im
jim
2)2/()2/(,
2 ),(~),(~)(
Diffusion Distance
The diffusion distance measures the rate of connectivity between xi and xj by paths of length m in the data.
Diffusion vs. Geodesic Distance
),(),( .. BCDBAD geodgeod ),(),( BCDBAD mm
Data Embedding
)(,),(),(001100 xxxxx d
md
mm
• By mapping the original data into (often )
• The diffusion distance can be accurately approximated
dRx 0dRx0dd
)(1),( 22 mm eOyxyxD
Example: curves
Umist face database: 36 pictures (92x112 pixels) of the same person being randomly permuted.
Goal: recover the geometry of the data set.
.,0)cos(
on tosimilar very a graph obtains oneordered,- re isnumbers ofset this When image.each to
numbera real assigns ioneigenfunct second The 1
t
Original ordering
Re-ordering
The natural parameter (angle of the head) is recovered, the data points are re-organized and the structure is identified as a curve with 2 endpoints.
Original set: 1275 images (75x81 pixels) of the word “3D”.
Example: surface
Diffusion Wavelet
• A function f defined on the data admits a multiscale representation of the form:
• Need a method compute and efficiently represent the powers Am.
1
0 iondecompositwavelet
1
scalecoarsest
00
s
s
sss fAAfAf
• Multi-scale analysis of diffusion
Discretize the semi-group {At:t>0} of the powers of A at a logarithmic scale
which satisfy
Diffusion Wavelet
Diffusion Wavelet
• The detail subspaces
• Downsampling, orthogonalization, and operator compression
- diffusion maps: X is the data set
}:{ , jkjj Xk
}:{0 Xkk
A - diffusion operator, G – Gram-Schmidt ortho-normalization, M - AG
Diffusion multi-resolution analysis on the circle. Consider 256 points on the unit circle, starting with 0,k=k and with the standard diffusion. Plot several scaling functions in each approximation space Vj.
Diffusion multi-resolution analysis on the circle. We plot the compressed matrices representing powers of the diffusion operator. Notice the shrinking of the size of the matrices which are being compressed at the different scales.
Multiscale Analysis of MDPs
[1] S. Mahadevan, “Proto-value Functions: Developmental Reinforcement Learning”, ICML05.
[2] S. Mahadevan, M. Maggioni, “Value Function Approximation with Diffusion Wavelets and Laplacian Eigenfunctions”, NIPS05.
[3] M. Maggioni, S. Mahadevan, “Fast Direct Policy Evaluation using Multiscale Analysis of Markov Diffusion Processes”, ICML06.
To be discussed a third-round led by Nilanjan
Thanks!