Upload
evelyn-phillips
View
221
Download
1
Tags:
Embed Size (px)
Citation preview
Data Mining Course 2
OutlineOutline
• Manifold and Manifold Learning
• Classical Dimensionality Reduction
• Semi-Supervised Nonlinear Dimensionality Reduction
• Experiment Results
• Conclusions
Data Mining Course 7
Manifold learningManifold learning• Raw format of natural data is often
high dimensional, but in many cases it is the outcome of some process involving only few degrees of freedom.
Data Mining Course 8
Manifold learningManifold learning
• Intrinsic Dimensionality Estimation
• Dimensionality Reduction
Data Mining Course 9
Dimensionality ReductionDimensionality Reduction• Classical Method:
Linear: MDS & PCA (Hastie 2001)
Nonlinear: LLE (Roweis & Saul, 2000) , ISOMAP (Tenebaum 2000), LTSA (Zhang & Zha 2004)
-- in general, low dimensional coordinates lack physical meaning
Data Mining Course 10
Semi-supervised NDRSemi-supervised NDR• Prior information
Can be obtained from experts or by performing experiments
Eg: moving object tracking
Data Mining Course 11
Semi-supervised NDRSemi-supervised NDR• Assumption:
Assuming the prior information has a physical meaning, then the global low dimensional coordinates bear the same physical meaning.
Data Mining Course 13
Basic LTSABasic LTSA• Characterized the geometry by
computing an approximate tangent space
Data Mining Course 14
SS-LLE & SS-LTSASS-LLE & SS-LTSA• Give m the exact mapping data
points .• Partition Y as • Our problem :
Data Mining Course 15
SS-LLE & SS-LTSASS-LLE & SS-LTSA• To solve this minimization problem,
partition M as:
• Then the minimization problem can be written as
Data Mining Course 16
SS-LLE & SS-LTSASS-LLE & SS-LTSA
• Or equivalently
• Solve it by setting its gradient to be zero, we get:
Data Mining Course 17
Sensitivity AnalysisSensitivity Analysis• With the increase of prior points, the
condition number of the coefficient matrix gets smaller and smaller, the computed solution gets less sensitive to the noise in and
Data Mining Course 18
Sensitivity AnalysisSensitivity Analysis• The sensitivity of the solution
depends on the condition number of the matrix
Data Mining Course 19
Inexact Prior InformationInexact Prior Information• Add a regularization term, weighted
with a parameter
Data Mining Course 20
Inexact Prior InformationInexact Prior Information• Its minimizer can be computed by
solving the following linear system:
Data Mining Course 21
Experiment ResultsExperiment Results• “incomplete tire”
--compare with basic LLE and LTSA--test on different number of prior points
• Up body tracking --use SSLTSA--test on inexact prior information
algorithm
Data Mining Course 24
Relative error with different Relative error with different number of prior pointsnumber of prior points
Data Mining Course 27
Results of inexact prior Results of inexact prior information algorithminformation algorithm
Data Mining Course 28
ConclusionsConclusions• Manifold and manifold learning
• Semi-supervised manifold learning
• Future work