View
217
Download
0
Category
Preview:
Citation preview
A Unified View of Kernel k-means, Spectral Clustering and
Graph Cuts
Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis
Outline
• (Kernel) kmean, weighted kernel kmean
• Spectral clustering algorithms
• The connect of kernel kmean and spectral clustering algorithms
• The Uniformed Problem and the ways to solve the problem
• Experiment results
K means and Kernel K means
Weighted Kernel k means
Matrix Form
Distance from ai to cluster c
Spectral Methods
• Represent the data by a graph– Each data points corresponds to a node on
the graph– The weight of the edge between two nodes
represent the similarity between the two corresponding data points
– The similarity can be a kernel function, such as the RBF kernel
• Use spectral theory to find the cut for the graph: Spectral Clustering
Spectral Methods
Spectral Methods
Similar in the cluster
Difference between clusters
Represented with Matrix
( , ) Tc c c clinks V V x Ax ( , \ ) T
c c c clinks V V V x Lx
| | Tc c cV x x ( ) T
c c cdegree V x Dx
L for Ncut
Ratio assoc
Ratio cut
Norm assoc
Weighted Graph CutWeighted association
Weighted cut
Conclusion
• Spectral Methods are special case of Kernel K means
Solve the unified problem
• A standard result in linear algebra states that if we relax the trace maximizations, such that Y is an arbitrary orthonormal matrix, then the optimal Y is of the form Vk Q, where Vk consists of the leading k eigenvectors of W1/2KW1/2 and Q is an arbitrary k × k orthogonal matrix.
• As these eigenvectors are not indicator vectors, we must then perform postprocessing on the eigenvectors to obtain a discrete clustering of the point
From Eigen Vector to Cluster Indicator
Normalized U with L2 norm equal to 1
2
1
The Other Way
• Using k means to solve the graph cut problem: (random start points+ EM, local optimal).
• To make sure k mean converge, the kernel matrix must be positive definite.
• This is not true for arbitrary kernel matrix
The effect of the regularizationai is in c
cai is not in
Experiment results
Results (ratio association)
Results (normalized association)
Image Segmentation
Thank you. Any Question?
Recommended