View
217
Download
0
Category
Preview:
Citation preview
Clustering by Pattern Similarity in Large Data Sets
Haixun Wang, Wei Wang, Jiong Yang, Philip S. YuIBM T. J. Watson Research CenterPresented by Edmond Wu
DB-Seminar Slide 2
Talk Outline
Introduction
Related Work
pCluster Model
Performance Analysis
Conclusion
DB-Seminar Slide 3
Motivation
Why discovery of clusters based on pattern similarity is interesting and important?
DNA micro-array analysis
E-commerce: Recommendation systems & target marketing
DB-Seminar Slide 4
Background Knowledge
Clustering: the process of grouping a set of objects into classes of similar objects.
Subspace clustering: discovering clusters embedded in the subspace of a high dimensional datasets.
Pattern similarity: coherent pattern on a subset of dimensions. ( Not require to have close values on at least one attribute)
DB-Seminar Slide 5
Example of Similar pattern on a subset of dimensions
DB-Seminar Slide 6
Challenges
Identifying subspace clusters in high-dimensional data sets is difficult.
Traditional distance functions can not capture the pattern similarity among the objects.
DB-Seminar Slide 7
How to detect shifting pattern?
Given N attributes a1,…,an
Define a derived attribute Aij=ai-aj for every
pair of attributes ai-aj Thus, the problem equals to mine subspace clusters on the objects with the derived set of attributes.
Drawback: The converted dataset will have
N(N-1)/2 dimensions
intractable even for a small N
DB-Seminar Slide 8
Related Work
Bicluster Model (Cheng et al):
AIJ: sub Matrix of a DNA array, with the following mean squared residue score H(I,J):
δ- bicluster: AIJ is called a δ- bicluster if H(I,J) ≤δ
DB-Seminar Slide 9
Bicluster Model (Example)
(1) Shifting pattern (2) Scaling patternH(I,J)=0 H(I,J)=2/3
(3) Not similar pattern (4) Submatrix of (2)
H(I,J)=8 H(I,J)=2.25>2/3
If we set δ=2, (3),(4) are not δ- bicluster.
a1 a2 a3
O1 1 2 3
O2 5 6 7
a1 a2 a3
O1 1 2 4
O2 2 4 8
a1 a2 a3
O1 2 4 12
O2 4 6 2
a1 a3
O1 1 4
O2 2 8
DB-Seminar Slide 10
Drawbacks of Bicluster Model
A submatrix of a δ- bicluster is not necessarily a δ- bicluster.
Not sure to find all qualified clusters (randomly greedy algorithm provides only an approximate answer).
Can not exclude outlier in a bicluster.
Difficulties in designing efficient algorithm.
DB-Seminar Slide 11
Bicluster Model (Example)
The bicluster shown in Figure (a) contains an obvious outlier but it still has a fairly small mean squared residue (4.238).
If we get rid of such outliers by reducing the δ threshold, it will exclude many biclusters which do exhibit similar patterns.
DB-Seminar Slide 12
The pCluster Model
pScore of a 2× 2 matrix:
O : subset of objects in the database
T : subset of attributes; (O,T): submatrix of dataset
δ: user specified clustering threshold
dxa: value of object X on attribute a
Given x, y O, and ∈ a, b ∈T
)()( ybyaxbxaybya
xbxadddd
dd
ddpScore
DB-Seminar Slide 13
The pCluster Model (Cont.)
pScore(X) ≤ δ means that the change of values on the two attributes between the two objects in X is confined byδ, a user-specified threshold.
Pair (O, T ) forms a δ-pCluster if for any 2 × 2 submatrix X in (O, T ), we have pScore(X) ≤ δ for some δ ≥ 0.
DB-Seminar Slide 14
The pCluster Model (Example)
In Figure (a): Object 2, 3 and {b, c} form a 2× 2 submatrix X: d2b=12, d2c=15, d3b=40, d3c=43 pScore(X)=|(12-15)-(40-43)|=0
Objects 1,2,3 and {b,c,h,j,e} form a pCluster (δ=0)
DB-Seminar Slide 15
The pCluster Model (Cont.)
Compact property of pCluster:
let (O,T) be a δ-pCluster. Any of its submatrix, (O’,T’) is also a δ-pCluster (Based on the definition of pCluster);
The volume of a pCluster: |O|×|T|;
Definition of pCluster is symmetric:
|(dxa－ dxb) － (dya－ dyb)|
= |(dxa－ dya) － (dxb－ dyb)|
DB-Seminar Slide 16
Problem Statement
Task: To find all pairs (O,T) such that (O,T) is a δ-pCluster according to its definition, and |O|≥ nr, |T|≥ nc.
Parameters: D : dataset δ: a cluster threshold nc : a minimal number of columns nr : a minimal number of rows
DB-Seminar Slide 17
The Algorithm
Definition of MDS: Assuming c = (O, T) is a δ-pCluster. Column set T is a Maximum Dimension Set (MDS) of c if
there does not exist T’ T such that (O, T’) is also a δ-pCluster.
Objects can form pClusters on multiple MDSs. The algorithm is depth-first, meaning only generate pClusters that cluster on MDSs.
DB-Seminar Slide 18
Pair-wise Clustering
Pairwise Clustering Principle:
Given objects X and Y, and a dimension set T, X and Y form a δ-pCluster on T iff the difference between the largest and smallest value in
S(X, Y, T) is below δ.
In other word, ({X,Y},T) is a pCluster if the following is true:
),(max,
bafTba
)()(),( ybxbyaxa ddddbaf
DB-Seminar Slide 19
Pair-wise Clustering (Example)
Sorted sequence of S(X, Y, T) =s1,…,sk ,…,sn
Object x and y forms a δ-pCluster if
Three MDSs were found: {e,g,c}, {a,d,b,h}, {h,f}
1,...,1, niss ik
DB-Seminar Slide 20
MDS Pruning
MDS Pruning Principle:
Let Txy be an MDS for objects x, y, and a ∈Txy. For any O and T , a necessary condition of ({x, y} ∪O, {a} ∪ T ) being a δ-pCluster is b ∈ T , Oab {x, y}.
The pruning criterion can be stated as follows:
For any dimension a in a MDS Txy, count the number
of Oab that contain {x, y}. If the number of such Oab is
less than nc-1, remove a from Txy. Furthermore, if the
removal of a makes |Txy| < nc, we remove Txy as well.
DB-Seminar Slide 21
MDS Pruning (Example)
DB-Seminar Slide 22
The Main Algorithm
First step: Scan the dataset to find column-pair MDSs and object-pair MDSs.Second step: Prune object-pair MDSs and column-pair MDSs by turn until no pruning can be made.Third step: Insert the remaining object-pair MDSs into a prefix tree. (Each node represents a cluster of objects, each edge represents the column selected)
DB-Seminar Slide 23
Construct a prefix tree
Sort the order of columns e.g., a,b,c,…Insert 2-object pCluster(O,T) into the prefix tree. Perform a post-order traversal of the prefix tree. Prune nodes that |O|<nr. ( Add the objects in O to nodes whose column set
T’ T and |T’|=|T|-1
DB-Seminar Slide 24
Construct a prefix tree (Example)
DB-Seminar Slide 25
Algorithm Complexity
Main algorithm for mining pClusters has time complexity :
where M is the # of columns and N is the # of
objects.The worse case:However, the complexity can be greatly reduced because of the MDS pruning process.
)loglog( 22 MMNNNMO
)( 22NkMO
DB-Seminar Slide 26
Experiments
DatasetsSynthetic datasets (parameters: different nr, nc, # of embedded perfect pCluster with δ=0)
Gene expression data (yeast microarray)
MovieLens dataset (E-commerce)
DB-Seminar Slide 27
Performance Analysis
Response time VS. data size
DB-Seminar Slide 28
Performance Analysis (Cont.)
Sensitiveness to mining parameters: δ, nc, and nr
DB-Seminar Slide 29
Performance Analysis (Cont.)
Compare the pCluster with an alternative approach based on the subspace clustering algorithm CLIQUE.
DB-Seminar Slide 30
Performance Analysis (Cont.)
The pruning process is essential in the pCluster algorithm.
Without pruning, the pCluster Algorithm can not beyond 3,000 objects. As the number of the MDS become too large to put into a Prefix tree.
DB-Seminar Slide 31
Conclusion
pCluster Model: capture the closeness of objects and pattern similarity among the objects in subsets of dimensions.Advantages :
-Discover all the qualified pClusters. -The depth-first clustering algorithm avoids generating clusters which are part of other clusters. -More efficient than current algorithm. -Resilient to outliers
DB-Seminar Slide 32
References
Y. Cheng and G. Church. Biclustering of expression data. In Proc. of 8th International Conference on Intelligent System for Molecular Biology, 2000.S. Tavazoie, J. Hughes, M. Campbell, R. Cho, and
G. Church. Yeast micro data set, 2000. In http://arep.med.harvard.edu/biclustering/yeast.matrix,
R. C. Agarwal, C. C. Aggarwal, and V. Parsad. Depth first generation of long patterns. In SIGKDD, 2000.J. Yang, W. Wang, H. Wang, and P. S. Yu. δ-clusters:
Capturing subspace correlation in a large data set. In ICDE, pages 517–528, 2002.
Thanks!!
Recommended