• Home

  • Documents

  • Determining the best K for clustering transactional datasets – A coverage density-based approach
13
Intelligent Database Systems Lab N.Y.U.S. T. I. M. Determining the best K for clustering transactional datasets – A coverage density-based approach Presenter : Lin, Shu-Han Authors : Hua Yan, Keke Chen, Ling Liu, Joonsoo Bae Data & Knowledge Engineering (DKE) 68 (2009) 28–48

Determining the best K for clustering transactional datasets – A coverage density-based approach

  • Upload
    olympe

  • View
    22

  • Download
    1

Embed Size (px)

DESCRIPTION

Determining the best K for clustering transactional datasets – A coverage density-based approach. Presenter : Lin, Shu -Han Authors : Hua Yan, Keke Chen, Ling Liu, Joonsoo Bae. Data & Knowledge Engineering (DKE) 68 (2009) 28–48. Outline. Motivation Objective Methodology - PowerPoint PPT Presentation

Citation preview

Page 1: Determining the best K for clustering transactional datasets – A coverage density-based approach

Intelligent Database Systems Lab

N.Y.U.S.T.I. M.

Determining the best K for clustering transactional datasets –

A coverage density-based approach

Presenter : Lin, Shu-HanAuthors : Hua Yan, Keke Chen, Ling Liu, Joonsoo Bae

Data & Knowledge Engineering (DKE) 68 (2009) 28–48

Page 2: Determining the best K for clustering transactional datasets – A coverage density-based approach

Intelligent Database Systems Lab

N.Y.U.S.T.I. M.

2

Outline

Motivation Objective Methodology Experiments Conclusion Comments

Page 3: Determining the best K for clustering transactional datasets – A coverage density-based approach

Intelligent Database Systems Lab

N.Y.U.S.T.I. M.Motivation

Cluster the transactional datasets – a kind of special categorical data

Time complexity: O(dmN2logN)

3

Name Buy

Jane Coke, MilkMary Coke, PepsiTom Milk, Water

Denny Milk, Juice

Tina Juice, Red Wine, Pepsi

Boolean values

Name Coke Milk Pepsi Water Juice Red Wine

Jane 1 1 0 0 0 0Mary 1 0 1 0 0 0Tom 0 1 0 1 0 0

Denny 0 1 0 0 1 0Tina 0 0 1 0 1 1

Page 4: Determining the best K for clustering transactional datasets – A coverage density-based approach

Intelligent Database Systems Lab

N.Y.U.S.T.I. M.Objectives

To design a method ACTD (Agglomerative Clustering algorithm with Transactional-cluster-modes Dissimilarity) especially for transactional data

Instead of ACE (Agglomerative Categorical clustering with Entropy criterion) Find best-K More efficiently

4

Page 5: Determining the best K for clustering transactional datasets – A coverage density-based approach

Intelligent Database Systems Lab

N.Y.U.S.T.I. M.

ACE ACTD

Methodology – Overview of SCALE

5

(Sampling, Clustering structure Assessment, cLustering & domain-specfic Evaluation)

Agglomerative

BKPlot DMDI

Page 6: Determining the best K for clustering transactional datasets – A coverage density-based approach

Intelligent Database Systems Lab

N.Y.U.S.T.I. M.

Coverage Density

Transactional-cluster-mode A subset of items

Methodology – ACTDIntra-cluster similarity

6

97

337

Nk

Mk

1.c2/3,b2/3,a.8,

in this case, only c is the transactional-cluster-mode

Page 7: Determining the best K for clustering transactional datasets – A coverage density-based approach

Intelligent Database Systems Lab

N.Y.U.S.T.I. M.

Transactional-cluster-mode dissimilarity

Time complexity: O(dmN2logN) O(MN2logN)

Methodology – ACTDInter-cluster similarity

7

03233-1

52

106-1

5233-1

21

126-1

6233-1

[0, .5]

Page 8: Determining the best K for clustering transactional datasets – A coverage density-based approach

Intelligent Database Systems Lab

N.Y.U.S.T.I. M.Methodology – DMDI

8

Valleys、change dramatically

Page 9: Determining the best K for clustering transactional datasets – A coverage density-based approach

Intelligent Database Systems Lab

N.Y.U.S.T.I. M.Experiments – Performance

9

Page 10: Determining the best K for clustering transactional datasets – A coverage density-based approach

Intelligent Database Systems Lab

N.Y.U.S.T.I. M.Experiments – Quality

10

Page 11: Determining the best K for clustering transactional datasets – A coverage density-based approach

Intelligent Database Systems Lab

N.Y.U.S.T.I. M.Experiments – Quality on sample dataset

11

With noise

Page 12: Determining the best K for clustering transactional datasets – A coverage density-based approach

Intelligent Database Systems Lab

N.Y.U.S.T.I. M.

12

Conclusions

The ACTD The Coverage Density-based method is promising for

transactional datasets Faster More stable

than entropy-based method

The Agglomerative Hierarchical clustering algorithm and DMDI can help to find best-K

Page 13: Determining the best K for clustering transactional datasets – A coverage density-based approach

Intelligent Database Systems Lab

N.Y.U.S.T.I. M.

13

Comments

Advantage …

Drawback …

Application …


Algorithms for clustering large datasets in arbitrary metric spaces

Algorithms for clustering large datasets in arbitrary metric spaces

Documents
Dan Richardson - Using transactional datasets with partners - FutureData 2017

Dan Richardson - Using transactional datasets with partners - FutureData 2017

Data & Analytics
CURE: EFFICIENT CLUSTERING ALGORITHM FOR LARGE DATASETS VULAVALA VAMSHI PRIYA

CURE: EFFICIENT CLUSTERING ALGORITHM FOR LARGE DATASETS VULAVALA VAMSHI PRIYA

Documents
Intelligent Database Systems Lab N.Y.U.S.T. I. M. Determining the best K for clustering transactional datasets – A coverage density-based approach Presenter

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Determining the best K for clustering transactional datasets – A coverage density-based approach Presenter

Documents
Distributed approximate spectral clustering for large scale datasets

Distributed approximate spectral clustering for large scale datasets

Data & Analytics
Spectral Embedded Deep Clustering · 2020-05-19 · JULE (Joint Unsupervised LEarning) learns a clustering friendly low dimensional representation for image datasets by using a convolutional

Spectral Embedded Deep Clustering · 2020-05-19 · JULE (Joint Unsupervised LEarning) learns a clustering friendly low dimensional representation for image datasets by using a convolutional

Documents
Function Clustering Self-Organization Maps (FCSOMs) for ... · mechanism may be not suitable for the overlapping datasets, fuzzy clustering is efficient for gene function annotation

Function Clustering Self-Organization Maps (FCSOMs) for ... · mechanism may be not suitable for the overlapping datasets, fuzzy clustering is efficient for gene function annotation

Documents
Incremental Model-Based Clustering for Large Datasets With ... fileIncremental Model-Based Clustering for Large Datasets With Small Clusters ChrisF RALEY,AdrianR AFTERY and RonW EHRENS

Incremental Model-Based Clustering for Large Datasets With ... fileIncremental Model-Based Clustering for Large Datasets With Small Clusters ChrisF RALEY,AdrianR AFTERY and RonW EHRENS

Documents
Anomaly intrusion detection by clustering transactional audit streams in a host computer

Anomaly intrusion detection by clustering transactional audit streams in a host computer

Documents
Strategies and Algorithms for Clustering Large Datasets: A ... · Strategies and Algorithms for Clustering Large Datasets: A Review Javier Béjar ... Classical methods for clustering

Strategies and Algorithms for Clustering Large Datasets: A ... · Strategies and Algorithms for Clustering Large Datasets: A Review Javier Béjar ... Classical methods for clustering

Documents
Clustering 1 Wu-Jun Li Department of Computer Science and Engineering Shanghai Jiao Tong University Lecture 8: Clustering Mining Massive Datasets

Clustering 1 Wu-Jun Li Department of Computer Science and Engineering Shanghai Jiao Tong University Lecture 8: Clustering Mining Massive Datasets

Documents
Efficiently Clustering Transactional data with Weighted Coverage Density

Efficiently Clustering Transactional data with Weighted Coverage Density

Documents
A clustering-based data reduction for very large spatio ... · A clustering-based data reduction for very large spatio-temporal datasets Nhien-An Le-Khac1, Martin Bue2, Michael Whelan1,

A clustering-based data reduction for very large spatio ... · A clustering-based data reduction for very large spatio-temporal datasets Nhien-An Le-Khac1, Martin Bue2, Michael Whelan1,

Documents
Hierarchical Model-Based Clustering of Large Datasets Through Fractionation and Refractionation Advisor ： Dr. Hsu Graduate ： You-Cheng Chen Author ： Jeremy

Hierarchical Model-Based Clustering of Large Datasets Through Fractionation and Refractionation Advisor ： Dr. Hsu Graduate ： You-Cheng Chen Author ： Jeremy

Documents
Big-Data Clustering: K-Means or K-Indicators? · The K-means algorithm is arguably the most popular data clustering method, commonly applied to processed datasets in some "feature

Big-Data Clustering: K-Means or K-Indicators? · The K-means algorithm is arguably the most popular data clustering method, commonly applied to processed datasets in some "feature

Documents
군집분석 (Clustering). 2 설명모델 (description model) 실습 데이터 Iris Data datasets 패키지 Wine Data Hdclassif 패키지 AdultUCI Data arules 패키지

군집분석 (Clustering). 2 설명모델 (description model) 실습 데이터 Iris Data datasets 패키지 Wine Data Hdclassif 패키지 AdultUCI Data arules 패키지

Documents
Geometry on “Gene Expression Datasets”richard.myweb.cs.uwindsor.ca/cs510/puspal_bhabak_survey.pdf · clustering problem is converted to a tree partitioning problem. Zhou et al

Geometry on “Gene Expression Datasets”richard.myweb.cs.uwindsor.ca/cs510/puspal_bhabak_survey.pdf · clustering problem is converted to a tree partitioning problem. Zhou et al

Documents
SocalBSI 2008: Clustering Microarray Datasets Sagar Damle, Ph.D. Candidate, Caltech Distance Metrics: Measuring similarity using the Euclidean and Correlation

SocalBSI 2008: Clustering Microarray Datasets Sagar Damle, Ph.D. Candidate, Caltech Distance Metrics: Measuring similarity using the Euclidean and Correlation

Documents
Co-clustering of multi-view datasets: a parallelizable approach

Co-clustering of multi-view datasets: a parallelizable approach

Education
CS246: Mining Massive Datasets Jure Leskovec, ... · ¡Clustering in two dimensions looks easy ¡Clustering small amounts of data looks easy ¡And in most cases, looks are not deceiving

CS246: Mining Massive Datasets Jure Leskovec, ... · ¡Clustering in two dimensions looks easy ¡Clustering small amounts of data looks easy ¡And in most cases, looks are not deceiving

Documents
CLUSTERING Eitan Lifshits Big Data Processing Seminar Prof. Amir Averbuch 18.1.2015 Mining of Massive Datasets, Jure Leskovec, Anand Rajaraman, Jeffery

CLUSTERING Eitan Lifshits Big Data Processing Seminar Prof. Amir Averbuch 18.1.2015 Mining of Massive Datasets, Jure Leskovec, Anand Rajaraman, Jeffery

Documents
Clustering - Visualization · Hierarchical clustering for large datasets? • OK for small datasets (e.g.,

Clustering - Visualization · Hierarchical clustering for large datasets? • OK for small datasets (e.g.,

Documents
Clustering of Data with Missing Entries using Non-convex ...€¦ · Clustering is an exploratory data analysis technique used to discover natural groupings in large datasets, with

Clustering of Data with Missing Entries using Non-convex ...€¦ · Clustering is an exploratory data analysis technique used to discover natural groupings in large datasets, with

Documents
An Extended Density based Clustering Algorithm for Large ... · 2.5. Clustering High- Dimensional Data: Most of the clustering methods stated above are implemented in 2D spatial datasets

An Extended Density based Clustering Algorithm for Large ... · 2.5. Clustering High- Dimensional Data: Most of the clustering methods stated above are implemented in 2D spatial datasets

Documents
Clustering Francisco Moreno Extractos de Mining of Massive Datasets Rajamaran, Leskovec & Ullman

Clustering Francisco Moreno Extractos de Mining of Massive Datasets Rajamaran, Leskovec & Ullman

Documents
Incremental Model-Based Clustering for Large Datasets With ... · Incremental Model-Based Clustering for Large Datasets With Small Clusters Chris Fraley, Adrian Rafteryz and Ron Wehrensy

Incremental Model-Based Clustering for Large Datasets With ... · Incremental Model-Based Clustering for Large Datasets With Small Clusters Chris Fraley, Adrian Rafteryz and Ron Wehrensy

Documents
A Very Fast Method for Clustering Big Text Datasets

A Very Fast Method for Clustering Big Text Datasets

Documents
Deep Subspace Clustering Networks extensively evaluate our method on face clustering, using the Extended Yale B [21] and ORL [39] datasets, and on general object clustering, using

Deep Subspace Clustering Networks extensively evaluate our method on face clustering, using the Extended Yale B [21] and ORL [39] datasets, and on general object clustering, using

Documents
Clustering on Intel MIC with Huge Datasets fileEinleitung Clustering: Gruppierung zusammengeh origer Datenpunkte Implementierung des Clustering-Algorithmus von Pehersdorfer et al

Clustering on Intel MIC with Huge Datasets fileEinleitung Clustering: Gruppierung zusammengeh origer Datenpunkte Implementierung des Clustering-Algorithmus von Pehersdorfer et al

Documents
Clustering Very Large Multi-dimensional Datasets with MapReduce

Clustering Very Large Multi-dimensional Datasets with MapReduce

Documents