Algorithms: The Basic MethodsWitten – Chapter 4

Charles Tappert Professor of Computer ScienceSchool of CSIS, Pace University

1. Inferring Rudimentary Rules1R (1-rule) Method

This method tests a single attribute and creates a rule that assigns the most frequent class to that attribute

2. Statistical ModelingNaïve Bayes Method

Assumes statistical independence – multiply probabilities

2. Statistical ModelingNaïve Bayes Method

3. Divide-and-Conquer:Construct Decision Trees: ID3 Method

3. Divide-and-Conquer:Construct Decision Trees: ID3 Method

3. Divide-and-Conquer:Construct Decision Trees: ID3 Method

3. Divide-and-Conquer:Construct Decision Trees: ID3 Method

Compare: Example from Naïve Bayes Method

4. Covering Algorithms: Constructing Rules

5. Mining Association Rules

5. Mining Association Rules

6. Linear ModelsPrediction by linear regression

6. Linear ModelsLinear Classification via Perceptron

Non-parametric algorithm

7. Instance-Based Learningk-nearest-neighbor method

8. Clustering: k-means TechniqueTop down method

• Specify in advance number of clusters, k• Randomly choose k seed points• Find the closest points to the seed points• Compute the means of points closest to

each seed point –> seeds for next iteration• Stop when the seed points become stable

8. Clustering: k-means TechniqueTop down method

Clustering: Hierarchy - DendrogramBottom up method

Also, see Witten p 81, p 275-278

Mary Manfredi dissertation