START OF DAY 6 Reading: Chap. 8. Group Project Progress Report

START OF DAY 6Reading: Chap. 8

Group Project Progress Report

3 Minute Synopsis

• What have you done?• Where are you going?• Thoughts on how you are going to get there

Model Combination

Prophetic Warning

Now it is not common that the voice of the people desireth anything contrary to that which is right; but it is common for the lesser part of the people to desire that which is not right; therefore this shall ye observe and make it your law--to do your business by the voice of the people. (Mosiah 29:26)

What is the point?One person may get it

wrongMany less likely so

Following the Prophet

• Learning algorithms have different biases– They probably do not make the same mistakes– If one makes a mistake, the others may not

• Solution: model combination– Exploit variation in data• Bagging, Boosting

– Exploit variation in algorithms• Ensemble, Stacking, Cascade Generalization, Cascading,

Delegating, Arbitrating

Sometimes called metalearning

Bagging (I)

Bagging (II)

Boosting (I)

Boosting (II)

Ensemble (I)

Ensemble (II)

Key issue: diversity

Classifier Output Distance

• Measures difference in behavior• Accuracy problematic– A and are both 50% accurate on T– Appear the same, yet A misses what B gets right,

and vice versa!• COD = ratio of number of disagreements

between A and B to the total number of instances– COD(A,B)=1 (maximum)

Stacking (I)

Stacking (II)

Cascade Generalization (I)

Cascade Generalization (II)2-step

Cascade Generalization (III)n-step

Cascading (I)

Cascading (II)

Delegating (I)

Delegating (II)

Arbitrating (I)

Arbitrating (II)

END OF DAY 6Homework: Classification Model Evaluation