Distance Metric Learning for Large Margin Nearest Neighbor Classification (LMNN) NIPS 2006 Kilian Q....

Distance Metric Learning for Large Margin Nearest Neighbor Classification (LMNN)

NIPS 2006Kilian Q. Weinberger, John Blitzer and Lawrence K. Saul

Problem

• Distance metric learning for kNN

Goal:k-nearest neighbors belong to same classExample from different classes are separated by a large margin

kNN LMNN

linear classification Linear SVM

counterpart

• Mahalanobis distance for metric learning

• Goal: learn the linear transformation L

• Cost function: Indicates if a target neighbor

SVM:Hinge loss

How to solve it?

• Convex optimization with semidefinite programming– The matrix whose elements are linear with the

unknown variables is required to be positive semidefinite.

Slack variable

Classification

• kNN

• Energy-based classification

Results

Documents