Distance Metric Learning for Large Margin Nearest Neighbor Classification (LMNN)

NIPS 2006Kilian Q. Weinberger, John Blitzer and Lawrence K. Saul

Problem

• Distance metric learning for kNN

Goal:k-nearest neighbors belong to same classExample from different classes are separated by a large margin

kNN LMNN

linear classification Linear SVM

counterpart

Model

• Mahalanobis distance for metric learning

Model

• Goal: learn the linear transformation L

• Cost function: Indicates if a target neighbor

SVM:Hinge loss

How to solve it?

• Convex optimization with semidefinite programming– The matrix whose elements are linear with the

unknown variables is required to be positive semidefinite.

Slack variable

Classification

• kNN

• Energy-based classification

Results

Results

Results