Nonlinear Learning Using Local Coordinate Coding K. Yu, T. Zhang and Y. Gong, NIPS 2009 Improved...

Nonlinear Learning Using Local Coordinate CodingK. Yu, T. Zhang and Y. Gong, NIPS 2009

 Improved Local Coordinate Coding Using Local Tangents K. Yu and T. Zhang, ICML 2010

Locality-Constrained Linear Coding for Image Classification

J. Wang, J. Yang, K. Yu, F. Lv, T. Huang and Y. Gong, CVPR2010

Presented by: Mingyuan ZhouDuke University, ECESeptember 17, 2010

Local Coordinate Coding: TheoryNonlinear Learning Using Local Coordinate Coding

Local Coordinate Coding: Practice

• Sparse coding:

• LLC:

Nonlinear Learning Using Local Coordinate Coding

Experiments: linear ridge regression based on the sparse codes or LCC

Nonlinear Learning Using Local Coordinate Coding

Experiments: linear ridge regression

Nonlinear Learning Using Local Coordinate Coding

Experiments: Handwritten Digit Recognition

Nonlinear Learning Using Local Coordinate Coding

Experiments: Handwritten Digit Recognition

Nonlinear Learning Using Local Coordinate Coding

Conclusion

Nonlinear Learning Using Local Coordinate Coding

Improved Local Coordinate Coding Using Local Tangents

Motivation• For smooth but highly nonlinear function, local linear

approximation may not necessarily be optimal, which means that many anchor points are needed to achieve accurate approximation.

• The improved LCC has better approximation of high dimensional nonlinear functions when the underlying data manifold is locally relatively flat.

• It significantly reduces the number of anchor points, leading to reduced computational complexity and improved prediction.

Improved Local Coordinate Coding Using Local Tangents

VQ, LCC and Improved LCC

VQ

Improved Local Coordinate Coding Using Local Tangents

VQ, LCC and Improved LCC

Support:

Coding: (Extended LCC)

(LCC with local Tangents)

Improved Local Coordinate Coding Using Local Tangents

Algorithm

Improved Local Coordinate Coding Using Local Tangents

Experiments The feature dimension is increased from |C| to |C|(1+m) for LCC with local Tangents.

Locality-Constrained Linear Coding for Image Classification

Introduction

• VQ + SPM + Nonlinear SVM

• SC + SPM + Linear SVM

• LLC + SPM + Linear SVM

Locality-Constrained Linear Coding for Image Classification

Objective functions

• VQ

• SC

• LLC

Locality-Constrained Linear Coding for Image Classification

Properties of LLC

• Better reconstruction• Local smooth sparsity• Analytical solution

• Approximate solution with KNN constraint

Locality-Constrained Linear Coding for Image Classification

Codebook Optimization

Locality-Constrained Linear Coding for Image Classification

Experiments

Locality-Constrained Linear Coding for Image Classification

Experiments