View
218
Download
0
Tags:
Embed Size (px)
Citation preview
Paper Source
Honglak Lee, Alexis Battle, Rajat Raina, Andrew Y. Ng. Efficient Sparse Coding Algorithms. NIPS 2007.
Main Contributions
It proposes a new efficient algorithm to solve LASSO
A two stage optimization algorithm is proposed to the coding problem
Sparse Coding Problem Sparse coding is a method for discovering good ba
sis vectors automatically using only unlabeled data
It is similar to PCA Given a training set of m vectors where
, we attempt to find a succinct representation for each xi using basis vectors
and a sparse vector such that Note that the basis can be overcomplete, i.e., n>k
1 2, ,[ ], mx xX x k
ix R
1 2, , , knbb b R nsR
1 21
[ , , , ]n
i j j nj
x s b b b b s
The basis act as the principal components in PCA, and they capture a large number of patterns in the input data
The optimization problem in sparse coding
where and is a sparse penalty function
1 1, , , , ,[ ], [ ], [ 1, ]m n mx bB b S sX x s
A New Algorithm to Solve LASSO
The formulation of LASSO
where x, y are vectors and A is a matrix
Basic idea of the new algorithm The difficulty of this problem lies in We guess the sign of each component
of x
1x
Feature-sign Search Algorithm
Proof of Feature-sign Search Algorithm – (I)
Proof of Feature-sign Search Algorithm – (II)
Proof of Feature-sign Search Algorithm – (III)
Use Dual to simplify computation
Original Problem
Experiment – (I) The comparison of Feature-sign search
algorithm and other algorithms for LASSO
Experiment – (II) The comparison of the two-stage
algorithm and other algorithms