Factorization Machine

Factorization MachineIm JerryFactorization MachineFactorization MethodsFactorization MachineSupport Vector MachineFactorization ModelRatingsUser FeaturesItem FeatureSupport Vector Machine (SVM)D = {(xi , yi) | xi RP, yi {-1, 1}}i = 1~nLine: y(x) = wx + b = 0For all yi = 1, y(xi) = wxi + b 1For all yi = -1, y(xi) = wxi + b -1

Minimize |w|Support Vector Machine (SVM)

Recommender GroupY U NO USE SVM?Y U NO USE SVM?Real Value V.S. Classification

Sparsity

y(x) = wx + b = wu + wi + bActually We Do Use SVMOn EnsembleEnsemble modelsModel 1Model 2Model 3UserItemEnsemble modelsModel 1Model 2Model 3UserItemxyEnsemble modelsModel 1Model 2Model 3UserItem+++=Predictions on train setTrain set answerPredictions on train setTrain set answerSVMModel WeightsPredictions on train setTrain set answerSVMModel WeightsModel WeightsPredictions on test setPredictions on train setTrain set answerSVMModel WeightsModel WeightsPredictions on test setFinal PredictionSVM Calculates weight of featuresFactorization MachineOriginal SVM:y(x) = wx + b = b + wixi

Factorization Machine:y(x) = b + wixi + (vivj) xixjFactorization MachineOriginal SVM:y(x) = wx + b = b + wixi

Factorization Machine:y(x) = b + wixi + (vivj) xixjInteraction between variablesi=0j=i+1(vivj )?W(vivj )?W

(vivj )?W

?(vivj )?W

CF Matrix(vivj )?VVT=kW(vivj )?WVVT=

y(x) = b + wixi + (vivj) xixji=0j=i+1(vivj )?WVVT=

y(x) = b + wixi + (vivj) xixji=0j=i+1

(vivj )?WVVT=

y(x) = b + wixi + (vivj) xixji=0j=i+1

(vivj )?WVVT=

y(x) = b + wixi + (vivj) xixji=0j=i+1

= vAvTI(vivj )?WVVT=

y(x) = b + wixi + (vivj) xixji=0j=i+1= vAvTI(vivj )?WVVT=

y(x) = b + wixi + (vivj) xixji=0j=i+1= vAvTI(vivj )?WVVT=

y(x) = b + wixi + (vivj) xixji=0j=i+1Factorization(vivj )?WVVT=

y(x) = b + wixi + (vivj) xixji=0j=i+1FactorizationMachineFactorization Machine

W

FM V.S. SVMSVM fails with sparsityFM learn with sgd, SVM learn with dual

FM V.S. SVM

Polynomial kernel SVMCompare to FM:Wi, j are all independent to each other.FM V.S. MFMF:y( x ) = b + wu + wi + vuvi

SVD++: y( x ) = b + wu + wi + vuvi + (1/|Nu|)vivl

Claims that FM is more generalThanks!