1

CFM:ConvolutionalFactorizationMachinesforContext-Aware

Recommendation

XinXin, BoChen,Xiangnan He,etal.

School of ComputingScience,UniversityofGlasgowSchoolofSoftwareEngineering,ShanghaiJiaotongUniversity

SchoolofDataScience,UniversityofScienceandTechnologyofChina

1 2 3

1

2

PresentedbyXinXin@IJCAI19,Aug.16,2019

3

FactorizationMachines(FM)• FM[Rendel etal.,ICDM2010]isoneofthemosteffectivefeature-basedrecommendationalgorithms

2

FactorizationMachines(FM)• FM[Rendel etal.,ICDM2010]isoneofthemosteffectivefeature-basedrecommendationalgorithms

• One/Multi-hotfeaturevectorsasinputs– Encodesbothitem/usersideinformationandcontextinformation

3

FactorizationMachines(FM)• FM[Rendel etal.,ICDM2010]isoneofthemosteffectivefeature-basedrecommendationalgorithms

• One/Multi-hotfeaturevectorsasinputs• Combineslinearregressionandsecond-orderfeatureinteraction

4

linearregression second-orderfeatureinteraction

feature embeddingfor

LimitationsofFM• Innerproductbasedfeatureinteraction– Embeddingdimensionsareindependentwitheachother

– Theremaybecorrelationsbetweendifferentdimensions[Zhangetal.,SIGIR2014]

• Higher-orderinteraction&Non-linearity– NFM[Heetal.,SIGIR2017]– DeepFM [Guo etal.,IJCAI2017]

5

Innerproduct ?

Contributions• Utilizeanouterproduct-basedinteractioncubetorepresentfeatureinteractions,whichencodesbothinteractionsignalsanddimensioncorrelations.

• Employ3DCNNabovetheinteractioncubetocapturehigh-orderinteractionsinanexplicitway.

• Leverageanattentionmechanismtoperformfeaturepooling,reducingtimecomplexity.

6

ConvolutionalFactorizationMachines(CFM)

• Predictionrule:

• Overallstructure:

7

CFM• InputandEmbeddingLayer– sparsefeaturevectors==>embeddingtablelookup

• Attentionpoolinglayer

8

CFM• InputandEmbeddingLayer– sparsefeaturevectors==>embeddingtablelookup

• Attentionpoolinglayer

– attentionscore

– softmax

– weightedsum

9

CFM• InteractionCube

• 3DCNN

10

CFM• ModelTraining– Pair-wiserankingloss(BPR)[Rendle etal.,UAI2009]

– L2regularization– Drop-out

11

Experiments• Researchquestions:– DoesCFMmodeloutperformstate-of-the-artmethodsfortop-k recommendation?

– HowdothespecialdesignsofCFM(i.e.,interactioncubeand3DCNN)affectthemodelperformance?

– What’stheeffectoftheattention-basedfeaturepooling?• Datasets:– Frappe– Last.fm– MovieLens

• Evaluation:– Leave-one-out– HR&NDCG

12

Experiments• Baselines:– PopRank:popularity-basedrecommendation– FM[Rendle etal.,ICDM2010]:originalFMwithBPRloss– NFM[Heetal.,SIGIR17]:stackingMLPuponFM– DeepFM[Guo etal.,IJCAI2017]:wide&deep+FM– ONCF[Heetal.,IJCAI2018]:outerproduct+MF

13

Experiments• RQ1(performance)

– DeepstructurehelpstoimproveFM(DeepFM&NFM)– CFMachievesthebestperformance

Frappe PopRank FM DeepFM NFM ONCF CFM

HR@10 0.3493 0.5486 0.6035 0.6197 0.6531 0.6720

NDCG@10 0.1898 0.3469 0.3765 0.3924 0.4320 0.4560

Last.fm PopRank FM DeepFM NFM ONCF CFM

HR@10 0.0023 0.2382 0.2612 0.2676 0.3208 0.3538

NDCG@10 0.0011 0.1374 0.1473 0.1488 0.1823 0.1948

Frappe PopRank FM DeepFM NFM ONCF CFM

HR@10 0.0235 0.0998 0.1170 0.1192 0.1110 0.1323

NDCG@10 0.0107 0.0452 0.0526 0.0553 0.0514 0.0627

14

Results• RQ2(modelablation)– Interactioncube&3DCNN

15

3Darchitecturehelpstoimproveperformance

Results• RQ3(featurepooling)– Effectofattention

– Runtime&performance

16

Attentionpooinglayerhelpstoimprovebothefficiencyandeffectiveness

Conclusion&FutureWork• CFMforfeature-basedrecommendation– Outer product-basedinteractioncube– 3DCNNtoexplicitlylearnhigh-orderinteractions– Attention-basedfeaturepoolinglayertoreducecomputationalcost

• Futurework– Improveefficiency– Residuallearning

17

Reference• [Rendle etal.,2010]Factorizationmachines.InICDM.• [Rendle etal.,2009]Bpr:Bayesianpersonalizedrankingfrom

implicitfeedback.InUAI.• [He,etal.,2017]Neuralfactorizationmachinesforsparse

predictiveanalytics.InSIGIR.• [Guo etal.,2017] Deepfm:A factorization-machinebased

neuralnetworkforctr prediction.InIJCAI.• [Heetal.,2018]Outerproduct-basedneuralcollaborative

filtering.InIJCAI.• [Zhangetal.,2014]Explicitfactormodelsforexplainable

recommendationbasedonphrase-levelsentimentanalysis.InSIGIR.

18

ThankyouQ&A

19