Research ArticleA Tri-Attention Neural Network Model-BasedRecommendation

Nanxin Wang1 Libin Yang 1 Yu Zheng2 Xiaoyan Cai3 Xin Mei1 and Hang Dai1

1School of Cyberspace Security Northwestern Polytechnical University Xirsquoan 7100072 China2Faculty of Information Technology Monash University Wellington Road Clayton VIC 3800 Australia3School of Automation Northwestern Polytechnical University Xirsquoan 710072 China

Correspondence should be addressed to Libin Yang libinynwpueducn

Received 16 June 2020 Revised 13 August 2020 Accepted 22 September 2020 Published 9 October 2020

Academic Editor Huizhi Liang

Copyright copy 2020 NanxinWang et alis is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

Heterogeneous information network (HIN) which contains various types of nodes and links has been applied in recommendersystems Although HIN-based recommendation approaches perform better than the traditional recommendation approachesthey still have the following problems for example meta-paths are manually selected not automatically meta-path repre-sentations are rarely explicitly learned and the global and local information of each node in HIN has not been simultaneouslyexplored To solve the above deficiencies we propose a tri-attention neural network (TANN)model for recommendation taskeproposed TANN model applies the stud genetic algorithm to automatically select meta-paths at first en it learns global andlocal representations of each node as well as the representations of meta-paths existing in HIN After that a tri-attentionmechanism is proposed to enhance the mutual influence among users items and their related meta-paths Finally the encodedinteraction information among the user the item and their related meta-paths which contain more semantic information can beused for recommendation task Extensive experiments on the Douban Movie MovieLens and Yelp datasets have demonstratedthe outstanding performance of the proposed approach

1 Introduction

With the increasing amount of data on the Internet usersfind it difficult to obtain useful information In recent yearsrecommender systems which can only retain relevant in-formation have received increasing attention [1ndash3] Re-searchers proposed many methods to solve therecommendation task which can be classified into fourclasses neighborhood-based methods [4ndash7] model-basedmethods [8ndash11] graph-based methods [12ndash16] and deepneural network based methods [17ndash21] Neighborhood-basedmethods contain user-based collaborative filtering anditem-based collaborative filtering this kind of methodsutilize neighbor information of users to make prediction[22] Model-based methods first construct a descriptivemodel using the usersrsquo preferences and the recommenda-tions are generated based on the model [23] Graph-basedmethods not only consider the neighborhood information of

each node but also consider the network structure [16 20]Inspired by the great success of deep neural networks incomputer vision and natural language process recent re-searchers have exploited deep neural networks in recom-mendation task

Besides since multiple types of auxiliary informationbecome available many methods propose to use this in-formation to improve the performance of recommendation[24 25] As auxiliary information has heterogeneity andcomplexity it is challenging to leverage this information inrecommender systems Heterogeneous information network(HIN) containing various kinds of nodes connected bymultiple types of relations that can model rich auxiliary datahas been applied in recommender systems [26 27] Al-though the existing HIN-based recommendation ap-proaches enhanced the performance than the traditionalrecommendation approaches they still have the followingproblems first the meta-paths are manually selected not

HindawiComplexityVolume 2020 Article ID 3857871 10 pageshttpsdoiorg10115520203857871

automatically second meta-path representations are rarelyexplicitly learned third the global and local information ofeach node in HIN has not been simultaneously explored

To solve the above deficiencies we develop a tri-atten-tion neural network (TANN) model for recommendationtask e proposed TANN model applies the stud geneticalgorithm to automatically select meta-paths at first en itlearns global and local representations of each node as wellas the representations of meta-paths existing in the HINAfter that a tri-attention mechanism is proposed to enhancethe mutual influence among users items and their relatedmeta-paths Finally the encoded interaction informationamong the user the item and their related meta-pathswhich contain much semantic information can be used forthe recommendation task To summarization our majorcontributions are

(1) Meta-paths are automatically selected via the studgenetic algorithm

(2) A novel tri-attention neural network (TANN) modelis developed to enhance the influence among theuser the item and their related meta-paths in amutually reinforced way

(3) A TANNmodel-based recommendation approach isdeveloped and the extensive experiments on theDouban Movie MovieLens and Yelp datasetsdemonstrate the outstanding performance of theapproach

We organize the rest of this paper as follows Section 2reviews related work Section 3 presents the preliminariesand notations in the paper Section 4 illustrates the proposedtri-attention neural network (TANN) model-based recom-mendation approach Section 5 illustrates the experimentalresults and Section 6 concludes the paper

2 Related Work

Accurately finding useful information in many e-resourcesbecomes more and more difficult for users due to the de-velopment of information technology and the increasingcontent of information However a recommendation systemcan overcome this obstacle

Alqadah et al [28] utilized each userrsquos local biclusteringneighborhood and developed a collaborative filteringmethod Yao et al [29] applied a clustering analysis andlatent factor model to enhance the neighborhood-basedrecommendationrsquos performance Neighborhood-based sys-tems use the stored ratings to make a recommendationwhereas model-based approaches learn a predictive modelby using the ratings Cremonesi et al [30] presented aPureSVD-basedmatrix factorization method which uses theuser-item rating matrixrsquos most principle singular vectors todescribe users and items Pan et al proposed a consensusfactorization based framework for coclustering networkeddata [31] Sindhwani et al [32] developed a weightednonnegative matrix factorization method Xiong et alpropose an information propagation-based social recom-mendation method (SoInp) and model the implicit user

influence from the perspective of information propagation[33] Hofmann [34] utilized PLSA (Probabilistic LatentSemantic Analysis) to solve collaborative filtering andshowed that the PLSA is equivalent to nonnegative matrixfactorization

Most graph-based recommendation approaches arebased on a random walk [35] Christoffel et al [12] proposeda graph random walk based recommendation algorithmKang et al [16] presented a graph-based top-n recom-mendation model which not only considers the neigh-borhood information encoding by user graph and itemgraph but also takes into account the datarsquos hiddenstructure

Since deep learning techniques have been successfullyapplied in speech recognition and computer vision someresearchers began to utilize deep learning techniques inrecommendation task and found that deep learning basedrecommendation approaches achieve better results than theconventional recommendation approaches Oord et al [36]applied deep convolution neural networks to generate songsrsquolatent factors Wang and Yang [37] combined probabilisticgraphical models and deep belief networks to simultaneouslylearn audio contentrsquos features and make personalized rec-ommendations Xue et al [38] presented a deep matrixfactorization approach to solve the top-n recommendationtask Kim et al [39] integrated convolution neural networkinto probabilistic matrix factorization and proposed acontext-aware hybrid model for the recommender system

In recent years researchers adopted the heterogeneousinformation network (HIN) which characterizes rich aux-iliary data in recommender systems Pham et al [40]modeled the rich information based on the constructedheterogeneous graph to solve the recommendation taskChen et al [41] developed a heterogeneous informationnetwork based projected metric embedding approach forlink prediction Yu et al [42] presented a recommendationmodel based on a constructed attribute-rich HIN Jiang et al[43] modeled the user preferences using a generalizedrandom walk with restart model and developed a hetero-geneous information network based personalized recom-mendation method Hu et al [44] incorporated meta-pathbased context and proposed a co-attentionmechanism baseddeep neural network to solve the recommendation task

Although HIN-based deep learning approaches haveachieved good performance in recommendation task theyusually select meta-paths manually ignoring how to auto-matically select meta-paths Moreover the existing ap-proaches seldom consider the interaction between nodeinformation and meta-path information Our work appliesthe stud genetic algorithm to automatically select meta-paths proposes a tri-attention mechanism that considersinteractions among user-item-meta-path triplets and de-velops a recommendation approach to further enhance therecommendation performance

3 Preliminaries and Notations

We use the definitions of heterogeneous information net-work (HIN) and meta-path in [45]

2 Complexity

Definition 1 A heterogeneous information network is aninformation network which contains various kinds of ob-jects and various kinds of links It is defined asG(VA E RW) where V is the set of different types of vertices A is theobject type set E is the union of different types of links Rdenotes the link type set andW is the union of the weight oneach link An edge e isin E is definedaseijr (vi r vj)vi vj isin V r isin R

Definition 2 A meta-path P is defined as a path in the formof A1⟶R1 A2⟶R2 ⟶Rl Al+1 (abbreviated asA1 A2 Al+1) which describes a composite relation R

R1 ∘R2 ∘ middot middot middot ∘Rl between A1andAl+1 where ∘ denotes thecomposition operator on relations

e meta-path Prsquos length is the number of relationscontained in P Taking the user-movie network as an ex-ample we can use a 4-length meta-path to describe the user-movie relation such as user⟶have seenmovie⟶seen byuser ⟶have seenmovie or short as UMUM

Definition 3 A path instance is a sequence of entity nodes itis an explicit path in a meta-path that is p isin P

4 Tri-Attention Neural Network (TANN)Model-Based Recommendation

In this section we present the proposed tri-attention neuralnetwork (TANN) model at first and then illustrate theTANN model-based recommendation approach

41 TANN Model e whole architecture of the proposedTANN is presented in Figure 1

411 Embeddings for Users and Items In order to make theusersrsquo and itemsrsquo representations more meaningful wepropose global representation to represent coarse-grainedfeatures of users and items and develop local representationto represent fine-grained features of users and items thenwe integrate global information and local information ofeach node in HIN

(1) Global Representations of Users and Items Following[46] we use a lookup layer to map the usersrsquo and itemsrsquo one-hot representations to low-dimensional dense vectors Givena user-item pair ltu igt let lu isin R||times1and zi isin R|I|times1denotetheir one-hot representations L isin R||d and Z isin R|I|timesd rep-resent the lookup layerrsquos corresponding parameter matriceswhich preserve usersrsquo and itemsrsquo information the userembeddingrsquos and item embeddingrsquos number of dimension isdenoted as d and ||and |I| denote the usersrsquo number anditemsrsquo number respectively We apply HIN2VEC algorithm[47] to obtain the matrix L and Z e user ursquos and item irsquosglobal representations are represented as

xu LT middot lu (1)

yi ZTmiddot zi (2)

(2) Local Representations of Users and Items As each usercan be represented as a sequence of item and each item canbe represented as a sequence of user we learn local rep-resentations of users (items) according to the correspondingitem (user) sequence Here we use Sn(u) isin R|ℓu|times|I| andSn(i) isin R|ℓi|times|| to represent the sequencematrix of the user ursquosand the item irsquos neighbors respectively For each neighbornode in the sequence we use one-hot representation torepresent it |ℓu| and |ℓi| are the number of ursquos neighbors andthe number of item irsquos neighbors in HIN and n(u) and n(i)denote the neighbor set of user u and item i respectivelyen we apply a lookup layer to obtain the low-dimensionalvector of each node in the item (user) sequence of the user(item) After that the local representation of the user (item)is obtained based on a neighbor attention mechanism whichcan be described as follows

xuprime cn(u) ⊙ bn(u)1113872 1113873

Tmiddot 1 (3)

yiprime γn(i) ⊙ gn(i)1113872 1113873

Tmiddot 1 (4)

where

bn(u) Sn(u) middot Z (5)

gn(i) Sn(i) middot L (6)

γn(u) softmax bn(u)1113872 1113873 (7)

γn(i) softmax gn(i)1113872 1113873 (8)

We concatenate the global representation vector andlocal representation vector of user u and item i and feed theconcatenated vectors into MLP component to get the finalrepresentation that is

1113957xu MLP x⊺uoplusxrsquo⊺u1113872 1113873 (9)

1113957yi MLP y⊺i oplusyrsquo⊺i1113872 1113873 (10)

412 Meta-Path Embedding

(1) Meta-Path Selection Items Assuming there existMmeta-paths in the heterogeneous information network G weconstruct a phenotype matrix H (the size of the matrix isCX

M times X(XleM)) representing all possible combinations ofXmeta-paths where each row represents a meta-path enwe apply the stud genetic algorithm (SGA) [48] to auto-matically select optimal X meta-paths

(2) Meta-Path Instance Selection Traditional HIN embed-ding models mainly use simple random walk strategies toobtain path instances However the path instances obtainedby such strategies are of low value and cannot be directlyapplied to the recommendation system erefore wepropose a weighted selecting strategy with priority In eachstep the walker considers that the next step should walk to a

Complexity 3

higher-priority neighbor and using such walking strategy apath instance which contains more semantic informationcan be obtained for recommendation task en how todefine the priority of each node in a sequence is a keyproblem

Inspired by He et al [46] and Hinton and Salakhutdinov[49] we use a similar pretraining technique to measure eachcandidate nodersquos priority e basic idea is to take the scorebetween different nodes in the heterogeneous network as theweight allocation standard For example we define the scoreranges from 1 to 5 in film evaluation if the score of user u formovie i is 5 then we deem the weight value of the link betweenuser u andmovie i is the highest For each node in HIN it has aweight value for each node of its neighbors and the similarityscore between the node and the corresponding neighbor nodecan also be obtainedWemeasure the priority of each neighbornode by the product value between the weight value and thecorresponding similarity score Such score can reflect thecorrelationrsquos degree between the two nodes We use the abovepriority score strategy to construct each meta-pathrsquos instances

Finally for each meta-path we obtain a differentnumber of meta-path instances with a given length of L andthen calculate scores of these meta-path instances as fol-lows for a path instance we sum the product of weight andcosine similarity between adjacent nodes and then obtainthat the sum value is divided by L as the score of thecorresponding meta-path instance So we can obtain meta-path instancesrsquo scores of each meta-path and select the kpath instances with high scores as the selected meta-pathinstance

(3) Meta-Path Instance Embedding Since a meta-path is asequence of entity nodes we apply the convolution neuralnetwork (CNN) to map a meta-path into a low-dimensionalvector For a meta-path P we useXp isin RLtimesd to represent thepath embedding matrix where p is a path instance Lrepresents the path instancersquos length and d is the nodesrsquoembedding dimension e path instance prsquos embedding iscomputed as follows

hp CNN XpΘ( 1113857 (11)

where Θ denotes CNNrsquos related parameters

(4)Meta-Path Embedding As eachmeta-path contains manypath instances we first apply the proposed meta-path in-stance selection strategy to obtain each meta-pathrsquos top kpath instances en we use the maximum pooling oper-ation to get important dimensional features from the se-lected path instances Let hp1113966 1113967

k

p1denote k selected pathinstancesrsquo embedding e embedding of the meta-path P iscomputed as

cP max minus pooling hp1113966 1113967k

p11113874 1113875 (12)

(5) Tri-Attention Mechanism Based Interaction EmbeddingAs meta-paths contain rich semantic information differentusers through different meta-paths show different preferenceseven when the same user interacts with different items throughthe same meta-path semantic information contained in themeta-path is also different In order to better represent thesemantic information existing among users items and meta-paths we develop a tri-attention mechanism to assign differentweights to different triplets of user-item-meta-path

Given the user embedding xu item embedding yi meta-path embedding cpwhich exists between the user u and itemi we use two full-connection layers to get the triattentivescore as

β1P f Wu1 1113957xu + Wi

11113957yi + WP1 cP + b11113872 1113873 (13)

β2P f w2β1 + b2( 1113857 (14)

where Wlowast1 is the first layerrsquos weight matrix b1 is the firstlayerrsquos bias vectorw2 is the second layerrsquos weight vector andb2 is the second layerrsquos bias f(middot) is set to the ReLU function

We use softmax function to obtain the final interactionweights that is

βP exp β2P( 1113857

1113936ρprimeisinPu⟶iexp β2P( 1113857

(15)

Heterogeneous information network

Meta-path selection via

SGA

UMUM

UMTM

UMMM

UUUM

Path instance sampling

CNN

Pool

ing

Pool

ing

Pool

ing

Pool

ing

CNN

CNN

CNN

User-item-meta-pathtriattention MLP

1 0 1User Lookup

MLP

MLP

1 0 1Item Lookup

Neighbor attention Lookup

3

6

9

User neighbors embedding All items embedding

Embedding

User neighbor

Neighbor attention Lookup

3

6

9

Item neighbors embedding All users embedding

Embedding

Item neighbor

r~u i

X~

uiy~

i

X~

u

yiy

i

XuX

u

Figure 1 e whole architecture of the TANN model

4 Complexity

where Pu⟶i is the set of meta-paths existing between theuser u and the item i e interaction embedding among theuser u the item i and their related meta-path P can berepresented as

1113957xui 1113944pisinPu⟶i

βPmiddot 1113957xuoplus1113957yiopluscp1113872 1113873 (16)

where oplus represents the vector concatenation operation

42 TANN Model-Based Recommendation ApproachOnce we obtain the interaction embedding we apply anMLP component to model complicated interactions

1113957rui MLP 1113957xui1113872 1113873 (17)

in which the MLP contains two hidden layers with ReLUnonlinear activation functions and an output layer withsigmoid functions 1113957rui is interpreted as the relevance scorebetween the user u and the item i and 1113957rui is used to generatea recommendation list for the user u

Defining an appropriate objective function is a criticalstep for model optimization following [14 17] we usenegative sampling to learn the modelrsquos parameters

ℓui minuslog 1113957rui minus 1113944c

j1log 1 minus 1113957ruj1113872 1113873 (18)

in which the first term is the observed interactions and thesecond term indicates the negative feedback drawn from thenoise distribution which is set to the uniform distribution (itcan be set to other biased distributions) and c is the numberof negative item sampling

5 Experiment and Evaluation

51 Datasets We use three datasets that is Douban Moviedataset (httpsgithubcomlibrahu) MovieLens moviedataset (httpsgrouplensorgdatasetsmovielens) and Yelpbusiness dataset (httpswwwyelpcomdataset) As a ratingindicates whether a user has rated an item we deem therating as an interaction record [47 50] Table 1 lists thedetailed description of the datasets Each datasetrsquos first rowshows the number of users items and their interactions andthe other rows list the other relationsrsquo statistics e man-ually selected meta-paths and SGA selected meta-paths ofeach dataset are listed in Table 2 Since long meta-paths mayimport noisy semantics [51] we set the length of meta-pathto 4 and set the number of each meta-pathrsquos selected pathinstances to 5

52 Evaluation Methods In order to evaluate the recom-mendationrsquos performance each datasetrsquos user implicitfeedback records are divided into a training set and test setaccording to a certain proportion For example we use 90feedback records to predict the remaining 10 feedbackrecords As it is a waste of time ranking each userrsquos itemsespecially for large datasets for each user in the dataset werandomly select 200 negative samples having no interactionrecords with the user at first After that we obtain each userrsquos

recommendation list by ranking the positive items andnegative items of the list We use PreK RecallK andNDCGK to evaluate the experimental results

When we apply the SGA algorithm to select the optimal4 meta-paths we follow the principle that ldquothe smaller theobjective function value is the larger the fitness value isrdquothat is we take negative evaluation scores We implementthe TANN model using TensorFlow with Keras (httpskerasio) e batch size is set to 256 the regularizationparameter is set to 00001 the learning rate is set to 0001and the dimension of user embedding and item embeddingis set to 64 We apply Adaptive Moment Estimation (Adam)[52] to optimize the model

53 Performance of TANN-Based Recommendation Toillustrate the benefits of applying the SGA algorithmweighted random walk strategy and integrating HINrsquosglobal and local information of TANN model we compareTANN against the following

(1) TANN wo SGA which applies manually selectedmeta-paths we use the meta-paths in the secondcolumn of Table 2

(2) TANN wo WRWS which applies a random walkstrategy (RWS) instead of weighted random walkstrategy (WRWS) to select meta-path instances

(3) TANN wo local which considers global informa-tion of HIN only ignoring local information that isit uses a global representation of users and itemsonly ignoring local representation of users anditems

Table 3 shows that the performance of TANN wo SGAmethod is the poorest which indicates that the selection ofmeta-paths has a great influence on the recommendationresults e performance of TANN wo RWRS is better thanthe performance of TANN wo SGA it indicates that themeta-path selection strategy plays an important role com-pared to the weighted random walk strategy for recom-mendation task In addition as results illustrate theperformance of TANN wo local is better than that of theabove two methods is can be mainly credited to the factthat the nodes and meta-paths in HIN contain rich implicit

Table 1 Statistics of the three datasets

Datasets Relations (A-B) A B A-B

MovieLens

User-movie 943 1682 100000User-user 943 943 47150

Movie-movie 1682 1682 82798Movie-type 1682 18 2891

Douban movie

User-movie 1830 12337 697311User-user 1830 1830 9155

Movie-movie 12337 12337 61690Movie-type 12337 38 12337

Yelp

User-business 300 450 15109User-user 300 300 1505

Business-business 450 450 2255Business-category 450 47 450

Complexity 5

Definition 1 A heterogeneous information network is aninformation network which contains various kinds of ob-jects and various kinds of links It is defined asG(VA E RW) where V is the set of different types of vertices A is theobject type set E is the union of different types of links Rdenotes the link type set andW is the union of the weight oneach link An edge e isin E is definedaseijr (vi r vj)vi vj isin V r isin R

Definition 2 A meta-path P is defined as a path in the formof A1⟶R1 A2⟶R2 ⟶Rl Al+1 (abbreviated asA1 A2 Al+1) which describes a composite relation R

R1 ∘R2 ∘ middot middot middot ∘Rl between A1andAl+1 where ∘ denotes thecomposition operator on relations

e meta-path Prsquos length is the number of relationscontained in P Taking the user-movie network as an ex-ample we can use a 4-length meta-path to describe the user-movie relation such as user⟶have seenmovie⟶seen byuser ⟶have seenmovie or short as UMUM

Definition 3 A path instance is a sequence of entity nodes itis an explicit path in a meta-path that is p isin P

4 Tri-Attention Neural Network (TANN)Model-Based Recommendation

In this section we present the proposed tri-attention neuralnetwork (TANN) model at first and then illustrate theTANN model-based recommendation approach

41 TANN Model e whole architecture of the proposedTANN is presented in Figure 1

411 Embeddings for Users and Items In order to make theusersrsquo and itemsrsquo representations more meaningful wepropose global representation to represent coarse-grainedfeatures of users and items and develop local representationto represent fine-grained features of users and items thenwe integrate global information and local information ofeach node in HIN

(1) Global Representations of Users and Items Following[46] we use a lookup layer to map the usersrsquo and itemsrsquo one-hot representations to low-dimensional dense vectors Givena user-item pair ltu igt let lu isin R||times1and zi isin R|I|times1denotetheir one-hot representations L isin R||d and Z isin R|I|timesd rep-resent the lookup layerrsquos corresponding parameter matriceswhich preserve usersrsquo and itemsrsquo information the userembeddingrsquos and item embeddingrsquos number of dimension isdenoted as d and ||and |I| denote the usersrsquo number anditemsrsquo number respectively We apply HIN2VEC algorithm[47] to obtain the matrix L and Z e user ursquos and item irsquosglobal representations are represented as

xu LT middot lu (1)

yi ZTmiddot zi (2)

(2) Local Representations of Users and Items As each usercan be represented as a sequence of item and each item canbe represented as a sequence of user we learn local rep-resentations of users (items) according to the correspondingitem (user) sequence Here we use Sn(u) isin R|ℓu|times|I| andSn(i) isin R|ℓi|times|| to represent the sequencematrix of the user ursquosand the item irsquos neighbors respectively For each neighbornode in the sequence we use one-hot representation torepresent it |ℓu| and |ℓi| are the number of ursquos neighbors andthe number of item irsquos neighbors in HIN and n(u) and n(i)denote the neighbor set of user u and item i respectivelyen we apply a lookup layer to obtain the low-dimensionalvector of each node in the item (user) sequence of the user(item) After that the local representation of the user (item)is obtained based on a neighbor attention mechanism whichcan be described as follows

xuprime cn(u) ⊙ bn(u)1113872 1113873

Tmiddot 1 (3)

yiprime γn(i) ⊙ gn(i)1113872 1113873

Tmiddot 1 (4)

where

bn(u) Sn(u) middot Z (5)

gn(i) Sn(i) middot L (6)

γn(u) softmax bn(u)1113872 1113873 (7)

γn(i) softmax gn(i)1113872 1113873 (8)

We concatenate the global representation vector andlocal representation vector of user u and item i and feed theconcatenated vectors into MLP component to get the finalrepresentation that is

1113957xu MLP x⊺uoplusxrsquo⊺u1113872 1113873 (9)

1113957yi MLP y⊺i oplusyrsquo⊺i1113872 1113873 (10)

412 Meta-Path Embedding

(1) Meta-Path Selection Items Assuming there existMmeta-paths in the heterogeneous information network G weconstruct a phenotype matrix H (the size of the matrix isCX

M times X(XleM)) representing all possible combinations ofXmeta-paths where each row represents a meta-path enwe apply the stud genetic algorithm (SGA) [48] to auto-matically select optimal X meta-paths

(2) Meta-Path Instance Selection Traditional HIN embed-ding models mainly use simple random walk strategies toobtain path instances However the path instances obtainedby such strategies are of low value and cannot be directlyapplied to the recommendation system erefore wepropose a weighted selecting strategy with priority In eachstep the walker considers that the next step should walk to a

Complexity 3

higher-priority neighbor and using such walking strategy apath instance which contains more semantic informationcan be obtained for recommendation task en how todefine the priority of each node in a sequence is a keyproblem

Inspired by He et al [46] and Hinton and Salakhutdinov[49] we use a similar pretraining technique to measure eachcandidate nodersquos priority e basic idea is to take the scorebetween different nodes in the heterogeneous network as theweight allocation standard For example we define the scoreranges from 1 to 5 in film evaluation if the score of user u formovie i is 5 then we deem the weight value of the link betweenuser u andmovie i is the highest For each node in HIN it has aweight value for each node of its neighbors and the similarityscore between the node and the corresponding neighbor nodecan also be obtainedWemeasure the priority of each neighbornode by the product value between the weight value and thecorresponding similarity score Such score can reflect thecorrelationrsquos degree between the two nodes We use the abovepriority score strategy to construct each meta-pathrsquos instances

Finally for each meta-path we obtain a differentnumber of meta-path instances with a given length of L andthen calculate scores of these meta-path instances as fol-lows for a path instance we sum the product of weight andcosine similarity between adjacent nodes and then obtainthat the sum value is divided by L as the score of thecorresponding meta-path instance So we can obtain meta-path instancesrsquo scores of each meta-path and select the kpath instances with high scores as the selected meta-pathinstance

(3) Meta-Path Instance Embedding Since a meta-path is asequence of entity nodes we apply the convolution neuralnetwork (CNN) to map a meta-path into a low-dimensionalvector For a meta-path P we useXp isin RLtimesd to represent thepath embedding matrix where p is a path instance Lrepresents the path instancersquos length and d is the nodesrsquoembedding dimension e path instance prsquos embedding iscomputed as follows

hp CNN XpΘ( 1113857 (11)

where Θ denotes CNNrsquos related parameters

(4)Meta-Path Embedding As eachmeta-path contains manypath instances we first apply the proposed meta-path in-stance selection strategy to obtain each meta-pathrsquos top kpath instances en we use the maximum pooling oper-ation to get important dimensional features from the se-lected path instances Let hp1113966 1113967

k

p1denote k selected pathinstancesrsquo embedding e embedding of the meta-path P iscomputed as

cP max minus pooling hp1113966 1113967k

p11113874 1113875 (12)

(5) Tri-Attention Mechanism Based Interaction EmbeddingAs meta-paths contain rich semantic information differentusers through different meta-paths show different preferenceseven when the same user interacts with different items throughthe same meta-path semantic information contained in themeta-path is also different In order to better represent thesemantic information existing among users items and meta-paths we develop a tri-attention mechanism to assign differentweights to different triplets of user-item-meta-path

Given the user embedding xu item embedding yi meta-path embedding cpwhich exists between the user u and itemi we use two full-connection layers to get the triattentivescore as

β1P f Wu1 1113957xu + Wi

11113957yi + WP1 cP + b11113872 1113873 (13)

β2P f w2β1 + b2( 1113857 (14)

where Wlowast1 is the first layerrsquos weight matrix b1 is the firstlayerrsquos bias vectorw2 is the second layerrsquos weight vector andb2 is the second layerrsquos bias f(middot) is set to the ReLU function

We use softmax function to obtain the final interactionweights that is

βP exp β2P( 1113857

1113936ρprimeisinPu⟶iexp β2P( 1113857

(15)

Heterogeneous information network

Meta-path selection via

SGA

UMUM

UMTM

UMMM

UUUM

Path instance sampling

CNN

Pool

ing

Pool

ing

Pool

ing

Pool

ing

CNN

CNN

CNN

User-item-meta-pathtriattention MLP

1 0 1User Lookup

MLP

MLP

1 0 1Item Lookup

Neighbor attention Lookup

3

6

9

User neighbors embedding All items embedding

Embedding

User neighbor

Neighbor attention Lookup

3

6

9

Item neighbors embedding All users embedding

Embedding

Item neighbor

r~u i

X~

uiy~

i

X~

u

yiy

i

XuX

u

Figure 1 e whole architecture of the TANN model

4 Complexity

where Pu⟶i is the set of meta-paths existing between theuser u and the item i e interaction embedding among theuser u the item i and their related meta-path P can berepresented as

1113957xui 1113944pisinPu⟶i

βPmiddot 1113957xuoplus1113957yiopluscp1113872 1113873 (16)

where oplus represents the vector concatenation operation

42 TANN Model-Based Recommendation ApproachOnce we obtain the interaction embedding we apply anMLP component to model complicated interactions

1113957rui MLP 1113957xui1113872 1113873 (17)

in which the MLP contains two hidden layers with ReLUnonlinear activation functions and an output layer withsigmoid functions 1113957rui is interpreted as the relevance scorebetween the user u and the item i and 1113957rui is used to generatea recommendation list for the user u

Defining an appropriate objective function is a criticalstep for model optimization following [14 17] we usenegative sampling to learn the modelrsquos parameters

ℓui minuslog 1113957rui minus 1113944c

j1log 1 minus 1113957ruj1113872 1113873 (18)

in which the first term is the observed interactions and thesecond term indicates the negative feedback drawn from thenoise distribution which is set to the uniform distribution (itcan be set to other biased distributions) and c is the numberof negative item sampling

5 Experiment and Evaluation

51 Datasets We use three datasets that is Douban Moviedataset (httpsgithubcomlibrahu) MovieLens moviedataset (httpsgrouplensorgdatasetsmovielens) and Yelpbusiness dataset (httpswwwyelpcomdataset) As a ratingindicates whether a user has rated an item we deem therating as an interaction record [47 50] Table 1 lists thedetailed description of the datasets Each datasetrsquos first rowshows the number of users items and their interactions andthe other rows list the other relationsrsquo statistics e man-ually selected meta-paths and SGA selected meta-paths ofeach dataset are listed in Table 2 Since long meta-paths mayimport noisy semantics [51] we set the length of meta-pathto 4 and set the number of each meta-pathrsquos selected pathinstances to 5

52 Evaluation Methods In order to evaluate the recom-mendationrsquos performance each datasetrsquos user implicitfeedback records are divided into a training set and test setaccording to a certain proportion For example we use 90feedback records to predict the remaining 10 feedbackrecords As it is a waste of time ranking each userrsquos itemsespecially for large datasets for each user in the dataset werandomly select 200 negative samples having no interactionrecords with the user at first After that we obtain each userrsquos

recommendation list by ranking the positive items andnegative items of the list We use PreK RecallK andNDCGK to evaluate the experimental results

When we apply the SGA algorithm to select the optimal4 meta-paths we follow the principle that ldquothe smaller theobjective function value is the larger the fitness value isrdquothat is we take negative evaluation scores We implementthe TANN model using TensorFlow with Keras (httpskerasio) e batch size is set to 256 the regularizationparameter is set to 00001 the learning rate is set to 0001and the dimension of user embedding and item embeddingis set to 64 We apply Adaptive Moment Estimation (Adam)[52] to optimize the model

53 Performance of TANN-Based Recommendation Toillustrate the benefits of applying the SGA algorithmweighted random walk strategy and integrating HINrsquosglobal and local information of TANN model we compareTANN against the following

(1) TANN wo SGA which applies manually selectedmeta-paths we use the meta-paths in the secondcolumn of Table 2

(2) TANN wo WRWS which applies a random walkstrategy (RWS) instead of weighted random walkstrategy (WRWS) to select meta-path instances

(3) TANN wo local which considers global informa-tion of HIN only ignoring local information that isit uses a global representation of users and itemsonly ignoring local representation of users anditems

Table 3 shows that the performance of TANN wo SGAmethod is the poorest which indicates that the selection ofmeta-paths has a great influence on the recommendationresults e performance of TANN wo RWRS is better thanthe performance of TANN wo SGA it indicates that themeta-path selection strategy plays an important role com-pared to the weighted random walk strategy for recom-mendation task In addition as results illustrate theperformance of TANN wo local is better than that of theabove two methods is can be mainly credited to the factthat the nodes and meta-paths in HIN contain rich implicit

Table 1 Statistics of the three datasets

Datasets Relations (A-B) A B A-B

MovieLens

User-movie 943 1682 100000User-user 943 943 47150

Movie-movie 1682 1682 82798Movie-type 1682 18 2891

Douban movie

User-movie 1830 12337 697311User-user 1830 1830 9155

Movie-movie 12337 12337 61690Movie-type 12337 38 12337

Yelp

User-business 300 450 15109User-user 300 300 1505

Business-business 450 450 2255Business-category 450 47 450

Complexity 5

higher-priority neighbor and using such walking strategy apath instance which contains more semantic informationcan be obtained for recommendation task en how todefine the priority of each node in a sequence is a keyproblem

Inspired by He et al [46] and Hinton and Salakhutdinov[49] we use a similar pretraining technique to measure eachcandidate nodersquos priority e basic idea is to take the scorebetween different nodes in the heterogeneous network as theweight allocation standard For example we define the scoreranges from 1 to 5 in film evaluation if the score of user u formovie i is 5 then we deem the weight value of the link betweenuser u andmovie i is the highest For each node in HIN it has aweight value for each node of its neighbors and the similarityscore between the node and the corresponding neighbor nodecan also be obtainedWemeasure the priority of each neighbornode by the product value between the weight value and thecorresponding similarity score Such score can reflect thecorrelationrsquos degree between the two nodes We use the abovepriority score strategy to construct each meta-pathrsquos instances

Finally for each meta-path we obtain a differentnumber of meta-path instances with a given length of L andthen calculate scores of these meta-path instances as fol-lows for a path instance we sum the product of weight andcosine similarity between adjacent nodes and then obtainthat the sum value is divided by L as the score of thecorresponding meta-path instance So we can obtain meta-path instancesrsquo scores of each meta-path and select the kpath instances with high scores as the selected meta-pathinstance

(3) Meta-Path Instance Embedding Since a meta-path is asequence of entity nodes we apply the convolution neuralnetwork (CNN) to map a meta-path into a low-dimensionalvector For a meta-path P we useXp isin RLtimesd to represent thepath embedding matrix where p is a path instance Lrepresents the path instancersquos length and d is the nodesrsquoembedding dimension e path instance prsquos embedding iscomputed as follows

hp CNN XpΘ( 1113857 (11)

where Θ denotes CNNrsquos related parameters

(4)Meta-Path Embedding As eachmeta-path contains manypath instances we first apply the proposed meta-path in-stance selection strategy to obtain each meta-pathrsquos top kpath instances en we use the maximum pooling oper-ation to get important dimensional features from the se-lected path instances Let hp1113966 1113967

k

p1denote k selected pathinstancesrsquo embedding e embedding of the meta-path P iscomputed as

cP max minus pooling hp1113966 1113967k

p11113874 1113875 (12)

(5) Tri-Attention Mechanism Based Interaction EmbeddingAs meta-paths contain rich semantic information differentusers through different meta-paths show different preferenceseven when the same user interacts with different items throughthe same meta-path semantic information contained in themeta-path is also different In order to better represent thesemantic information existing among users items and meta-paths we develop a tri-attention mechanism to assign differentweights to different triplets of user-item-meta-path

Given the user embedding xu item embedding yi meta-path embedding cpwhich exists between the user u and itemi we use two full-connection layers to get the triattentivescore as

β1P f Wu1 1113957xu + Wi

11113957yi + WP1 cP + b11113872 1113873 (13)

β2P f w2β1 + b2( 1113857 (14)

where Wlowast1 is the first layerrsquos weight matrix b1 is the firstlayerrsquos bias vectorw2 is the second layerrsquos weight vector andb2 is the second layerrsquos bias f(middot) is set to the ReLU function

We use softmax function to obtain the final interactionweights that is

βP exp β2P( 1113857

1113936ρprimeisinPu⟶iexp β2P( 1113857

(15)

Heterogeneous information network

Meta-path selection via

SGA

UMUM

UMTM

UMMM

UUUM

Path instance sampling

CNN

Pool

ing

Pool

ing

Pool

ing

Pool

ing

CNN

CNN

CNN

User-item-meta-pathtriattention MLP

1 0 1User Lookup

MLP

MLP

1 0 1Item Lookup

Neighbor attention Lookup

3

6

9

User neighbors embedding All items embedding

Embedding

User neighbor

Neighbor attention Lookup

3

6

9

Item neighbors embedding All users embedding

Embedding

Item neighbor

r~u i

X~

uiy~

i

X~

u

yiy

i

XuX

u

Figure 1 e whole architecture of the TANN model

4 Complexity

where Pu⟶i is the set of meta-paths existing between theuser u and the item i e interaction embedding among theuser u the item i and their related meta-path P can berepresented as

1113957xui 1113944pisinPu⟶i

βPmiddot 1113957xuoplus1113957yiopluscp1113872 1113873 (16)

where oplus represents the vector concatenation operation

42 TANN Model-Based Recommendation ApproachOnce we obtain the interaction embedding we apply anMLP component to model complicated interactions

1113957rui MLP 1113957xui1113872 1113873 (17)

in which the MLP contains two hidden layers with ReLUnonlinear activation functions and an output layer withsigmoid functions 1113957rui is interpreted as the relevance scorebetween the user u and the item i and 1113957rui is used to generatea recommendation list for the user u

Defining an appropriate objective function is a criticalstep for model optimization following [14 17] we usenegative sampling to learn the modelrsquos parameters

ℓui minuslog 1113957rui minus 1113944c

j1log 1 minus 1113957ruj1113872 1113873 (18)

in which the first term is the observed interactions and thesecond term indicates the negative feedback drawn from thenoise distribution which is set to the uniform distribution (itcan be set to other biased distributions) and c is the numberof negative item sampling

5 Experiment and Evaluation

51 Datasets We use three datasets that is Douban Moviedataset (httpsgithubcomlibrahu) MovieLens moviedataset (httpsgrouplensorgdatasetsmovielens) and Yelpbusiness dataset (httpswwwyelpcomdataset) As a ratingindicates whether a user has rated an item we deem therating as an interaction record [47 50] Table 1 lists thedetailed description of the datasets Each datasetrsquos first rowshows the number of users items and their interactions andthe other rows list the other relationsrsquo statistics e man-ually selected meta-paths and SGA selected meta-paths ofeach dataset are listed in Table 2 Since long meta-paths mayimport noisy semantics [51] we set the length of meta-pathto 4 and set the number of each meta-pathrsquos selected pathinstances to 5

52 Evaluation Methods In order to evaluate the recom-mendationrsquos performance each datasetrsquos user implicitfeedback records are divided into a training set and test setaccording to a certain proportion For example we use 90feedback records to predict the remaining 10 feedbackrecords As it is a waste of time ranking each userrsquos itemsespecially for large datasets for each user in the dataset werandomly select 200 negative samples having no interactionrecords with the user at first After that we obtain each userrsquos

recommendation list by ranking the positive items andnegative items of the list We use PreK RecallK andNDCGK to evaluate the experimental results

When we apply the SGA algorithm to select the optimal4 meta-paths we follow the principle that ldquothe smaller theobjective function value is the larger the fitness value isrdquothat is we take negative evaluation scores We implementthe TANN model using TensorFlow with Keras (httpskerasio) e batch size is set to 256 the regularizationparameter is set to 00001 the learning rate is set to 0001and the dimension of user embedding and item embeddingis set to 64 We apply Adaptive Moment Estimation (Adam)[52] to optimize the model

53 Performance of TANN-Based Recommendation Toillustrate the benefits of applying the SGA algorithmweighted random walk strategy and integrating HINrsquosglobal and local information of TANN model we compareTANN against the following

(1) TANN wo SGA which applies manually selectedmeta-paths we use the meta-paths in the secondcolumn of Table 2

(2) TANN wo WRWS which applies a random walkstrategy (RWS) instead of weighted random walkstrategy (WRWS) to select meta-path instances

(3) TANN wo local which considers global informa-tion of HIN only ignoring local information that isit uses a global representation of users and itemsonly ignoring local representation of users anditems

Table 3 shows that the performance of TANN wo SGAmethod is the poorest which indicates that the selection ofmeta-paths has a great influence on the recommendationresults e performance of TANN wo RWRS is better thanthe performance of TANN wo SGA it indicates that themeta-path selection strategy plays an important role com-pared to the weighted random walk strategy for recom-mendation task In addition as results illustrate theperformance of TANN wo local is better than that of theabove two methods is can be mainly credited to the factthat the nodes and meta-paths in HIN contain rich implicit

Table 1 Statistics of the three datasets

Datasets Relations (A-B) A B A-B

MovieLens

User-movie 943 1682 100000User-user 943 943 47150

Movie-movie 1682 1682 82798Movie-type 1682 18 2891

Douban movie

User-movie 1830 12337 697311User-user 1830 1830 9155

Movie-movie 12337 12337 61690Movie-type 12337 38 12337

Yelp

User-business 300 450 15109User-user 300 300 1505

Business-business 450 450 2255Business-category 450 47 450

Complexity 5

where Pu⟶i is the set of meta-paths existing between theuser u and the item i e interaction embedding among theuser u the item i and their related meta-path P can berepresented as

1113957xui 1113944pisinPu⟶i

βPmiddot 1113957xuoplus1113957yiopluscp1113872 1113873 (16)

where oplus represents the vector concatenation operation

42 TANN Model-Based Recommendation ApproachOnce we obtain the interaction embedding we apply anMLP component to model complicated interactions

1113957rui MLP 1113957xui1113872 1113873 (17)

in which the MLP contains two hidden layers with ReLUnonlinear activation functions and an output layer withsigmoid functions 1113957rui is interpreted as the relevance scorebetween the user u and the item i and 1113957rui is used to generatea recommendation list for the user u

Defining an appropriate objective function is a criticalstep for model optimization following [14 17] we usenegative sampling to learn the modelrsquos parameters

ℓui minuslog 1113957rui minus 1113944c

j1log 1 minus 1113957ruj1113872 1113873 (18)

in which the first term is the observed interactions and thesecond term indicates the negative feedback drawn from thenoise distribution which is set to the uniform distribution (itcan be set to other biased distributions) and c is the numberof negative item sampling

5 Experiment and Evaluation

51 Datasets We use three datasets that is Douban Moviedataset (httpsgithubcomlibrahu) MovieLens moviedataset (httpsgrouplensorgdatasetsmovielens) and Yelpbusiness dataset (httpswwwyelpcomdataset) As a ratingindicates whether a user has rated an item we deem therating as an interaction record [47 50] Table 1 lists thedetailed description of the datasets Each datasetrsquos first rowshows the number of users items and their interactions andthe other rows list the other relationsrsquo statistics e man-ually selected meta-paths and SGA selected meta-paths ofeach dataset are listed in Table 2 Since long meta-paths mayimport noisy semantics [51] we set the length of meta-pathto 4 and set the number of each meta-pathrsquos selected pathinstances to 5

52 Evaluation Methods In order to evaluate the recom-mendationrsquos performance each datasetrsquos user implicitfeedback records are divided into a training set and test setaccording to a certain proportion For example we use 90feedback records to predict the remaining 10 feedbackrecords As it is a waste of time ranking each userrsquos itemsespecially for large datasets for each user in the dataset werandomly select 200 negative samples having no interactionrecords with the user at first After that we obtain each userrsquos

recommendation list by ranking the positive items andnegative items of the list We use PreK RecallK andNDCGK to evaluate the experimental results

When we apply the SGA algorithm to select the optimal4 meta-paths we follow the principle that ldquothe smaller theobjective function value is the larger the fitness value isrdquothat is we take negative evaluation scores We implementthe TANN model using TensorFlow with Keras (httpskerasio) e batch size is set to 256 the regularizationparameter is set to 00001 the learning rate is set to 0001and the dimension of user embedding and item embeddingis set to 64 We apply Adaptive Moment Estimation (Adam)[52] to optimize the model

53 Performance of TANN-Based Recommendation Toillustrate the benefits of applying the SGA algorithmweighted random walk strategy and integrating HINrsquosglobal and local information of TANN model we compareTANN against the following

(1) TANN wo SGA which applies manually selectedmeta-paths we use the meta-paths in the secondcolumn of Table 2

(2) TANN wo WRWS which applies a random walkstrategy (RWS) instead of weighted random walkstrategy (WRWS) to select meta-path instances

(3) TANN wo local which considers global informa-tion of HIN only ignoring local information that isit uses a global representation of users and itemsonly ignoring local representation of users anditems

Table 3 shows that the performance of TANN wo SGAmethod is the poorest which indicates that the selection ofmeta-paths has a great influence on the recommendationresults e performance of TANN wo RWRS is better thanthe performance of TANN wo SGA it indicates that themeta-path selection strategy plays an important role com-pared to the weighted random walk strategy for recom-mendation task In addition as results illustrate theperformance of TANN wo local is better than that of theabove two methods is can be mainly credited to the factthat the nodes and meta-paths in HIN contain rich implicit

Table 1 Statistics of the three datasets

Datasets Relations (A-B) A B A-B

MovieLens

User-movie 943 1682 100000User-user 943 943 47150

Movie-movie 1682 1682 82798Movie-type 1682 18 2891

Douban movie

User-movie 1830 12337 697311User-user 1830 1830 9155

Movie-movie 12337 12337 61690Movie-type 12337 38 12337

Yelp

User-business 300 450 15109User-user 300 300 1505

Business-business 450 450 2255Business-category 450 47 450

Complexity 5

and effective information TANN wo local method appliesSGA to automatically obtain meta-paths avoiding artificialinterference and it uses the weighted random walk strategyto get optimal meta-path instances that can represent theinformation of heterogeneous information network struc-ture TANN method which not only considers informationof users items andmeta-paths but also considers themutualinfluence among them consistently outperforms the otherthree methods

54 Comparison with Other Recommendation ApproachesMoreover we compare our proposed TANNmethod withother four recommendation methods (1) BPR (BayesianPersonalized Ranking) [53] which is a Bayesian posterioroptimization based personalized ranking algorithm (2)LRML (Latent Relational Metric Learning) [54] whichemploys an augmented memory model to construct latentrelations between each user-item interaction (3) CDAE(Collaborative Denoising Autoencoders) [55] which usesa Denoising Autoencoder structure to learn users anditemsrsquo distributed representations and (4) MCRec (Meta-path based Context for RECommendation) [44] whichleverages rich meta-paths and co-attention mechanismTable 4 compares the experimental results

From Table 4 we can see that LRML performs the poorestas it only learns relations that describe each user-item inter-action CDAE learns the corrupted user-item preferencesrsquo latentrepresentations that can best reconstruct the full input so itperforms better than LRML Both LRML and CADE con-centrate on explicit feedback BPR uses not only explicitfeedback but also implicit feedback thus it can obtain betterresults than LRML andCADEMCRec learns representations ofusers items and meta-path based context as it encodes muchmore information than the previous methods its performanceis better than the above three methods MCRec selects meta-path manually and only utilizes global information of users anditems while our proposed TANN method selects meta-pathautomatically uses local and global information of users anditems applies a tri-attention mechanism to enhance the users

items and meta-pathsrsquo representations and its performanceconsistently outperforms the other four recommendationmethods

55 Impact of Meta-Paths Selection In this set of experi-ments we examine whether the automatically selected meta-paths can produce better recommendation performancethan manually selected meta-paths e experiments areconducted on theMovieLens datasete optimal meta-pathset selected by SGA is UMTM UUMM UMUM andUMMM (P1) We randomly selected three different meta-path sets UMTM UMUM UUUM and UMMM (P2)UMUM UUMM UUUM and UMMM (P3) and UUMMUUUM UMTM and UMMM (P4) e recommendationperformance with different meta-path sets is shown inFigure 2

We can observe in Figure 2 that the recommendationperformance based on the meta-path set which is selected bySGA algorithm is better than the performance based on thethree manually selected meta-path sets e experimentalresults show that the interference of human factors should beavoided in the construction of heterogeneous informationnetwork

56 Impact of Usersrsquo and Itemsrsquo Local Information We studywhether incorporating usersrsquo and itemsrsquo local informationcan further enhance the recommendation performance eexperiments are conducted on the Yelp dataset We ran-domly select a user from the user list and obtain the rec-ommendation list withwithout local information of usersand items In Figure 3 the ground-truth of the userrsquospreference movie ids is listed in the middle column themovie ids using TANN wo local information method arelisted in the left column and the movie ids using TANN wilocal information method are listed in the right column ebold ids in the left and right columns indicate the matchedresults with the ground-truth ids We can see that whenintegrating local information the recommendation methodcan find more accurate results than without local

Table 2 Manually selected meta-paths and the meta-paths selected by SGA in three datasets

Datasets Manually selected meta-paths Meta-paths selected by SGAMovieLens UMTM UUUM UMUM UMMM UMTM UUMM UMUM UMMMDouban movie UMMM UMUM UUUM UUMM UMTM UMUM UMMM UUMMYelp UUUB UUBB UBUB UBCB UUBB UBUB UBBB UBCB

Table 3 Experimental results of TANN-based recommendation approach on the four datasets

ModelMovieLens Douban movie Yelp

NDCG10 Prec10 Recall10 NDCG10 Prec10 Recall10 NDCG10 Prec10 Recall10TANN 06968 03476 02250 09397 08082 01186 04285 01363 02554TANN wo local 06957 03467 02242 09333 08042 01177 04184 01332 02488TANN wo RWRS 06901 03463 02235 09301 07878 01156 04120 01330 02442TANN wo SGA 06895 03451 02234 09087 07468 01094 04063 01293 02323

6 Complexity

Table 4 Comparison with other recommendation approaches on the three datasets

ModelMovieLens Douban movie Yelp

NDCG10 Prec10 Recall10 NDCG10 Prec10 Recall10 NDCG10 Prec10 Recall10TANN 06968 03467 02235 09397 08082 01186 04285 01363 02554MCRec 06909 03411 02192 09365 08080 01194 04212 01343 02435BPR 06905 03386 02176 06129 02811 00393 04192 01297 02342CDAE 06303 03039 01906 06008 02716 00374 03948 01167 02241LRML 05381 02242 01418 04855 02649 00256 03934 01147 02124

Precision10 Recall10 NDCG100

0102030405060708

Eval

uatio

n sc

ore

P1P2

P3P4

Figure 2 Recommendation performance with different meta-path sets

Movie ids obtained byTANN wo local information method

Movie ids obtained byTANN wi local information method

129174

66621991261212333164582650126369

u1The movie ids of the users preference

4531609144003458958355179055817412917351

74 80 949 2282 34584129 4400 4531 59946091 6728 7145 81748945 9018 9055 9583

Figure 3 An example of removing local information in the TANN method

16 32 64 128 256016

023

03

037

Eval

uatio

n sc

ore

Precision10Recall10

(a)

01

019

028

037

Eval

uatio

n sc

ore

3 5 7 91

Precision10Recall10

(b)

Figure 4 Continued

Complexity 7

information It illustrates that local information has animpact on improving the recommendation result Com-pared with the global information the local information canreflect the neighborhood characteristics of the nodescentrally

57 Parameter Tuning Our model includes a few importantparameters to tune In this section we examine three pa-rametersrsquo performance effect on MovieLens dataset that isthe embedding size for the weighting vector of attentivescores (ie the embedding size of w2 in equation (14)) thenegative samplesrsquo number (in equation (18)) and the outputlayerrsquos embedding size

For the embedding size of the tri-attention mechanismwe vary it in the set of 16 32 64 128 256 As shown inFigure 4(a) our method achieves the best performance whenit is 128 For the negative samplesrsquo number we vary it in theset of 1 3 5 7 9 We can find from Figure 4(b) that when 5negative items are taken for each positive item the evalu-ation result is the best For the embedding size of the outputlayer we vary it in the set of 8 16 32 64 and the best resultcan be obtained when the embedding size of the output layeris 8 e optimal performance is obtained with 128-di-mension of the tri-attention mechanism 5 negative samplesand 8-dimension of the output layer

6 Conclusion

We present a tri-attention neural network (TANN)model for the recommendation in this paper We firstapply the stud genetic algorithm to automatically selectmeta-paths and propose a tri-attention mechanism toenhance the mutual influence among users items andtheir related meta-paths en we encode the interactioninformation among the above three objects which containmore semantic information and can be used for therecommendation task Extensive experiments on theDouban Movie MovieLens and Yelp datasets demon-strated the outstanding performance of the proposedapproach In the future we will explore other auxiliary

information in the heterogeneous information networkfor the recommendation task

Data Availability

e authors have presented the website of the three datasetsin Section 51 Readers can download the datasets from thecorresponding link or can contact the correspondingauthor

Conflicts of Interest

e authors declare that they have no conflicts of interest

Acknowledgments

is work was partially supported by the National NaturalScience Foundation of China (nos 61872296 and 61772429)MOE (Ministry of Education in China) Project of Hu-manities and Social Sciences (18YJC870001) and the Fun-damental Research Funds for the Central Universities underGrant 3102019ZDHKY04

References

[1] L Sharma and A Gera ldquoA survey of recommendation systemResearch challengesrdquo International Journal of EngineeringTrends and Technology vol 4 no 5 pp 1989ndash1992 2013

[2] M Balabanovic and Y Shoham ldquoFab content-based col-laborative recommendationrdquo Communications of the ACMvol 40 no 3 pp 66ndash72 1997

[3] KG SarwarBM and RJ KonstanJA ldquoApplication of dimen-sionality reduction in recommender system mdash a case studyrdquoin Proceedings of the ACM WebKDD Web Mining forE-Commerce Workshop vol 82 p 90 Boston MA USA2000

[4] X Ning C Desrosiers and G Karypis A ComprehensiveSurvey of Neighborhood-Based Recommendation MethodsRecommender Systems Handbook pp 37ndash76 Springer Bos-ton MA USA 2015

[5] Z Li H Chen F Xiong X Wang and X Xiong ldquoTopologicalinfluence-aware recommendation on social networksrdquoComplexity vol 2019 Article ID 6325654 12 pages 2019

01

019

028

037

Eval

uatio

n sc

ore

16 32 648

Precision10Recall10

(c)

Figure 4 Parameter tuning of TANN on MovieLens dataset (a) e embedding size for the weighting vector of attentive scores (b) enumber of negative samples (c) e embedding size for the output layer

8 Complexity

[6] X Ning and G K Slim ldquoSparse linear methods for top-nrecommender systemsrdquo in Proceedings of the 2011 IEEE 11thInternational Conference on Data Mining pp 497ndash506Vancouver Canada 2011

[7] Y C Hu ldquoRecommendation using neighborhood methodswith preference-relation-based similarityrdquo Information Sci-ences vol 284 pp 18ndash30 2014

[8] C Paolo Y Koren and R Turrin ldquoPerformance of recom-mender algorithms on top-n recommendation tasksrdquo inProceedings of the Fourth ACM Conference on RecommenderSystems ACM Barcelona Spain pp 39ndash46 2010

[9] S Pan X Wang H Yang et al ldquoSocial recommendation withevolutionary opinion dynamicsrdquo IEEE Transactions on Sys-tems Man and Cybernetics Systems 2018

[10] Y Hu Y Koren and C Volinsky ldquoCollaborative filtering forimplicit feedback datasetsrdquo in Proceedings of the InternationalConference on Data Mining pp 263ndash272 Pisa Italy De-cember 2008

[11] A N Nikolakopoulos V Kalantzis E Gallopoulos andJ D Garofalakis ldquoEigenRec generalizing PureSVD for ef-fective and efficient top-N recommendationsrdquoKnowledge andInformation Systems vol 58 no 1 pp 59ndash81 2019

[12] F Christoffel B Paudel C Newell and A BernsteinldquoBlockbusters and wallflowers accurate diverse and scalablerecommendations with random walksrdquo in Proceedings of the9th ACM Conference on Recommender Systems pp 163ndash170Vienna Austria September 2015

[13] S Pan R Hu S Fung et al ldquoLearning graph embedding withadversarial training methodsrdquo IEEE Transactions on Cyber-netics vol 50 2019

[14] C Cooper S H Lee T Radzik and Y Siantos ldquoRandomwalks in recommender systems exact computation andsimulationsrdquo in Proceedings of the 23rd International Con-ference on World Wide Web ACM New York NY USApp 811ndash816 2014

[15] S Ji S Pan E Cambria P Marttinen and P S Yu ldquoA surveyon knowledge graphs representation acquisition and ap-plicationsrdquo 2020 httparxivorgabs200200388

[16] Z Kang C Peng M Yang and Q Cheng ldquoTop-N recom-mendation on graphsrdquo in Proceedings of the 25th ACM In-ternational on Conference on Information and KnowledgeManagement pp 2101ndash2106 Toronto Canada 2016

[17] A M Elkahky Y Song and X He ldquoA multi-view deeplearning approach for cross domain user modeling in rec-ommendation systemsrdquo in Proceedings of the Twenty-FourthInternational Conference on World Wide Web ACM NewYork NY USA pp 278ndash288 2015

[18] B Hidasi A Karatzoglou L Baltrunas and D Tikk ldquoSession-based recommendations with recurrent neural networksrdquo inProceedings of the International Conference on LearningRepresentations Vancouver Canada 2016

[19] A V Den Oord S Dieleman and B Schrauwen ldquoDeepcontent-based music recommendationrdquo Neural InformationProcessing Systems pp 2643ndash2651 2013

[20] R Hu G Long C Zhang et al ldquoLearning graph embeddingwith adversarial training methodsrdquo IEEE Transactions OnCybernetics vol 50 2019

[21] H Wang N Wang and D Yeung ldquoCollaborative deeplearning for recommender systemsrdquo in Proceedings of theTwenty-First ACM SIGKDD International Conference onKnowledge Discovery and Data Mining ACM New York NYUSA pp 1235ndash1244 2015

[22] X Yao B Tan C Hu W Li Z Xu and Z Zhang ldquoRec-ommend algorithm combined user-user neighborhood

approach with latent factor modelrdquo in Proceedings of theInternational Conference on Mechatronics and IntelligentRobotics Springer Cham Switzerland pp 275ndash280 2017

[23] H Zhang F Min and SWang ldquoA random forest approach tomodel-based recommendationrdquo Journal of Information andComputational Science vol 11 no 15 pp 5341ndash5348 2014

[24] G mavicius and A Tuzhilin ldquoContext-aware recommendersystemsrdquo in Proceedings of the 2008 ACM Conference onRecommender Systems RecSys 2008 Lausanne Switzerlandpp 191ndash226 October 2008

[25] R Ronen E Yomtov and G Lavee ldquoRecommendations meetweb browsing enhancing collaborative filtering using internetbrowsing logsrdquo in Proceedings of the IEEE 32nd InternationalConference on Data Engineering pp 1230ndash1238 HelsinkiFinland 2016

[26] X Yu X Ren Y Sun et al ldquoPersonalized entity recom-mendation a heterogeneous information network approachrdquoin Proceedings of the 7th ACM International Conference onWeb Search and Data Mining pp 283ndash292 New York CityNY USA 2014

[27] H Zhao Q Yao J Li Y Song and D L Lee ldquoMetagraphbased recommendation fusion over heterogeneous informa-tion networksrdquo in Proceedings of the 23rd ACM SIGKDDInternational Conference on Knowledge Discovery and DataMining pp 635ndash644Halifax Nova Scotia Canada 2017

[28] F Alqadah C K Reddy J Hu and H F AlqadahldquoBiclustering neighborhood-based collaborative filteringmethod for top-n recommender systemsrdquo Knowledge andInformation Systems vol 44 no 2 pp 475ndash491 2015

[29] Z Yao J Wang and Y Han ldquoAn improved neighborhood-based recommendation algorithm optimization with clus-tering analysis and latent factor modelrdquo in Proceedings of the38th Chinese Control Conference pp 27ndash30 GuangzhouChina 2019

[30] P Cremonesi Y Koren and R Turrin ldquoPerformance ofrecommender algorithms on top-n recommendation tasksrdquoin Proceedings of the Fourth ACM Conference on Recom-mender Systems ACM New York NY USA pp 39ndash46 2010

[31] T Guo S Pan X Zhu and C Zhang ldquoCFOND consensusfactorization for co-clustering networked datardquo IEEETransactions on Knowledge and Data Engineering vol 31no 4 pp 706ndash719 2018

[32] V Sindhwani S S Bucak J Hu and AMojsilovic ldquoOne-classmatrix completion with low-density factorizationsrdquo in Pro-ceedings of the 2010 IEEE International Conference on DataMining Ser ICDM rsquo10 IEEE Computer Society WashingtonDC USA pp 1055ndash1060 2010

[33] F Xiong W Shen H Chen et al ldquoExploiting implicit in-fluence from information propagation for social recom-mendationrdquo IEEE Transactions on Cybernetics 2019

[34] T Hofmann ldquoLatent semantic models for collaborative fil-teringrdquo ACM Transactions on Information Systems (TOIS)vol 22 no 1 pp 89ndash115 2004

[35] S Baluja R Seth D Sivakumar et al ldquoVideo suggestion anddiscovery for youtube taking random walks through the viewgraphrdquo in Proceedings of the 17th International Conference onWorld Wide Web pp 895ndash904 Beijing China 2008

[36] A Van den Oord S Dieleman and B Schrauwen ldquoDeepcontent-based music recommendationrdquo in Proceedings of theAdvances in Neural Information Processing Systemspp 2643ndash2651 Lake Tahoe NE USA 2013

[37] X Wang and Y Wang Improving Content-Based and HybridMusic Recommendation Using Deep Learning pp 627ndash636ACM Multimedia Seattle WA USA 2014

Complexity 9

Table 4 Comparison with other recommendation approaches on the three datasets

ModelMovieLens Douban movie Yelp

NDCG10 Prec10 Recall10 NDCG10 Prec10 Recall10 NDCG10 Prec10 Recall10TANN 06968 03467 02235 09397 08082 01186 04285 01363 02554MCRec 06909 03411 02192 09365 08080 01194 04212 01343 02435BPR 06905 03386 02176 06129 02811 00393 04192 01297 02342CDAE 06303 03039 01906 06008 02716 00374 03948 01167 02241LRML 05381 02242 01418 04855 02649 00256 03934 01147 02124

Precision10 Recall10 NDCG100

0102030405060708

Eval

uatio

n sc

ore

P1P2

P3P4

Figure 2 Recommendation performance with different meta-path sets

Movie ids obtained byTANN wo local information method

Movie ids obtained byTANN wi local information method

129174

66621991261212333164582650126369

u1The movie ids of the users preference

4531609144003458958355179055817412917351

74 80 949 2282 34584129 4400 4531 59946091 6728 7145 81748945 9018 9055 9583

Figure 3 An example of removing local information in the TANN method

16 32 64 128 256016

023

03

037

Eval

uatio

n sc

ore

Precision10Recall10

(a)

01

019

028

037

Eval

uatio

n sc

ore

3 5 7 91

Precision10Recall10

(b)

Figure 4 Continued

Complexity 7

information It illustrates that local information has animpact on improving the recommendation result Com-pared with the global information the local information canreflect the neighborhood characteristics of the nodescentrally

57 Parameter Tuning Our model includes a few importantparameters to tune In this section we examine three pa-rametersrsquo performance effect on MovieLens dataset that isthe embedding size for the weighting vector of attentivescores (ie the embedding size of w2 in equation (14)) thenegative samplesrsquo number (in equation (18)) and the outputlayerrsquos embedding size

For the embedding size of the tri-attention mechanismwe vary it in the set of 16 32 64 128 256 As shown inFigure 4(a) our method achieves the best performance whenit is 128 For the negative samplesrsquo number we vary it in theset of 1 3 5 7 9 We can find from Figure 4(b) that when 5negative items are taken for each positive item the evalu-ation result is the best For the embedding size of the outputlayer we vary it in the set of 8 16 32 64 and the best resultcan be obtained when the embedding size of the output layeris 8 e optimal performance is obtained with 128-di-mension of the tri-attention mechanism 5 negative samplesand 8-dimension of the output layer

6 Conclusion

We present a tri-attention neural network (TANN)model for the recommendation in this paper We firstapply the stud genetic algorithm to automatically selectmeta-paths and propose a tri-attention mechanism toenhance the mutual influence among users items andtheir related meta-paths en we encode the interactioninformation among the above three objects which containmore semantic information and can be used for therecommendation task Extensive experiments on theDouban Movie MovieLens and Yelp datasets demon-strated the outstanding performance of the proposedapproach In the future we will explore other auxiliary

information in the heterogeneous information networkfor the recommendation task

Data Availability

e authors have presented the website of the three datasetsin Section 51 Readers can download the datasets from thecorresponding link or can contact the correspondingauthor

Conflicts of Interest

e authors declare that they have no conflicts of interest

Acknowledgments

is work was partially supported by the National NaturalScience Foundation of China (nos 61872296 and 61772429)MOE (Ministry of Education in China) Project of Hu-manities and Social Sciences (18YJC870001) and the Fun-damental Research Funds for the Central Universities underGrant 3102019ZDHKY04

References

[1] L Sharma and A Gera ldquoA survey of recommendation systemResearch challengesrdquo International Journal of EngineeringTrends and Technology vol 4 no 5 pp 1989ndash1992 2013

[2] M Balabanovic and Y Shoham ldquoFab content-based col-laborative recommendationrdquo Communications of the ACMvol 40 no 3 pp 66ndash72 1997

[3] KG SarwarBM and RJ KonstanJA ldquoApplication of dimen-sionality reduction in recommender system mdash a case studyrdquoin Proceedings of the ACM WebKDD Web Mining forE-Commerce Workshop vol 82 p 90 Boston MA USA2000

[4] X Ning C Desrosiers and G Karypis A ComprehensiveSurvey of Neighborhood-Based Recommendation MethodsRecommender Systems Handbook pp 37ndash76 Springer Bos-ton MA USA 2015

[5] Z Li H Chen F Xiong X Wang and X Xiong ldquoTopologicalinfluence-aware recommendation on social networksrdquoComplexity vol 2019 Article ID 6325654 12 pages 2019

01

019

028

037

Eval

uatio

n sc

ore

16 32 648

Precision10Recall10

(c)

Figure 4 Parameter tuning of TANN on MovieLens dataset (a) e embedding size for the weighting vector of attentive scores (b) enumber of negative samples (c) e embedding size for the output layer

8 Complexity

[6] X Ning and G K Slim ldquoSparse linear methods for top-nrecommender systemsrdquo in Proceedings of the 2011 IEEE 11thInternational Conference on Data Mining pp 497ndash506Vancouver Canada 2011

[7] Y C Hu ldquoRecommendation using neighborhood methodswith preference-relation-based similarityrdquo Information Sci-ences vol 284 pp 18ndash30 2014

[8] C Paolo Y Koren and R Turrin ldquoPerformance of recom-mender algorithms on top-n recommendation tasksrdquo inProceedings of the Fourth ACM Conference on RecommenderSystems ACM Barcelona Spain pp 39ndash46 2010

[9] S Pan X Wang H Yang et al ldquoSocial recommendation withevolutionary opinion dynamicsrdquo IEEE Transactions on Sys-tems Man and Cybernetics Systems 2018

[10] Y Hu Y Koren and C Volinsky ldquoCollaborative filtering forimplicit feedback datasetsrdquo in Proceedings of the InternationalConference on Data Mining pp 263ndash272 Pisa Italy De-cember 2008

[11] A N Nikolakopoulos V Kalantzis E Gallopoulos andJ D Garofalakis ldquoEigenRec generalizing PureSVD for ef-fective and efficient top-N recommendationsrdquoKnowledge andInformation Systems vol 58 no 1 pp 59ndash81 2019

[12] F Christoffel B Paudel C Newell and A BernsteinldquoBlockbusters and wallflowers accurate diverse and scalablerecommendations with random walksrdquo in Proceedings of the9th ACM Conference on Recommender Systems pp 163ndash170Vienna Austria September 2015

[13] S Pan R Hu S Fung et al ldquoLearning graph embedding withadversarial training methodsrdquo IEEE Transactions on Cyber-netics vol 50 2019

[14] C Cooper S H Lee T Radzik and Y Siantos ldquoRandomwalks in recommender systems exact computation andsimulationsrdquo in Proceedings of the 23rd International Con-ference on World Wide Web ACM New York NY USApp 811ndash816 2014

[15] S Ji S Pan E Cambria P Marttinen and P S Yu ldquoA surveyon knowledge graphs representation acquisition and ap-plicationsrdquo 2020 httparxivorgabs200200388

[16] Z Kang C Peng M Yang and Q Cheng ldquoTop-N recom-mendation on graphsrdquo in Proceedings of the 25th ACM In-ternational on Conference on Information and KnowledgeManagement pp 2101ndash2106 Toronto Canada 2016

[17] A M Elkahky Y Song and X He ldquoA multi-view deeplearning approach for cross domain user modeling in rec-ommendation systemsrdquo in Proceedings of the Twenty-FourthInternational Conference on World Wide Web ACM NewYork NY USA pp 278ndash288 2015

[18] B Hidasi A Karatzoglou L Baltrunas and D Tikk ldquoSession-based recommendations with recurrent neural networksrdquo inProceedings of the International Conference on LearningRepresentations Vancouver Canada 2016

[19] A V Den Oord S Dieleman and B Schrauwen ldquoDeepcontent-based music recommendationrdquo Neural InformationProcessing Systems pp 2643ndash2651 2013

[20] R Hu G Long C Zhang et al ldquoLearning graph embeddingwith adversarial training methodsrdquo IEEE Transactions OnCybernetics vol 50 2019

[21] H Wang N Wang and D Yeung ldquoCollaborative deeplearning for recommender systemsrdquo in Proceedings of theTwenty-First ACM SIGKDD International Conference onKnowledge Discovery and Data Mining ACM New York NYUSA pp 1235ndash1244 2015

[22] X Yao B Tan C Hu W Li Z Xu and Z Zhang ldquoRec-ommend algorithm combined user-user neighborhood

approach with latent factor modelrdquo in Proceedings of theInternational Conference on Mechatronics and IntelligentRobotics Springer Cham Switzerland pp 275ndash280 2017

[23] H Zhang F Min and SWang ldquoA random forest approach tomodel-based recommendationrdquo Journal of Information andComputational Science vol 11 no 15 pp 5341ndash5348 2014

[24] G mavicius and A Tuzhilin ldquoContext-aware recommendersystemsrdquo in Proceedings of the 2008 ACM Conference onRecommender Systems RecSys 2008 Lausanne Switzerlandpp 191ndash226 October 2008

[25] R Ronen E Yomtov and G Lavee ldquoRecommendations meetweb browsing enhancing collaborative filtering using internetbrowsing logsrdquo in Proceedings of the IEEE 32nd InternationalConference on Data Engineering pp 1230ndash1238 HelsinkiFinland 2016

[26] X Yu X Ren Y Sun et al ldquoPersonalized entity recom-mendation a heterogeneous information network approachrdquoin Proceedings of the 7th ACM International Conference onWeb Search and Data Mining pp 283ndash292 New York CityNY USA 2014

[27] H Zhao Q Yao J Li Y Song and D L Lee ldquoMetagraphbased recommendation fusion over heterogeneous informa-tion networksrdquo in Proceedings of the 23rd ACM SIGKDDInternational Conference on Knowledge Discovery and DataMining pp 635ndash644Halifax Nova Scotia Canada 2017

[28] F Alqadah C K Reddy J Hu and H F AlqadahldquoBiclustering neighborhood-based collaborative filteringmethod for top-n recommender systemsrdquo Knowledge andInformation Systems vol 44 no 2 pp 475ndash491 2015

[29] Z Yao J Wang and Y Han ldquoAn improved neighborhood-based recommendation algorithm optimization with clus-tering analysis and latent factor modelrdquo in Proceedings of the38th Chinese Control Conference pp 27ndash30 GuangzhouChina 2019

[30] P Cremonesi Y Koren and R Turrin ldquoPerformance ofrecommender algorithms on top-n recommendation tasksrdquoin Proceedings of the Fourth ACM Conference on Recom-mender Systems ACM New York NY USA pp 39ndash46 2010

[31] T Guo S Pan X Zhu and C Zhang ldquoCFOND consensusfactorization for co-clustering networked datardquo IEEETransactions on Knowledge and Data Engineering vol 31no 4 pp 706ndash719 2018

[32] V Sindhwani S S Bucak J Hu and AMojsilovic ldquoOne-classmatrix completion with low-density factorizationsrdquo in Pro-ceedings of the 2010 IEEE International Conference on DataMining Ser ICDM rsquo10 IEEE Computer Society WashingtonDC USA pp 1055ndash1060 2010

[33] F Xiong W Shen H Chen et al ldquoExploiting implicit in-fluence from information propagation for social recom-mendationrdquo IEEE Transactions on Cybernetics 2019

[34] T Hofmann ldquoLatent semantic models for collaborative fil-teringrdquo ACM Transactions on Information Systems (TOIS)vol 22 no 1 pp 89ndash115 2004

[35] S Baluja R Seth D Sivakumar et al ldquoVideo suggestion anddiscovery for youtube taking random walks through the viewgraphrdquo in Proceedings of the 17th International Conference onWorld Wide Web pp 895ndash904 Beijing China 2008

[36] A Van den Oord S Dieleman and B Schrauwen ldquoDeepcontent-based music recommendationrdquo in Proceedings of theAdvances in Neural Information Processing Systemspp 2643ndash2651 Lake Tahoe NE USA 2013

[37] X Wang and Y Wang Improving Content-Based and HybridMusic Recommendation Using Deep Learning pp 627ndash636ACM Multimedia Seattle WA USA 2014

Complexity 9

information It illustrates that local information has animpact on improving the recommendation result Com-pared with the global information the local information canreflect the neighborhood characteristics of the nodescentrally

57 Parameter Tuning Our model includes a few importantparameters to tune In this section we examine three pa-rametersrsquo performance effect on MovieLens dataset that isthe embedding size for the weighting vector of attentivescores (ie the embedding size of w2 in equation (14)) thenegative samplesrsquo number (in equation (18)) and the outputlayerrsquos embedding size

For the embedding size of the tri-attention mechanismwe vary it in the set of 16 32 64 128 256 As shown inFigure 4(a) our method achieves the best performance whenit is 128 For the negative samplesrsquo number we vary it in theset of 1 3 5 7 9 We can find from Figure 4(b) that when 5negative items are taken for each positive item the evalu-ation result is the best For the embedding size of the outputlayer we vary it in the set of 8 16 32 64 and the best resultcan be obtained when the embedding size of the output layeris 8 e optimal performance is obtained with 128-di-mension of the tri-attention mechanism 5 negative samplesand 8-dimension of the output layer

6 Conclusion

We present a tri-attention neural network (TANN)model for the recommendation in this paper We firstapply the stud genetic algorithm to automatically selectmeta-paths and propose a tri-attention mechanism toenhance the mutual influence among users items andtheir related meta-paths en we encode the interactioninformation among the above three objects which containmore semantic information and can be used for therecommendation task Extensive experiments on theDouban Movie MovieLens and Yelp datasets demon-strated the outstanding performance of the proposedapproach In the future we will explore other auxiliary

information in the heterogeneous information networkfor the recommendation task

Data Availability

e authors have presented the website of the three datasetsin Section 51 Readers can download the datasets from thecorresponding link or can contact the correspondingauthor

Conflicts of Interest

e authors declare that they have no conflicts of interest

Acknowledgments

is work was partially supported by the National NaturalScience Foundation of China (nos 61872296 and 61772429)MOE (Ministry of Education in China) Project of Hu-manities and Social Sciences (18YJC870001) and the Fun-damental Research Funds for the Central Universities underGrant 3102019ZDHKY04

References

[1] L Sharma and A Gera ldquoA survey of recommendation systemResearch challengesrdquo International Journal of EngineeringTrends and Technology vol 4 no 5 pp 1989ndash1992 2013

[2] M Balabanovic and Y Shoham ldquoFab content-based col-laborative recommendationrdquo Communications of the ACMvol 40 no 3 pp 66ndash72 1997

[3] KG SarwarBM and RJ KonstanJA ldquoApplication of dimen-sionality reduction in recommender system mdash a case studyrdquoin Proceedings of the ACM WebKDD Web Mining forE-Commerce Workshop vol 82 p 90 Boston MA USA2000

[4] X Ning C Desrosiers and G Karypis A ComprehensiveSurvey of Neighborhood-Based Recommendation MethodsRecommender Systems Handbook pp 37ndash76 Springer Bos-ton MA USA 2015

[5] Z Li H Chen F Xiong X Wang and X Xiong ldquoTopologicalinfluence-aware recommendation on social networksrdquoComplexity vol 2019 Article ID 6325654 12 pages 2019

01

019

028

037

Eval

uatio

n sc

ore

16 32 648

Precision10Recall10

(c)

Figure 4 Parameter tuning of TANN on MovieLens dataset (a) e embedding size for the weighting vector of attentive scores (b) enumber of negative samples (c) e embedding size for the output layer

8 Complexity

[6] X Ning and G K Slim ldquoSparse linear methods for top-nrecommender systemsrdquo in Proceedings of the 2011 IEEE 11thInternational Conference on Data Mining pp 497ndash506Vancouver Canada 2011

[7] Y C Hu ldquoRecommendation using neighborhood methodswith preference-relation-based similarityrdquo Information Sci-ences vol 284 pp 18ndash30 2014

[8] C Paolo Y Koren and R Turrin ldquoPerformance of recom-mender algorithms on top-n recommendation tasksrdquo inProceedings of the Fourth ACM Conference on RecommenderSystems ACM Barcelona Spain pp 39ndash46 2010

[9] S Pan X Wang H Yang et al ldquoSocial recommendation withevolutionary opinion dynamicsrdquo IEEE Transactions on Sys-tems Man and Cybernetics Systems 2018

[10] Y Hu Y Koren and C Volinsky ldquoCollaborative filtering forimplicit feedback datasetsrdquo in Proceedings of the InternationalConference on Data Mining pp 263ndash272 Pisa Italy De-cember 2008

[11] A N Nikolakopoulos V Kalantzis E Gallopoulos andJ D Garofalakis ldquoEigenRec generalizing PureSVD for ef-fective and efficient top-N recommendationsrdquoKnowledge andInformation Systems vol 58 no 1 pp 59ndash81 2019

[12] F Christoffel B Paudel C Newell and A BernsteinldquoBlockbusters and wallflowers accurate diverse and scalablerecommendations with random walksrdquo in Proceedings of the9th ACM Conference on Recommender Systems pp 163ndash170Vienna Austria September 2015

[13] S Pan R Hu S Fung et al ldquoLearning graph embedding withadversarial training methodsrdquo IEEE Transactions on Cyber-netics vol 50 2019

[14] C Cooper S H Lee T Radzik and Y Siantos ldquoRandomwalks in recommender systems exact computation andsimulationsrdquo in Proceedings of the 23rd International Con-ference on World Wide Web ACM New York NY USApp 811ndash816 2014

[15] S Ji S Pan E Cambria P Marttinen and P S Yu ldquoA surveyon knowledge graphs representation acquisition and ap-plicationsrdquo 2020 httparxivorgabs200200388

[16] Z Kang C Peng M Yang and Q Cheng ldquoTop-N recom-mendation on graphsrdquo in Proceedings of the 25th ACM In-ternational on Conference on Information and KnowledgeManagement pp 2101ndash2106 Toronto Canada 2016

[17] A M Elkahky Y Song and X He ldquoA multi-view deeplearning approach for cross domain user modeling in rec-ommendation systemsrdquo in Proceedings of the Twenty-FourthInternational Conference on World Wide Web ACM NewYork NY USA pp 278ndash288 2015

[18] B Hidasi A Karatzoglou L Baltrunas and D Tikk ldquoSession-based recommendations with recurrent neural networksrdquo inProceedings of the International Conference on LearningRepresentations Vancouver Canada 2016

[19] A V Den Oord S Dieleman and B Schrauwen ldquoDeepcontent-based music recommendationrdquo Neural InformationProcessing Systems pp 2643ndash2651 2013

[20] R Hu G Long C Zhang et al ldquoLearning graph embeddingwith adversarial training methodsrdquo IEEE Transactions OnCybernetics vol 50 2019

[21] H Wang N Wang and D Yeung ldquoCollaborative deeplearning for recommender systemsrdquo in Proceedings of theTwenty-First ACM SIGKDD International Conference onKnowledge Discovery and Data Mining ACM New York NYUSA pp 1235ndash1244 2015

[22] X Yao B Tan C Hu W Li Z Xu and Z Zhang ldquoRec-ommend algorithm combined user-user neighborhood

approach with latent factor modelrdquo in Proceedings of theInternational Conference on Mechatronics and IntelligentRobotics Springer Cham Switzerland pp 275ndash280 2017

[23] H Zhang F Min and SWang ldquoA random forest approach tomodel-based recommendationrdquo Journal of Information andComputational Science vol 11 no 15 pp 5341ndash5348 2014

[24] G mavicius and A Tuzhilin ldquoContext-aware recommendersystemsrdquo in Proceedings of the 2008 ACM Conference onRecommender Systems RecSys 2008 Lausanne Switzerlandpp 191ndash226 October 2008

[25] R Ronen E Yomtov and G Lavee ldquoRecommendations meetweb browsing enhancing collaborative filtering using internetbrowsing logsrdquo in Proceedings of the IEEE 32nd InternationalConference on Data Engineering pp 1230ndash1238 HelsinkiFinland 2016

[26] X Yu X Ren Y Sun et al ldquoPersonalized entity recom-mendation a heterogeneous information network approachrdquoin Proceedings of the 7th ACM International Conference onWeb Search and Data Mining pp 283ndash292 New York CityNY USA 2014

[27] H Zhao Q Yao J Li Y Song and D L Lee ldquoMetagraphbased recommendation fusion over heterogeneous informa-tion networksrdquo in Proceedings of the 23rd ACM SIGKDDInternational Conference on Knowledge Discovery and DataMining pp 635ndash644Halifax Nova Scotia Canada 2017

[28] F Alqadah C K Reddy J Hu and H F AlqadahldquoBiclustering neighborhood-based collaborative filteringmethod for top-n recommender systemsrdquo Knowledge andInformation Systems vol 44 no 2 pp 475ndash491 2015

[29] Z Yao J Wang and Y Han ldquoAn improved neighborhood-based recommendation algorithm optimization with clus-tering analysis and latent factor modelrdquo in Proceedings of the38th Chinese Control Conference pp 27ndash30 GuangzhouChina 2019

[30] P Cremonesi Y Koren and R Turrin ldquoPerformance ofrecommender algorithms on top-n recommendation tasksrdquoin Proceedings of the Fourth ACM Conference on Recom-mender Systems ACM New York NY USA pp 39ndash46 2010

[31] T Guo S Pan X Zhu and C Zhang ldquoCFOND consensusfactorization for co-clustering networked datardquo IEEETransactions on Knowledge and Data Engineering vol 31no 4 pp 706ndash719 2018

[32] V Sindhwani S S Bucak J Hu and AMojsilovic ldquoOne-classmatrix completion with low-density factorizationsrdquo in Pro-ceedings of the 2010 IEEE International Conference on DataMining Ser ICDM rsquo10 IEEE Computer Society WashingtonDC USA pp 1055ndash1060 2010

[33] F Xiong W Shen H Chen et al ldquoExploiting implicit in-fluence from information propagation for social recom-mendationrdquo IEEE Transactions on Cybernetics 2019

[34] T Hofmann ldquoLatent semantic models for collaborative fil-teringrdquo ACM Transactions on Information Systems (TOIS)vol 22 no 1 pp 89ndash115 2004

[35] S Baluja R Seth D Sivakumar et al ldquoVideo suggestion anddiscovery for youtube taking random walks through the viewgraphrdquo in Proceedings of the 17th International Conference onWorld Wide Web pp 895ndash904 Beijing China 2008

[36] A Van den Oord S Dieleman and B Schrauwen ldquoDeepcontent-based music recommendationrdquo in Proceedings of theAdvances in Neural Information Processing Systemspp 2643ndash2651 Lake Tahoe NE USA 2013

[37] X Wang and Y Wang Improving Content-Based and HybridMusic Recommendation Using Deep Learning pp 627ndash636ACM Multimedia Seattle WA USA 2014

Complexity 9

[6] X Ning and G K Slim ldquoSparse linear methods for top-nrecommender systemsrdquo in Proceedings of the 2011 IEEE 11thInternational Conference on Data Mining pp 497ndash506Vancouver Canada 2011

[7] Y C Hu ldquoRecommendation using neighborhood methodswith preference-relation-based similarityrdquo Information Sci-ences vol 284 pp 18ndash30 2014

[8] C Paolo Y Koren and R Turrin ldquoPerformance of recom-mender algorithms on top-n recommendation tasksrdquo inProceedings of the Fourth ACM Conference on RecommenderSystems ACM Barcelona Spain pp 39ndash46 2010

[9] S Pan X Wang H Yang et al ldquoSocial recommendation withevolutionary opinion dynamicsrdquo IEEE Transactions on Sys-tems Man and Cybernetics Systems 2018

[10] Y Hu Y Koren and C Volinsky ldquoCollaborative filtering forimplicit feedback datasetsrdquo in Proceedings of the InternationalConference on Data Mining pp 263ndash272 Pisa Italy De-cember 2008

[11] A N Nikolakopoulos V Kalantzis E Gallopoulos andJ D Garofalakis ldquoEigenRec generalizing PureSVD for ef-fective and efficient top-N recommendationsrdquoKnowledge andInformation Systems vol 58 no 1 pp 59ndash81 2019

[12] F Christoffel B Paudel C Newell and A BernsteinldquoBlockbusters and wallflowers accurate diverse and scalablerecommendations with random walksrdquo in Proceedings of the9th ACM Conference on Recommender Systems pp 163ndash170Vienna Austria September 2015

[13] S Pan R Hu S Fung et al ldquoLearning graph embedding withadversarial training methodsrdquo IEEE Transactions on Cyber-netics vol 50 2019

[14] C Cooper S H Lee T Radzik and Y Siantos ldquoRandomwalks in recommender systems exact computation andsimulationsrdquo in Proceedings of the 23rd International Con-ference on World Wide Web ACM New York NY USApp 811ndash816 2014

[15] S Ji S Pan E Cambria P Marttinen and P S Yu ldquoA surveyon knowledge graphs representation acquisition and ap-plicationsrdquo 2020 httparxivorgabs200200388

[16] Z Kang C Peng M Yang and Q Cheng ldquoTop-N recom-mendation on graphsrdquo in Proceedings of the 25th ACM In-ternational on Conference on Information and KnowledgeManagement pp 2101ndash2106 Toronto Canada 2016

[17] A M Elkahky Y Song and X He ldquoA multi-view deeplearning approach for cross domain user modeling in rec-ommendation systemsrdquo in Proceedings of the Twenty-FourthInternational Conference on World Wide Web ACM NewYork NY USA pp 278ndash288 2015

[18] B Hidasi A Karatzoglou L Baltrunas and D Tikk ldquoSession-based recommendations with recurrent neural networksrdquo inProceedings of the International Conference on LearningRepresentations Vancouver Canada 2016

[19] A V Den Oord S Dieleman and B Schrauwen ldquoDeepcontent-based music recommendationrdquo Neural InformationProcessing Systems pp 2643ndash2651 2013

[20] R Hu G Long C Zhang et al ldquoLearning graph embeddingwith adversarial training methodsrdquo IEEE Transactions OnCybernetics vol 50 2019

[21] H Wang N Wang and D Yeung ldquoCollaborative deeplearning for recommender systemsrdquo in Proceedings of theTwenty-First ACM SIGKDD International Conference onKnowledge Discovery and Data Mining ACM New York NYUSA pp 1235ndash1244 2015

[22] X Yao B Tan C Hu W Li Z Xu and Z Zhang ldquoRec-ommend algorithm combined user-user neighborhood

approach with latent factor modelrdquo in Proceedings of theInternational Conference on Mechatronics and IntelligentRobotics Springer Cham Switzerland pp 275ndash280 2017

[23] H Zhang F Min and SWang ldquoA random forest approach tomodel-based recommendationrdquo Journal of Information andComputational Science vol 11 no 15 pp 5341ndash5348 2014

[24] G mavicius and A Tuzhilin ldquoContext-aware recommendersystemsrdquo in Proceedings of the 2008 ACM Conference onRecommender Systems RecSys 2008 Lausanne Switzerlandpp 191ndash226 October 2008

[25] R Ronen E Yomtov and G Lavee ldquoRecommendations meetweb browsing enhancing collaborative filtering using internetbrowsing logsrdquo in Proceedings of the IEEE 32nd InternationalConference on Data Engineering pp 1230ndash1238 HelsinkiFinland 2016

[26] X Yu X Ren Y Sun et al ldquoPersonalized entity recom-mendation a heterogeneous information network approachrdquoin Proceedings of the 7th ACM International Conference onWeb Search and Data Mining pp 283ndash292 New York CityNY USA 2014

[27] H Zhao Q Yao J Li Y Song and D L Lee ldquoMetagraphbased recommendation fusion over heterogeneous informa-tion networksrdquo in Proceedings of the 23rd ACM SIGKDDInternational Conference on Knowledge Discovery and DataMining pp 635ndash644Halifax Nova Scotia Canada 2017

[28] F Alqadah C K Reddy J Hu and H F AlqadahldquoBiclustering neighborhood-based collaborative filteringmethod for top-n recommender systemsrdquo Knowledge andInformation Systems vol 44 no 2 pp 475ndash491 2015

[29] Z Yao J Wang and Y Han ldquoAn improved neighborhood-based recommendation algorithm optimization with clus-tering analysis and latent factor modelrdquo in Proceedings of the38th Chinese Control Conference pp 27ndash30 GuangzhouChina 2019

[30] P Cremonesi Y Koren and R Turrin ldquoPerformance ofrecommender algorithms on top-n recommendation tasksrdquoin Proceedings of the Fourth ACM Conference on Recom-mender Systems ACM New York NY USA pp 39ndash46 2010

[31] T Guo S Pan X Zhu and C Zhang ldquoCFOND consensusfactorization for co-clustering networked datardquo IEEETransactions on Knowledge and Data Engineering vol 31no 4 pp 706ndash719 2018

[32] V Sindhwani S S Bucak J Hu and AMojsilovic ldquoOne-classmatrix completion with low-density factorizationsrdquo in Pro-ceedings of the 2010 IEEE International Conference on DataMining Ser ICDM rsquo10 IEEE Computer Society WashingtonDC USA pp 1055ndash1060 2010

[33] F Xiong W Shen H Chen et al ldquoExploiting implicit in-fluence from information propagation for social recom-mendationrdquo IEEE Transactions on Cybernetics 2019

[34] T Hofmann ldquoLatent semantic models for collaborative fil-teringrdquo ACM Transactions on Information Systems (TOIS)vol 22 no 1 pp 89ndash115 2004

[35] S Baluja R Seth D Sivakumar et al ldquoVideo suggestion anddiscovery for youtube taking random walks through the viewgraphrdquo in Proceedings of the 17th International Conference onWorld Wide Web pp 895ndash904 Beijing China 2008

[36] A Van den Oord S Dieleman and B Schrauwen ldquoDeepcontent-based music recommendationrdquo in Proceedings of theAdvances in Neural Information Processing Systemspp 2643ndash2651 Lake Tahoe NE USA 2013

[37] X Wang and Y Wang Improving Content-Based and HybridMusic Recommendation Using Deep Learning pp 627ndash636ACM Multimedia Seattle WA USA 2014

Complexity 9

[38] H J Xue X Dai J Zhang S Huang and J Chen ldquoDeepmatrix factorization models for recommender systemsrdquo inProceedings of the International Joint Conference on Ar-tificial Intelligence pp 3203ndash3209 Melbourne AustraliaAugust 2017

[39] D Kim C Park J Oh and H Yu ldquoDeep hybrid recom-mender systems via exploiting document context and sta-tistics of itemsrdquo Information Sciences vol 417 pp 72ndash872017

[40] T-A N Pham X Li G Cong and Z Zhang ldquoA generalrecommendation model for heterogeneous networksrdquo IEEETransactions on Knowledge and Data Engineering vol 28no 12 pp 3140ndash3153 2016

[41] H Chen H Yin W Wang H Wang Q V H Nguyen andX Li ldquoPME projected metric embedding on heterogeneousnetworks for link predictionrdquo in Proceedings of the 24th ACMSIGKDD International Conference on Knowledge Discovery ampData Mining pp 1177ndash1186 London UK August 2018

[42] X Yu X Ren Y Sun et al ldquoRecommendation in hetero-geneous information networks with implicit user feedbackrdquoin Proceedings of the 7th ACM Conference on RecommenderSystems pp 347ndash350 Hong Kong China 2013 October

[43] Z Jiang H Liu B Fu Z Wu and T Zhang ldquoRecommen-dation in heterogeneous information networks based ongeneralized random walk model and bayesian personalizedrankingrdquo in Proceedings of the Eleventh ACM InternationalConference onWeb Search and Data Mining pp 288ndash296 LosAngeles CA USA October 2018

[44] B Hu C Shi W Zhao and P Yu ldquoLeveraging meta-pathbased context for top- N recommendation with a neural co-attention modelrdquo Knowledge Discovery and Data Miningpp 1531ndash1540 2018

[45] Y Sun J Han X Yan S P Yu and T Wu ldquoPathSim metapath-based top-K similarity search in heterogeneous infor-mation networksrdquo in Proceedings of the 37th InternationalConference on Very Large Data Base Seattle WashingtonSeptember 2011

[46] X He L Liao H Zhang L Nie X Hu and T Chua ldquoNeuralcollaborative filteringrdquo in Proceedings of the 26th Interna-tional Conference on World Wide Web pp 173ndash182 PerthAustralia April 2017

[47] C Desrosiers and G Karypis ldquoA comprehensive survey ofneighborhood-based recommendation methodsrdquo in Recom-mender Systems and Book pp 107ndash144 Springer Boston MAUSA 2011

[48] W Khatib and P J Fleming ldquoe stud ga a mini revolutionrdquoin Proceedings of the 5th International Conference on ParallelProblem Solving from Nature Springer New York NY USApp 683ndash691 1998

[49] G E Hinton and R Salakhutdinov ldquoA better way to pretraindeep Boltzmann machinesrdquo in Proceedings of the Advances inNeural Information Processing Systems pp 2447ndash2455 LakeTahoe NE USA 2012

[50] Y Koren ldquoFactor in the neighborsrdquo ACsM Transactions onKnowledge Discovery from Data vol 4 no 1 pp 1ndash24 2010

[51] G Adomavicius and A Tuzhilin ldquoContext-aware recom-mender systemsrdquo in Proceedings of the 2008 ACM Conferenceon Recommender Systems pp 23ndash25 Lausanne SwitzerlandOctober 2008

[52] D Kingma and J Ba Adam ldquoA method for stochastic op-timizationrdquo in Proceedings of the 3rd International Conferenceon Learning Representations San Diego CA USA 2015

[53] S Rendle C Freudenthaler Z Gantner and L Schmidt-ieme ldquoBPR bayesian personalized ranking from implicitfeedbackrdquo 2012 httparxivorgabs12052618

[54] Y Tay L Anh Tuan and S C Hui ldquoLatent relational metriclearning via memory-based attention for collaborativerankingrdquo in Proceedings of the 2018 World Wide Web Con-ference pp 729ndash739 Lyon France April 2018

[55] Y Wu C DuBois A X Zheng and M Ester ldquoCollaborativedenoising auto-encoders for top-n recommender systemsrdquo inProceedings of the Ninth ACM International Conference onWeb Search and Data Mining pp 153ndash162 San FranciscoCA USA 2016

10 Complexity