Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 1295–1309,November 16–20, 2020. c©2020 Association for Computational Linguistics

1295

Scalable Multi-Hop Relational Reasoning forKnowledge-Aware Question Answering

Yanlin Feng∫ò Xinyue Chenπò Bill Yuchen Lin∏ Peifeng Wang∏ Jun Yan∏ Xiang Ren∏

fengyanlin@pku.edu.cn, xinyuech@andrew.cmu.edu,{yuchen.lin, peifengw, yanjun, xiangren}@usc.edu

∏University of Southern California∫Peking University πCarnegie Mellon University

AbstractExisting work that augment question answer-ing (QA) models with external knowledge(e.g., knowledge graphs) either struggle tomodel multi-hop relations efficiently, or lacktransparency into the model’s prediction ra-tionale. In this paper, we propose a novelknowledge-aware approach that equips pre-trained language models (PTLMs) with amulti-hop relational reasoning module, namedmulti-hop graph relation network (MHGRN).It performs multi-hop, multi-relational reason-ing over subgraphs extracted from externalknowledge graphs. The proposed reasoningmodule unifies path-based reasoning methodsand graph neural networks and results in betterinterpretability and scalability. We also empir-ically show its effectiveness and scalability onCommonsenseQA and OpenbookQA datasets,and interpret its behaviors with case studies,with the code for experiments released1.

1 IntroductionMany recently proposed question answering tasksrequire not only machine comprehension of thequestion and context, but also relational reason-ing over entities (concepts) and their relationshipsby referencing external knowledge (Talmor et al.,2019; Sap et al., 2019; Clark et al., 2018; Mihaylovet al., 2018). For example, the question in Fig. 1requires a model to perform relational reasoningover mentioned entities, i.e., to infer latent rela-tions among the concepts: {CHILD, SIT, DESK,SCHOOLROOM}. Background knowledge such as

“a child is likely to appear in a schoolroom” may notbe readily contained in the questions themselves,but are commonsensical to humans.

Despite the success of large-scale pre-trainedlanguage models (PTLMs) (Devlin et al., 2019;

ò The first two authors contributed equally. The majorwork was done when both authors interned at USC.

1https://github.com/INK-USC/MHGRN

Desires

Learn

Child

Schoolroom

Classroom

Sit

Desk

AtLocation

Where does a child likely sit at a desk?

A. Schoolroom * B. Furniture store C. PatioD. Office building E. Library

Figure 1: Illustration of knowledge-aware QA. Asample question from CommonsenseQA can be betteranswered if a relevant subgraph of ConceptNet is pro-vided as evidence. Blue nodes correspond to entitiesmentioned in the question, and pink nodes correspondto those in the answer. The other nodes are associatedentities introduced in subgraph extraction. ì indicatesthe correct answer.

Liu et al., 2019b), these models fall short of pro-viding interpretable predictions, as the knowledgein their pre-training corpus is not explicitly stated,but rather is implicitly learned. It is thus difficult torecover the evidence used in the reasoning process.

This has led many to leverage knowledgegraphs (KGs) (Mihaylov and Frank, 2018; Linet al., 2019; Wang et al., 2019; Yang et al., 2019).KGs represent relational knowledge between en-tities with multi-relational edges for models toacquire. Incorporating KGs brings the potentialof interpretable and trustworthy predictions, asthe knowledge is now explicitly stated. For ex-ample, in Fig. 1, the relational path (CHILD �AtLocation � CLASSROOM � Synonym �SCHOOLROOM) naturally provides evidence for

1296

Figure 2: Number of K-hop relational paths w.r.t.the node count in extracted graphs on Common-senseQA. Left: The path count is polynomial w.r.t. thenumber of nodes. Right: The path count is exponentialw.r.t. the number of hops.

the answer SCHOOLROOM.A straightforward approach to leveraging a

knowledge graph is to directly model these rela-tional paths. KagNet (Lin et al., 2019) and MH-PGM (Bauer et al., 2018) extract multi-hop rela-tional paths from KG and encode them with se-quence models. Application of attention mecha-nisms upon these relational paths can further offergood interpretability. However, these models arehardly scalable because the number of possiblepaths in a graph is (1) polynomial w.r.t. the num-ber of nodes (2) exponential w.r.t. the path length(see Fig. 2). Therefore, some (Weissenborn et al.,2017; Mihaylov and Frank, 2018) resort to onlyusing one-hop paths, namely, triples, to balancescalability and reasoning capacities.

Graph neural networks (GNNs), in contrast, en-joy better scalability via their message passingformulation, but usually lack transparency. Themost commonly used GNN variant, Graph Con-volutional Networks (GCNs) (Kipf and Welling,2017), perform message passing by aggregatingneighborhood information for each node, but ig-nore the relation types. RGCNs (Schlichtkrull et al.,2018) generalize GCNs by performing relation-specific aggregation, making it applicable to multi-relational graphs. However, these models do notdistinguish the importance of different neighborsor relation types and thus cannot provide explicitrelational paths for model behavior interpretation.

In this paper, we propose a novel graph encod-ing architecture, Multi-hop Graph Relation Net-work (MHGRN), which combines the strengths ofpath-based models and GNNs. Our model inheritsscalability from GNNs by preserving the messagepassing formulation. It also enjoys interpretabilityof path-based models by incorporating structured

GCN RGCN KagNet MHGRN

Multi-Relational Encoding 7 3 3 3Interpretable 7 7 3 3

Scalable w.r.t. #node 3 3 7 3Scalable w.r.t. #hop 3 3 7 3

Table 1: Properties of our MHGRN and other repre-sentative models for graph encoding.

relational attention mechanism. Towards multi-hoprelational reasoning, our key motivation is to al-low each node to directly attend to its multi-hopneighbors by performing multi-hop message pass-ing within a single layer. We outline the desiredmerits of knowledge-aware QA models in Table 1and compare MHGRN with them.

We summarize the main contributions of thiswork as follows: 1) We propose MHGRN, a novelmodel architecture tailored to multi-hop relationalreasoning, which explicitly models multi-hop rela-tional paths at scale. 2) We propose a structuredrelational attention mechanism for efficient and in-terpretable modeling of multi-hop reasoning paths,along with its training and inference algorithms. 3)We conduct extensive experiments on two questionanswering datasets and show that our models bringsignificant improvements compared to knowledge-agnostic PTLMs, and outperform other graph en-coding methods by a large margin.

2 Problem Formulation and Overview

In this paper, we limit the scope to the task ofmultiple-choice question answering, although theformulation can be easily generalized to otherknowledge-guided tasks (e.g., natural languageinference). The overall paradigm of knowledge-aware QA is illustrated in Fig. 3. Formally, given aquestion q and an external knowledge graph (KG)as the knowledge source, our goal is to identify thecorrect answer from a set C of given options. Weturn this problem into measuring the plausibilityscore between q and each option a " C, after whichwe select the option with the highest plausibilityscore.

To measure the score for q and a, we first con-catenate q and a to form a statement s = [q; a] andencode the statement s into the statement represen-tation s. Then we extract from the external KGa subgraph G (i.e., schema graph in KagNet (Linet al., 2019)), with the guidance of s (detailed in§5.1). This contextualized subgraph is defined asa multi-relational graph G = (V, E ,�). Here V

1297

PlausibilityScore

Graph ExtractionFrom KG

Graph Encoder

Text Encoder

𝓖

𝒔𝒒

𝒂

Figure 3: Overview of the knowledge-aware QAframework. It integrates the output from graph en-coder (for relational reasoning over contextual sub-graphs) and text encoder (for textual understanding) togenerate the plausibility score for an answer option.

is a subset of entities in the external KG, contain-ing only those relevant to s. E N V ✓ R ✓ V isthe set of edges that connect nodes in V , whereR = {1,⇧,m} are the ids of all pre-defined re-lation types. The mapping function �(i) ⇥ V �T = {Eq,Ea,Eo} takes node i " V as input, andoutputs Eq if i is an entity mentioned in q, Ea ifit is mentioned in a, and Eo otherwise2. Finally,we encode G into g, and concatenate s and g tocalculate the plausibility score.

3 Background: Multi-Relational GraphEncoding Methods

We leave encoding of s to pre-trained languagemodels and focus on the challenge of encodinggraph G to capture latent relations between enti-ties. Current methods for encoding multi-relationalgraphs mainly fall into two categories: GNNs andpath-based models. GNNs encode structured in-formation by passing messages between nodes, di-rectly operating on the graph structure, while path-based methods first decompose the graph into pathsand then pool their representations to form a graphrepresentation.

Graph Encoding with GNNs. For a graph withn nodes, a graph neural network (GNN) takes aset of node features {h1,h2, . . . ,hn} as input, andcomputes their corresponding node embeddings{h¨

1,h¨2, . . . ,h

¨n} via message passing (Gilmer

et al., 2017). A compact graph representation forG can thus be obtained by pooling the node embed-dings {h¨

i}:

GNN(G) = Pool({h¨1,h

¨2, . . . ,h

¨n}). (1)

2It is plausible to accordingly re-design mapping functionsfor this paradigm to work in other NLP tasks.

As a notable variant of GNNs, graph convo-lutional networks (GCNs) (Kipf and Welling,2017) additionally update node embeddings byaggregating messages from its direct neighbors.RGCNs (Schlichtkrull et al., 2018) extend GCNs toencode multi-relational graphs by defining relation-specific weight matrix Wr for each edge type:

h¨i = �

�⇣�⇧=r"R ∂N ri ∂↵

�1

=r"R

=j"N r

i

Wrhj

�⌘✏ , (2)

where Nri denotes neighbors of node i under rela-

tion r.3

While GNNs have proved to have good scalabil-ity, their reasoning is done at the node level, and aretherefore incompatible with path modeling. Thisproperty also hinders path-level interpretation ofthe model’s decisions.

Graph Encoding with Path-Based Models. Inaddition to directly modeling the graph with GNNs,a graph can also be viewed as a set of relationalpaths connecting pairs of entities.

Relation Networks (RNs) (Santoro et al., 2017)can be adapted to multi-relational graph encodingunder QA settings. RNs use MLPs to encode alltriples (one-hop paths) in G whose head entity isin Q = {j ∂ �(j) = Eq} and tail entity is inA = {i ∂ �(i) = Ea}. It then pools the tripleembeddings to generate a vector for G as follows,

RN(G) = Pool⇤{MLP(hj h erh

hi) ∂ j " Q, i " A, (j, r, i) " E} . (3)

Here hj and hi are features for nodes j and i, er isthe embedding of relation r " R, h denotes vectorconcatenation.

To further equip RN with the ability to modelnondegenerate paths, KagNet (Lin et al., 2019)adopts LSTMs to encode all paths connecting ques-tion entities and answer entities with lengths nomore than K. It then aggregates all path embed-dings via the attention mechanism:

KAGNET(G) = Pool⇤{LSTM(j, r1, j1, . . . , rk, i) ∂(j, r1, j1),⇧, (jk�1, rk, i) " E , 1 & k & K} .

(4)3For simplicity, we assume a single graph convolutional

layer. In practice, multiple layers are stacked to enable mes-sage passing from multi-hop neighbors.

1298

AttentivePooling

2. Multi-hop Message Passing

Node Features

Text Encodere.g. BERT

1. Node Type Specific Transform

3. Nonlinear Activation

×#Layer

PlausibilityScore

MLP

퓖 𝒔

AggregateMessages

𝜶(𝒋, 𝒓ퟏ, … , 𝒓𝒌, 𝒊)

Structured Relational Att.

Learn

Child

Child CapableOf UsedFor AtLocation

Desires UsedFor Synonym

AtLocation Synonym

Sit AtLocationUsedFor

Desk AtLocation

…Multi-hop Message Passing

Figure 4: Our proposed MHGRN architecturefor relational reasoning. MHGRN takes a multi-relational graph G and a (question-answer) statementvector s as input, and outputs a scalar that represent theplausibility score of this statement.

4 Proposed Method: Multi-Hop GraphRelation Network (MHGRN)

This section presents Multi-hop Graph RelationNetwork (MHGRN), a novel GNN architecturethat unifies both GNNs and path-based models.MHGRN inherits path-level reasoning and inter-pretabilty from path-based models, while preserv-ing good scalability of GNNs.

4.1 MHGRN: Model Architecture

We follow the GNN framework introduced in §3,where node features can be initialized with pre-trained weights (details in Appendix B). Here wefocus on the computation of node embeddings.

Type-Specific Transformation. To make ourmodel aware of the node type �, we first performnode type specific linear transformation on the in-put node features:

xi = U�(i)hi + b�(i), (5)

where the learnable parameters U and b are specificto the type of node i.

Multi-Hop Message Passing. As mentioned be-fore, our motivation is to endow GNNs with thecapability of directly modeling paths. To this end,we propose to pass messages directly over all the

relational paths of lengths up to K. The set of validk-hop relational paths is defined as:

�k = {(j, r1, . . . , rk, i) ∂ (j, r1, j1),⇧, (jk�1, rk, i) " E} (1 & k & K). (6)

We perform k-hop (1 & k & K) message passingover these paths, which is a generalization of thesingle-hop message passing in RGCNs (see Eq. 2):

zki = =

(j,r1,...,rk,i)"�k

↵(j, r1, . . . , rk, i)/dki �WK0

⇧Wk+10 W

krk⇧W

1r1xj (1 & k & K), (7)

where the Wtr (1 & t & K, 0 & r & m) matrices

are learnable4, ↵(j, r1, . . . , rk, i) is an attentionscore elaborated in §4.2 and dki = <(j⇧i)"�k

↵(j⇧i)is the normalization factor. The {W k

rk⇧W1r1 ∂ 1 &

r1, . . . , rk & m} matrices can be interpreted as thelow rank approximation of a {m✓⇧✓m}k✓d✓dtensor that assigns a separate transformation foreach k-hop relation, where d is the dimension ofxi.

Incoming messages from paths of differentlengths are aggregated via attention mecha-nism (Vaswani et al., 2017):

zi =K

=k=1

softmax�bilinear�s, zki ⌥⌥ � zk

i . (8)

Non-linear Activation. Finally, we apply shortcutconnection and nonlinear activation to obtain theoutput node embeddings.

h¨i = � ⇥V hi + V

¨zi� , (9)

where V and V¨ are learnable model parameters,

and � is a non-linear activation function.

4.2 Structured Relational AttentionNaive parameterization of the attention score↵(j, r1, . . . , rk, i) in Eq. 7 would require O(mk) param-eters for k-hop paths. Towards efficiency, we firstregard it as the probability of a relation sequence(�(j), r1, . . . , rk,�(i)) conditioned on s:

↵(j, r1, . . . , rk, i) = p (�(j), r1, . . . , rk,�(i) ∂ s) ,(10)

4W

t0(0 & t & K) are introduced as padding matrices

so that K transformations are applied regardless of k, thusensuring comparable scale of zk

i across different k.

1299

which can naturally be modeled by a probabilisticgraphical model, such as conditional random field(Lafferty et al., 2001):

p (⇧ ∂ s) ö exp⇧f(�(j), s) + k

=t=1

�(rt, s)+

k�1

=t=1

⌧(rt, rt+1) + g(�(i), s)↵�= �(r1, . . . , rk, s)Õ “““““““““““““““““““““““““““““““““““““““““““““—“““““““““““““““““““““““““““““““““““““““““““““œ

Relation Type Attention

� �(�(j),�(i), s)Õ ““““““““““““““““““““““““““““““““““““““““““““““—““““““““““““““““““““““““““““““““““““““““““““““œNode Type Attention

, (11)

where f(�), �(�) and g(�) are parameterized bytwo-layer MLPs and ⌧(�) by a transition matrixof shape m ✓m. Intuitively, �(�) models the im-portance of a k-hop relation while �(�) models theimportance of messages from node type �(j) to�(i) (e.g., the model can learn to pass messagesonly from question entities to answer entities).

Our model scores a k-hop relation by decom-posing it into both context-aware single-hop re-lations (modeled by �) and two-hop relations(modeled by ⌧ ). We argue that ⌧ is indis-pensable, without which the model may assignhigh importance to illogical multi-hop relations(e.g., [AtLocation, CapableOf]) or noisy re-lations (e.g., [RelatedTo, RelatedTo]).

4.3 Computation Complexity AnalysisAlthough the message passing process in Eq. 7 andthe attention module in Eq.11 handle potentiallyexponential number of paths, computation can bedone in linear time with the help of dynamic pro-gramming (see Appendix C). As summarized inTable 2, both the time complexity and space com-plexity of MHGRN on a sparse graph are linearw.r.t. the maximum path length K or the numberof nodes n.

4.4 Expressive Power of MHGRNIn addition to efficiency and scalability, we now dis-cuss the modeling capacity of MHGRN. With themessage passing formulation and relation-specifictransformations, MHGRN is by nature the gener-alization of RGCN. It is also capable of directlymodeling paths, making it interpretable as are path-based models such as RN and KagNet. To showthis, we first generalize RN (Eq. 3) to the multi-hopsetting and introduce K-hop RN (formal definitionin Appendix D), which models multi-hop relationas the composition of single-hop relations. We

Model Time Space

G is a dense graph

K-hop KagNet O ⇥mKnK+1K� O ⇥mKnK+1K�K-layer RGCN O ⇥mn2K� O (mnK)MHGRN O ⇥m2n2K� O (mnK)G is a sparse graph with maximum node degree � 8 n

K-hop KagNet O ⇥mKnK�K� O ⇥mKnK�K�K-layer RGCN O (mnK�) O (mnK)MHGRN O ⇥m2nK�� O (mnK)

Table 2: Computation complexity of different K-hopreasoning models on a dense/sparse multi-relationalgraph with n nodes and m relation types. Despite thequadratic complexity w.r.t. m, MHGRN’s time cost issimilar to RGCN on GPUs with parallelizable matrixmultiplications (cf. Fig. 7).

can show that MHGRN is capable of representingK-hop RN (proof in Appendix E).

4.5 Learning, Inference and Path DecodingWe now discuss the learning and inference processof MHGRN instantiated for QA tasks. Followingthe problem formulation in §2, we aim to deter-mine the plausibility of an answer option a " C

given the question q with the information fromboth text s and graph G. We first obtain the graphrepresentation g by performing attentive poolingover the output node embeddings of answer entities{h¨

i ∂ i " A}. Next we concatenate it with the textrepresentation s and compute the plausibility scoreby ⇢(q, a) = MLP(sh g).

During training, we maximize the plausibilityscore of the correct answer a by minimizing thecross-entropy loss:

L = Eq,a,C �� logexp(⇢(q, a))

<a"C exp(⇢(q, a))⌧ . (12)

The whole model is trained end-to-end jointly withthe text encoder (e.g., RoBERTa).

During inference, we predict the most plau-sible answer by argmaxa"C ⇢(q, a). Addition-ally, we can decode a reasoning path as evidencefor model predictions, endowing our model withthe interpretability enjoyed by path-based mod-els. Specifically, we first determine the answerentity iò with the highest score in the pooling layerand the path length kò with the highest score inEq. 8. Then the reasoning path is decoded byargmax ↵(j, r1, . . . , rkò , iò), which can be com-puted in linear time using dynamic programming.

1300

Methods BERT-Base BERT-Large RoBERTa-Large

IHdev-Acc.(%) IHtest-Acc.(%) IHdev-Acc.(%) IHtest-Acc.(%) IHdev-Acc.(%) IHtest-Acc.(%)

w/o KG 57.31 (±1.07) 53.47 (±0.87) 61.06 (±0.85) 55.39 (±0.40) 73.07 (±0.45) 68.69(±0.56)

RGCN (Schlichtkrull et al., 2018) 56.94 (±0.38) 54.50 (±0.56) 62.98 (±0.82) 57.13 (±0.36) 72.69 (±0.19) 68.41 (±0.66)GconAttn (Wang et al., 2019) 57.27 (±0.70) 54.84 (±0.88) 63.17 (±0.18) 57.36 (±0.90) 72.61( ±0.39) 68.59 (±0.96)KagNet† (Lin et al., 2019) 55.57 56.19 62.35 57.16 - -RN (1-hop) 58.27 (±0.22) 56.20 (±0.45) 63.04 (±0.58) 58.46 (±0.71) 74.57 (±0.91) 69.08 (±0.21)RN (2-hop) 59.81 (±0.76) 56.61 (±0.68) 63.36 (±0.26) 58.92 (±0.14) 73.65 (±3.09) 69.59 (±3.80)

MHGRN 60.36 (±0.23) 57.23 (±0.82) 63.29(±0.51) 60.59 (±0.58) 74.45 (±0.10) 71.11 (±0.81)

Table 3: Performance comparison on CommonsenseQA in-house split. We report in-house Dev (IHdev) andTest (IHtest) accuracy (mean and standard deviation of four runs) using the data split of Lin et al. (2019) onCommonsenseQA. † indicates reported results in its paper.

5 Experimental Setup

We introduce how we construct G (§5.1), thedatasets (§5.2), as well as the baseline methods(§5.3). Appendix B shows more implementationand experimental details for reproducibility.

5.1 Extracting G from External KGWe use ConceptNet (Speer et al., 2017), a general-domain knowledge graph as our external KG to testmodels’ ability to harness structured knowledgesource. Following KagNet (Lin et al., 2019), wemerge relation types to increase graph density andadd reverse relations to construct a multi-relationalgraph with 34 relation types (details in AppendixA). To extract an informative contextualized graphG from KG, we recognize entity mentions in s andlink them to entities in ConceptNet, with which weinitialize our node set V . We then add to V all theentities that appear in any two-hop paths betweenpairs of mentioned entities. Unlike KagNet, we donot perform any pruning but instead reserve all theedges between nodes in V , forming our G.

5.2 DatasetsWe evaluate models on two multiple-choice ques-tion answering datasets, CommonsenseQA andOpenbookQA. Both require world knowledge be-yond textual understanding to perform well.

CommonsenseQA (Talmor et al., 2019) neces-sitates various commonsense reasoning skills. Thequestions are created with entities from Concept-Net. It is noteworthy that although Common-senseQA is built upon ConceptNet, its questionsare designed to probe multi-hop/compositional re-lations between entities that cannot be directly readfrom, or are even absent from ConceptNet. This de-

5Models based on ConceptNet are no longer shown on theleaderboard, and we got our results from the organizers.

Methods Single Ensemble

UnifiedQA† (Khashabi et al., 2020) 79.1 -

RoBERTa† 72.1 72.5RoBERTa + KEDGN† 72.5 74.4RoBERTa + KE† 73.3 -RoBERTa + HyKAS 2.0† (Ma et al., 2019) 73.2 -RoBERTa + FreeLB†(Zhu et al., 2020) 72.2 73.1XLNet + DREAM† 66.9 73.3XLNet + GR† (Lv et al., 2019) 75.3 -ALBERT† (Lan et al., 2019) - 76.5

RoBERTa + MHGRN (K = 2) 75.4 76.5

Table 4: Performance comparison on official test ofCommonsenseQA with leaderboard SoTAs5 (accuracyin %). † indicates reported results on leaderboard. Uni-fiedQA uses T5-11B as text encoder, whose number ofparameters is about 30 times more than other models.

Train Dev Test

CommonsenseQA (OF) 9, 741 1, 221 1, 140CommonsenseQA (IH) 8, 500 1, 221 1, 241OpenbookQA 4, 957 500 500

Table 5: Numbers of instances in different datasetsplits.

sign prevents short-cutting the reasoning by takingadvantage of the graph structure.

OpenBookQA (Mihaylov et al., 2018) provideselementary science questions together with an openbook of science facts. This dataset also probesgeneral common sense beyond the provided facts.

Dataset split specifications. Common-senseQA 6 and OpenbookQA 7 all have their leader-boards, with training and development set publiclyavailable. As the ground truth labels for Common-senseQA are not readily available, for model anal-

6https://www.tau-nlp.org/commonsenseqa7https://leaderboard.allenai.org/open_

book_qa/submissions/public

1301

ysis, we follow Lin et al. (2019) and take 1,241examples from official training examples as ourin-house test examples and regard the remaining8,500 ones as our in-house training examples, form-ing CommonsenseQA (IH). We also make use ofthe official split of CommonsenseQA, denoted asCommonsenseQA (OF). The numbers of instancesin different dataset splits are listed in Table 5.

5.3 Compared MethodsWe implement both knowledge-agnostic fine-tuning of pre-trained LMs and models that incorpo-rate external KG as our baselines. Additionally, wedirectly compare our model with the results fromcorresponding leaderboard. These methods typi-cally leverage textual knowledge or extra trainingdata, as opposed to external KG. In all our imple-mented models, we use pre-trained LMs as textencoders for s for fair comparison. We do compareour models with those (Ma et al., 2019; Lv et al.,2019; Khashabi et al., 2020) augmented by othertext-form external knowledge (e.g., Wikipedia), al-though we stick to our focus of encoding structuredKG.

Specifically, we fine-tune BERT-BASE, BERT-LARGE (Devlin et al., 2019), and ROBERTA (Liuet al., 2019b) for multiple-choice questions. Wetake RGCN (Eq. 2 in §3), RN8 (Eq. 3 in §3),KagNet (Eq. 4 in §3) and GconAttn (Wanget al., 2019) as baselines. GconAttn generalizesmatch-LSTM (Wang and Jiang, 2016) and achievessuccess in language inference tasks.

6 Results and Discussions

In this section, we present the results of our modelsin comparison with baselines as well as methodson the leaderboards for both CommonsenseQA andOpenbookQA. We also provide analysis of models’components and characteristics.

6.1 Main ResultsFor CommonsenseQA (Table 3), we first use thein-house data split (IH) (see §5.2) to compare ourmodels with implemented baselines. This is dif-ferent from the official split used in the leader-board methods. Almost all KG-augmented modelsachieve performance gain over vanilla pre-trainedLMs, demonstrating the value of external knowl-edge on this dataset. Additionally, we evaluate

8We use mean pooling for 1-hop RN and attentive poolingfor 2-hop RN (detailed in Appendix D).

Methods Dev (%) Test (%)

T5-3B† (Raffel et al., 2019) - 83.20UnifiedQA† (Khashabi et al., 2020) - 87.20

RoBERTa-Large (w/o KG) 66.76 (±1.14) 64.80 (±2.37)+ RGCN 64.65 (±1.96) 62.45 (±1.57)+ GconAttn 64.30 (±0.99) 61.90 (±2.44)+ RN (1-hop) 64.85 (±1.11) 63.65 (±2.31)+ RN (2-hop) 67.00 (±0.71) 65.20 (±1.18)+ MHGRN (K = 3) 68.10 (±1.02) 66.85 (±1.19)

AristoRoBERTaV7† 79.2 77.8+ MHGRN (K = 3) 78.6 80.6

Table 6: Performance comparison on OpenbookQA.† indicates reported results on leaderboard. T5-3B is 8times larger than our models. UnifiedQA is 30x larger.

our MHGRN (with text encoder being ROBERTA-LARGE) on official split, OF (Table 4) for faircomparison with other methods on leaderboard,in both single-model setting and ensemble-modelsetting. With backbone being T5 (Raffel et al.,2019), UnifiedQA (Khashabi et al., 2020) tops theleaderboard9. Considering its training cost, we donot intend to compare our ROBERTA-based modelwith it. We achieve the best performances amongthe other models.

For OpenbookQA (Table 6), we use official splitand build models with ROBERTA-LARGE as textencoder. MHGRN surpasses all implemented base-lines, with an absolute increase of ⇥2% on Test.Also, as our approach is naturally compatible withthe methods that utilize textual knowledge or ex-tra data, because in our paradigm the encodingof textual statement and graph are structurally-decoupled (Fig. 3). To empirically show MHGRNcan bring gain over textual-knowledge empow-ered systems, we replace our text encoder withAristoRoBERTaV710, and fine-tune our MHGRNupon OpenbookQA. Empirically, MHGRN contin-ues to bring benefits to strong-performing textual-knowledge empowered systems. One takeaway isthat textual knowledge and structured knowledgeare potentially complementary.

6.2 Performance AnalysisAblation Study on Model Components. Asshown in Table 7, disabling type-specific trans-formation results in ⇥ 1.3% drop in performance,demonstrating the need for distinguishing nodetypes for QA tasks. Our structured relational at-tention mechanism is also critical, with its two

9The leaderboard in August, 2020.10https://leaderboard.allenai.org/open_

book_qa/submission/blcp1tu91i4gm0vf484g

1302

Methods IHdev-Acc. (%)

MHGRN (K = 3) 74.45 (±0.10)- Type-specific transformation (§4.1) 73.16 (±0.28)- Structured relational attention (§4.2) 73.26 (±0.31)- Relation type attention (§4.2) 73.55 (±0.68)- Node type attention (§4.2) 73.92 (±0.65)

Table 7: Ablation study on model components (re-moving one component each time) using ROBERTA-LARGE as the text encoder. We report the IHdev ac-curacy on CommonsenseQA.

sub-components contributing almost equally.Impact of the Amount of Training Data. Weuse different fractions of training data of Common-senseQA and report results of fine-tuning text en-coders alone and jointly training text encoder andgraph encoder in Fig. 5. Regardless of training datafraction, our model shows consistently more per-formance improvement over knowledge-agnosticfine-tuning compared with the other graph encod-ing methods, indicating MHGRN’s complementarystrengths to text encoders.

Figure 5: Performance change (accuracy in %) w.r.t.the amounts of training data on CommonsenseQA IHT-est set (same as Lin et al. (2019)).

Impact of Number of Hops (K). We investigatethe impact of hyperparameter K for MHGRN withexperiments on CommonsenseQA (Fig. 6). Theincrease of K continues to bring benefits until K =4. However, performance begins to drop whenK > 3. This might be attributed to exponentialnoise in longer relational paths in the knowledgegraph.

6.3 Model ScalabilityFig. 7 presents the computation cost of MHGRNand RGCN (measured by training time) with thetext encoder removed. Both grow linearly w.r.t. K.Although the theoretical complexity of MHGRN

Hops

73.24

73.93

74.65

73.65

74.16

Figure 6: Effect of K in MHGRN. We show IHDev ac-curacy of MHGRN on CommonsenseQA w.r.t. # hops.

: # Reasoning Hops

RGCNMHGRN

Figure 7: Analysis of model scalability. Comparisonof per-batch training efficiency w.r.t. # hops K.

is m times that of RGCN, the ratio of their empiri-cal cost only approaches 2, demonstrating that ourmodel can be better parallelized. It is noteworthythat the text encoder part actually dominates theoverall cost. Therefore, the gap between our modeland the RGCN are further narrowed if we considerthe cost of the entire model.

6.4 Model Interpretability

We can analyze our model’s reasoning process bydecoding the reasoning path using the method de-scribed in §4.5. Fig. 8 shows two examples fromCommonsenseQA, where our model correctly an-swers the questions and provides reasonable pathevidences. In the example on the left, the modellinks question entities and answer entity in a chainto support reasoning, while the example on the rightprovides a case where our model leverage unmen-tioned entities to bridge the reasoning gap betweenquestion entity and answer entities, in a way that iscoherent with the latent relation between CHAPELand the desired answer in the question.

1303

Where is known for a multitude of wedding chapels?

A. town B. texasC. city D. church building E. Nevada*

RenoChapel

PartOf

AtLocation

UsedForଵ Synonym

AtLocation Synonym

Neveda

Baseball

Play_Baseball

HasProperty

UsedForଵ

Fun_to_Play

Why do parents encourage their kids to play baseball?

A. round B. cheap C. break window D. hardE. fun to play* Fun_to_Play

HasProperty

Reno

Chapel

AtLocation

Neveda

PartOf

MultiGRNFigure 8: Case study on model interpretability. Wepresent two sample questions from CommonsenseQAwith the reasoning paths output by MHGRN.

7 Related Work

Knowledge-Aware Methods for NLP Variouswork have investigated the potential to empowerNLP models with external knowledge. Many at-tempt to extract structured knowledge, either in theform of nodes (Yang and Mitchell, 2017; Wanget al., 2019), triples (Weissenborn et al., 2017;Mihaylov and Frank, 2018), paths (Bauer et al.,2018; Kundu et al., 2019; Lin et al., 2019), or sub-graphs (Li and Clark, 2015), and encode them toaugment textual understanding.

Recent success of pre-trained LMs motivatesmany (Pan et al., 2019; Ye et al., 2019; Zhang et al.,2018; Li et al., 2019; Banerjee et al., 2019) to probeLMs’ potential as latent knowledge bases. This lineof work turn to textual knowledge (e.g. Wikipedia)to directly impart knowledge to pre-trained LMs.They generally fall into two paradigms: 1) Fine-tuning LMs on large-scale general-domain datasets(e.g. RACE (Lai et al., 2017)) or on knowledge-richtext. 2) Providing LMs with evidence via informa-tion retrieval techniques. However, these modelscannot provide explicit reasoning and evidence,thus hardly trustworthy. They are also subject tothe availability of in-domain datasets and maxi-mum input token of pre-trained LMs.

Neural Graph Encoding Graph Attention Net-works (GAT) (Velickovic et al., 2018) incorpo-rates attention mechanism in feature aggregation,RGCN (Schlichtkrull et al., 2018) proposes rela-tional message passing which makes it applicableto multi-relational graphs. However they only per-form single-hop message passing and cannot beinterpreted at path level. Other work (Abu-El-Haijaet al., 2019; Nikolentzos et al., 2019) aggregate fora node its K-hop neighbors based on node-wisedistances, but they are designed for non-relationalgraphs. MHGRN addresses these issues by rea-soning on multi-relational graphs and being inter-pretable via maintaining paths as reasoning chains.

8 Conclusion

We present a principled, scalable method, MHGRN,that can leverage general knowledge via multi-hopreasoning over interpretable structures (e.g. Con-ceptNet). The proposed MHGRN generalizes andcombines the advantages of GNNs and path-basedreasoning models. It explicitly performs multi-hoprelational reasoning and is empirically shown tooutperform existing methods with superior scal-ablility and interpretability.

Acknowledgements

This research is based upon work supported in partby the Office of the Director of National Intelli-gence (ODNI), Intelligence Advanced ResearchProjects Activity (IARPA), via Contract No. 2019-19051600007, the DARPA MCS program underContract No. N660011924033 with the UnitedStates Office Of Naval Research, the DefenseAdvanced Research Projects Agency with awardW911NF-19-20271, and NSF SMA 18-29268. Theviews and conclusions contained herein are thoseof the authors and should not be interpreted asnecessarily representing the official policies, eitherexpressed or implied, of ODNI, IARPA, or theU.S. Government. We would like to thank all thecollaborators in USC INK research lab for theirconstructive feedback on the work.

ReferencesSami Abu-El-Haija, Bryan Perozzi, Amol Kapoor,

Nazanin Alipourfard, Kristina Lerman, Hrayr Haru-tyunyan, Greg Ver Steeg, and Aram Galstyan. 2019.Mixhop: Higher-order graph convolutional architec-tures via sparsified neighborhood mixing. In Pro-ceedings of the 36th International Conference onMachine Learning, ICML 2019, 9-15 June 2019,Long Beach, California, USA, volume 97 of Pro-ceedings of Machine Learning Research, pages 21–29. PMLR.

Pratyay Banerjee, Kuntal Kumar Pal, Arindam Mi-tra, and Chitta Baral. 2019. Careful selection ofknowledge to solve open book question answering.In Proceedings of the 57th Annual Meeting of theAssociation for Computational Linguistics, pages6120–6129, Florence, Italy. Association for Compu-tational Linguistics.

Lisa Bauer, Yicheng Wang, and Mohit Bansal. 2018.Commonsense for generative multi-hop question an-swering tasks. In Proceedings of the 2018 Confer-ence on Empirical Methods in Natural Language

1304

Processing, Brussels, Belgium, October 31 - Novem-ber 4, 2018, pages 4220–4230. Association for Com-putational Linguistics.

Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot,Ashish Sabharwal, Carissa Schoenick, and OyvindTafjord. 2018. Think you have solved questionanswering? try arc, the AI2 reasoning challenge.CoRR, abs/1803.05457.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, andKristina Toutanova. 2019. BERT: Pre-training ofdeep bidirectional transformers for language under-standing. In Proceedings of the 2019 Conferenceof the North American Chapter of the Associationfor Computational Linguistics: Human LanguageTechnologies, Volume 1 (Long and Short Papers),pages 4171–4186, Minneapolis, Minnesota. Associ-ation for Computational Linguistics.

Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley,Oriol Vinyals, and George E. Dahl. 2017. Neuralmessage passing for quantum chemistry. In Pro-ceedings of the 34th International Conference onMachine Learning, ICML 2017, Sydney, NSW, Aus-tralia, 6-11 August 2017, volume 70 of Proceedingsof Machine Learning Research, pages 1263–1272.PMLR.

Daniel Khashabi, Tushar Khot, Ashish Sabharwal,Oyvind Tafjord, Peter Clark, and Hannaneh Ha-jishirzi. 2020. Unifiedqa: Crossing format bound-aries with a single qa system.

Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutionalnetworks. In 5th International Conference on Learn-ing Representations, ICLR 2017, Toulon, France,April 24-26, 2017, Conference Track Proceedings.OpenReview.net.

Souvik Kundu, Tushar Khot, Ashish Sabharwal, andPeter Clark. 2019. Exploiting explicit paths formulti-hop reading comprehension. In Proceedingsof the 57th Annual Meeting of the Association forComputational Linguistics, pages 2737–2747, Flo-rence, Italy. Association for Computational Linguis-tics.

John D. Lafferty, Andrew McCallum, and FernandoC. N. Pereira. 2001. Conditional random fields:Probabilistic models for segmenting and labeling se-quence data. In Proceedings of the Eighteenth Inter-national Conference on Machine Learning (ICML2001), Williams College, Williamstown, MA, USA,June 28 - July 1, 2001, pages 282–289. MorganKaufmann.

Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang,and Eduard Hovy. 2017. RACE: Large-scale ReAd-ing comprehension dataset from examinations. InProceedings of the 2017 Conference on EmpiricalMethods in Natural Language Processing, pages785–794, Copenhagen, Denmark. Association forComputational Linguistics.

Zhen-Zhong Lan, Mingda Chen, Sebastian Good-man, Kevin Gimpel, Piyush Sharma, and RaduSoricut. 2019. Albert: A lite bert for self-supervised learning of language representations.ArXiv, abs/1909.11942.

Shiyang Li, Jianshu Chen, and Dian Yu. 2019. Teach-ing pretrained models with commonsense reason-ing: A preliminary kb-based approach. CoRR,abs/1909.09743.

Yang Li and Peter Clark. 2015. Answering elementaryscience questions by constructing coherent scenesusing background knowledge. In Proceedings ofthe 2015 Conference on Empirical Methods in Nat-ural Language Processing, pages 2007–2012, Lis-bon, Portugal. Association for Computational Lin-guistics.

Bill Yuchen Lin, Xinyue Chen, Jamin Chen, and Xi-ang Ren. 2019. KagNet: Knowledge-aware graphnetworks for commonsense reasoning. In Proceed-ings of the 2019 Conference on Empirical Methodsin Natural Language Processing and the 9th Inter-national Joint Conference on Natural Language Pro-cessing (EMNLP-IJCNLP), pages 2829–2839, HongKong, China. Association for Computational Lin-guistics.

Liyuan Liu, Haoming Jiang, Pengcheng He, WeizhuChen, Xiaodong Liu, Jianfeng Gao, and Jiawei Han.2019a. On the variance of the adaptive learning rateand beyond. CoRR, abs/1908.03265.

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man-dar Joshi, Danqi Chen, Omer Levy, Mike Lewis,Luke Zettlemoyer, and Veselin Stoyanov. 2019b.Roberta: A robustly optimized BERT pretraining ap-proach. CoRR, abs/1907.11692.

Shangwen Lv, Daya Guo, Jingjing Xu, Duyu Tang,Nan Duan, Ming Gong, Linjun Shou, Daxin Jiang,Guihong Cao, and Songlin Hu. 2019. Graph-based reasoning over heterogeneous external knowl-edge for commonsense question answering. CoRR,abs/1909.05311.

Kaixin Ma, Jonathan Francis, Quanyang Lu, Eric Ny-berg, and Alessandro Oltramari. 2019. Towardsgeneralizable neuro-symbolic systems for common-sense question answering. In Proceedings of theFirst Workshop on Commonsense Inference in Natu-ral Language Processing, pages 22–32, Hong Kong,China. Association for Computational Linguistics.

Todor Mihaylov, Peter Clark, Tushar Khot, and AshishSabharwal. 2018. Can a suit of armor conduct elec-tricity? a new dataset for open book question an-swering. In Proceedings of the 2018 Conference onEmpirical Methods in Natural Language Processing,pages 2381–2391, Brussels, Belgium. Associationfor Computational Linguistics.

Todor Mihaylov and Anette Frank. 2018. Knowledge-able reader: Enhancing cloze-style reading compre-hension with external commonsense knowledge. In

1305

Proceedings of the 56th Annual Meeting of the As-sociation for Computational Linguistics (Volume 1:Long Papers), pages 821–832, Melbourne, Australia.Association for Computational Linguistics.

Giannis Nikolentzos, George Dasoulas, and MichalisVazirgiannis. 2019. k-hop graph neural networks.CoRR, abs/1907.06051.

Xiaoman Pan, Kai Sun, Dian Yu, Heng Ji, and Dong Yu.2019. Improving question answering with externalknowledge. CoRR, abs/1902.00993.

Colin Raffel, Noam Shazeer, Adam Roberts, KatherineLee, Sharan Narang, Michael Matena, Yanqi Zhou,Wei Li, and Peter J. Liu. 2019. Exploring the limitsof transfer learning with a unified text-to-text trans-former.

Adam Santoro, David Raposo, David G. T. Barrett,Mateusz Malinowski, Razvan Pascanu, Peter W.Battaglia, and Tim Lillicrap. 2017. A simple neu-ral network module for relational reasoning. In Ad-vances in Neural Information Processing Systems30: Annual Conference on Neural Information Pro-cessing Systems 2017, 4-9 December 2017, LongBeach, CA, USA, pages 4967–4976.

Maarten Sap, Hannah Rashkin, Derek Chen, RonanLe Bras, and Yejin Choi. 2019. Social IQa: Com-monsense reasoning about social interactions. InProceedings of the 2019 Conference on EmpiricalMethods in Natural Language Processing and the9th International Joint Conference on Natural Lan-guage Processing (EMNLP-IJCNLP), pages 4463–4473, Hong Kong, China. Association for Computa-tional Linguistics.

Michael Sejr Schlichtkrull, Thomas N. Kipf, PeterBloem, Rianne van den Berg, Ivan Titov, and MaxWelling. 2018. Modeling relational data with graphconvolutional networks. In The Semantic Web - 15thInternational Conference, ESWC 2018, Heraklion,Crete, Greece, June 3-7, 2018, Proceedings, volume10843 of Lecture Notes in Computer Science, pages593–607. Springer.

Robyn Speer, Joshua Chin, and Catherine Havasi. 2017.Conceptnet 5.5: An open multilingual graph of gen-eral knowledge. In Proceedings of the Thirty-FirstAAAI Conference on Artificial Intelligence, Febru-ary 4-9, 2017, San Francisco, California, USA,pages 4444–4451. AAAI Press.

Alon Talmor, Jonathan Herzig, Nicholas Lourie, andJonathan Berant. 2019. CommonsenseQA: A ques-tion answering challenge targeting commonsenseknowledge. In Proceedings of the 2019 Conferenceof the North American Chapter of the Associationfor Computational Linguistics: Human LanguageTechnologies, Volume 1 (Long and Short Papers),pages 4149–4158, Minneapolis, Minnesota. Associ-ation for Computational Linguistics.

Ashish Vaswani, Noam Shazeer, Niki Parmar, JakobUszkoreit, Llion Jones, Aidan N. Gomez, LukaszKaiser, and Illia Polosukhin. 2017. Attention is allyou need. In Advances in Neural Information Pro-cessing Systems 30: Annual Conference on NeuralInformation Processing Systems 2017, 4-9 Decem-ber 2017, Long Beach, CA, USA, pages 5998–6008.

Petar Velickovic, Guillem Cucurull, Arantxa Casanova,Adriana Romero, Pietro Lio, and Yoshua Bengio.2018. Graph attention networks. In 6th Inter-national Conference on Learning Representations,ICLR 2018, Vancouver, BC, Canada, April 30 - May3, 2018, Conference Track Proceedings. OpenRe-view.net.

Shuohang Wang and Jing Jiang. 2016. Learning nat-ural language inference with LSTM. In Proceed-ings of the 2016 Conference of the North Ameri-can Chapter of the Association for ComputationalLinguistics: Human Language Technologies, pages1442–1451, San Diego, California. Association forComputational Linguistics.

Xiaoyan Wang, Pavan Kapanipathi, Ryan Musa,Mo Yu, Kartik Talamadupula, Ibrahim Abdelaziz,Maria Chang, Achille Fokoue, Bassem Makni,Nicholas Mattei, and Michael Witbrock. 2019. Im-proving natural language inference using externalknowledge in the science questions domain. In TheThirty-Third AAAI Conference on Artificial Intelli-gence, AAAI 2019, The Thirty-First Innovative Ap-plications of Artificial Intelligence Conference, IAAI2019, The Ninth AAAI Symposium on EducationalAdvances in Artificial Intelligence, EAAI 2019, Hon-olulu, Hawaii, USA, January 27 - February 1, 2019,pages 7208–7215. AAAI Press.

Dirk Weissenborn, Tomas Kocisky, and Chris Dyer.2017. Dynamic integration of background knowl-edge in neural nlu systems. arXiv preprintarXiv:1706.02596.

An Yang, Quan Wang, Jing Liu, Kai Liu, Yajuan Lyu,Hua Wu, Qiaoqiao She, and Sujian Li. 2019. En-hancing pre-trained language representations withrich knowledge for machine reading comprehension.In Proceedings of the 57th Annual Meeting of theAssociation for Computational Linguistics, pages2346–2357, Florence, Italy. Association for Compu-tational Linguistics.

Bishan Yang and Tom Mitchell. 2017. Leveragingknowledge bases in LSTMs for improving machinereading. In Proceedings of the 55th Annual Meet-ing of the Association for Computational Linguistics(Volume 1: Long Papers), pages 1436–1446, Van-couver, Canada. Association for Computational Lin-guistics.

Zhi-Xiu Ye, Qian Chen, Wen Wang, and Zhen-HuaLing. 2019. Align, mask and select: A sim-ple method for incorporating commonsense knowl-edge into language representation models. CoRR,abs/1908.06725.

1306

Yuyu Zhang, Hanjun Dai, Kamil Toraman, andLe Song. 2018. Kgˆ2: Learning to reason scienceexam questions with contextual knowledge graphembeddings. CoRR, abs/1805.12393.

Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Gold-stein, and Jingjing Liu. 2020. Freelb: Enhanced ad-versarial training for natural language understanding.In International Conference on Learning Represen-tations.

1307

A Merging Types of Relations inConceptNet

Relation Merged Relation

AtLocation AtLocationLocatedNear

CausesCausesCausesDesire

*MotivatedByGoal

Antonym AntonymDistinctFrom

HasSubevent

HasSubevent

HasFirstSubeventHasLastSubeventHasPrerequisite

EntailsMannerOf

IsAIsAInstanceOf

DefinedAs

PartOf PartOf*HasA

RelatedToRelatedToSimilarTo

Synonym

Table 8: Relations in ConceptNet that are being mergedin pre-processing. *RelationX indicates the reverse re-lation of RelationX.

We merge relations that are close in semanticsas well as in the general usage of triple instancesin ConceptNet.

B Implementation Details

CommonsenseQA OpenbookQA

BERT-BASE 3 ✓ 10�5 -BERT-LARGE 2 ✓ 10�5 -ROBERTA-LARGE 1 ✓ 10�5 1 ✓ 10�5

Table 9: Learning rate for text encoders on differentdatasets.

Our models are implemented in PyTorch. We usecross-entropy loss and RAdam (Liu et al., 2019a)optimizer. We find it beneficial to use separatelearning rates for the text encoder and the graph en-coder. We tune learning rates for text encoders and

CommonsenseQA OpenbookQA

RN 3 ✓ 10�4 3 ✓ 10�4

RGCN 1 ✓ 10�3 1 ✓ 10�3

GconAttn 3 ✓ 10�4 1 ✓ 10�4

MHGRN 1 ✓ 10�3 1 ✓ 10�3

Table 10: Learning rate for graph encoders on differentdatasets.

#Param

RN 399KRGCN 365KGconAttn 453KMHGRN 544K

Table 11: Numbers of parameters of different graph en-coders.

graph encoders on two datasets. We first fine-tuneROBERTA-LARGE, BERT-LARGE, BERT-BASEon CommonsenseQA and ROBERTA-LARGE onOpenbookQA respectively, and choose a dataset-specific learning rate from {1✓10�5, 2✓10�5, 3✓10�5, 6 ✓ 10�5, 1 ✓ 10�4} for each text encoder,based on the best performance on development set,as listed in Table 9. We report the performance ofthese fine-tuned text encoders and also adopt theirdataset-specific optimal learning rates in joint train-ing with graph encoders. For models that involveKG, the learning rate of their graph encoders arechosen from {1 ✓ 10�4, 3 ✓ 10�4, 1 ✓ 10�3, 3 ✓10�3}, based on their best development set perfor-mance with ROBERTA-LARGE as the text encoder.We report the optimal learning rates for graph en-coders in Table 10. In training, we set the max-imum input sequence length to text encoders to64, batch size to 32, and perform early stopping.AristoRoBERTaV7+MHGRN is the only exception.In order to host fair comparison, we follow Aris-toRoBERTaV7 and set the batch size to 16, maxinput sequence length to 128, and choose a decoderlearning rate from {1 ✓ 10�3, 2 ✓ 10�5}.

For the input node features, we first use tem-plates to turn knowledge triples in ConceptNet intosentences and feed them into pre-trained BERT-LARGE, obtaining a sequence of token embeddingsfrom the last layer of BERT-LARGE for each triple.For each entity in ConceptNet, we perform meanpooling over the tokens of the entity’s occurrencesacross all the sentences to form a 1024d vector asits corresponding node feature. We use this set offeatures for all our implemented models.

1308

We use 2-layer RGCN and single-layer MHGRNacross our experiments.

The numbers of parameter for each graph en-coder are listed in Table 11.

C Dynamic Programming Algorithm forEq. 7

To show that multi-hop message passing can becomputed in linear time, we observe that Eq. 7 canbe re-written in matrix form:

Zk = (Dk)�1 =

(r1,...,rk)"Rk

�(r1, . . . , rk, s)�GArk⇧Ar1FXW

1r1

„⇧W

krk

„

�W k+10

„⇧W

K0

„ (1 & k & K), (13)

where G = diag(exp([g(�(v1), s), . . . , g(�(vn),s)]) (F is similarly defined), Ar is the adjacencymatrix for relation r and D

k is defined as follows:

Dk = diag⇧ =

(r1,...,rk)"Rk

�(r1, . . . , rk, s)�GArk⇧Ar1FX1↵ (1 & k & K) (14)

Using this matrix formulation, we can compute Eq.7 using dynamic programming:

D Formal Definition of K-hop RN

Definition 1 (K-hop Relation Network) A multi-hop relation network is a function that maps amulti-relational graph to a fixed size vector:

KHopRN(G; W , E, H) = K

=k=1

=(j,r1,...,rk,i)"�k

j"Q i"A

�(j, r1, . . . , rk, i)�W (hjh(er1`⇧`erk)hhi),(15)

where ` denotes element-wise product and �(⇧) =1/(K∂A∂ � ∂{(j ¨, . . . , i) " G ∂ j ¨ " Q}∂) defines thepooling weights.

E Expressing K-hop RN with MultiGRN

Theorem 1 Given any W , E, H, there exists a pa-rameter setting such that the output of the modelbecomes KHopRN(G;W , E, H) for arbitrary G.

Algorithm 1 Dynamic programming algorithm formulti-hop message passing.Input: s,X,Ar(1 & r & m),W t

r (r " R, 1 & t &k),F ,G, �, ⌧Output: Z

1: WK ⇥ I

2: for k ⇥ K � 1 to 1 do3: W

k ⇥ Wk+10 W

k+1

4: end for5: for r " R do6: Mr ⇥ FX

7: end for8: for k ⇥ 1 to K do9: if k > 1 then

10: for r " R do11: M

¨r ⇥ e�(r,s)Ar <r¨"R e⌧(r¨,r,s)Mr¨W

kr

„

12: end for13: for r " R do14: Mr ⇥ M

¨r

15: end for16: else17: for r " R do18: Mr ⇥ e�(r,s) �ArMrW

kr

„

19: end for20: end if21: Z

k ⇥ G<r"R MrWk

22: end for23: Replace W

tr (0 & r & m, 1 & t & k) with identity

matrices and X with 1 and re-run line 1 - line 19 tocompute d

1, . . . ,dK

24: for k ⇥ 1 to K do25: Z

k ⇥ (diag(dk))�1Zk

26: end for27: return Z

1,Z2, . . . ,Zk

1309

Proof. Suppose W = [W1, W2, W3], whereW1, W3 " Rd3✓d1 , W2 " Rd3✓d2 . ForMHRGN, we set the parameters as follows:H = H,Uò = [I;0] " R(d1+d2)✓d1 , bò =[0,1]„ " Rd1+d2 ,W t

r = diag(1 h er) "R(d1+d2)✓(d1+d2)(r " R, 1 & t & K),V = W3 "Rd3✓d1 ,V ¨ = [W1, W2] " Rd3✓(d1+d2). We dis-able the relation type attention module and enablemessage passing only from Q to A. By furtherchoosing � as the identity function and perform-ing pooling over A, we observe that the output ofMultiGRN becomes:

1∂A∂ =i"A

h¨i

= 1∂A∂ ⇥V hi + V¨zi�

= 1

K∂A∂K

=k=1

⇥V hi + V¨zki �

=K

=k=1

=(j,r1,...,rk,i)"�k

j"Q, i"A

�(j, r1, . . . , rk, i)⇤V hi+

V¨W

krk⇧W

1r1xj

=K

=k=1

=(j,r1,...,rk,i)"�k

j"Q

�(⇧)⇤V hi + V¨W

krk

⇧W1r1U�(j)hj + V

¨W

krk⇧W

1r1b�(j)

=K

=k=1

=(j,r1,...,rk,i)"�k

j"Q, i"A

�(⇧)⇤W3hi + W1hj

+ W2(er1 `⇧ ` erk) =

K

=k=1

=(j,r1,...,rk,i)"�k

j"Q, i"A

�(⇧)W ⇤hjh

(er1 `⇧ ` erk)h hi = RN(G; W , E, H)

(16)