Combining Relational and Attributional Similarityfor Semantic Relation Classification
Preslav Nakov, National University of Singapore
Zornitsa Kozareva, University of Southern California
RANLP 2011
RANLP: Hissar, Bulgaria: September 14, 2011
2
What causes tumors to shrink?
The period of tumor shrinkage after radiation therapy is often long and varied.
CAUSE-EFFECT
How do we build a system to classify the relation between two nouns?
RANLP: Hissar, Bulgaria: September 14, 2011
3 Nakov & Kozareva: Combining Relational and Attributional Similarity ... 3
Relation Extraction (between Nouns)
• Given a pair of nouns, identify the semantic relation(s) between them
malaria mosquito
effect-cause
content-container
orange basket
RANLP: Hissar, Bulgaria: September 14, 2011
4Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Semantic Relations: Applications
Help real applications:
information extraction
document summarization
machine translation
construction of thesauri and semantic networks
Facilitate auxiliary tasks:
word sense disambiguation
language modeling
paraphrasing
recognizing textual entailment
RANLP: Hissar, Bulgaria: September 14, 2011
5 Nakov & Kozareva: Combining Relational and Attributional Similarity ...
PART-WHOLE (Winston, Chaffin, and Hermann 1987) Component-Integral object
cup-handle, kitchen-apartment, wing-bird
Member-Collectionsoldier-army, professor-faculty, tree-forest
Portion-Massslice-pie, meter-kilometer
Stuff-Objectsilk-dress, steel-car, and alcohol-wine
Feature-Activitypaying-shopping, chewing-eating
Place-AreaGeographic
oasis-desert, county-state, path-forest
Geometricthe end (of a stick) is part of (that) stickthe surface (of a lake) is part of (that) lakethe side (of a building) is part of (that) building
Semantic Relations:Can Be Heterogeneous
RANLP: Hissar, Bulgaria: September 14, 2011
6 Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Thus, often handled using instance-based classifiers
We need to measure SIMILARITY…
Semantic Relations:Can Be Heterogeneous
RANLP: Hissar, Bulgaria: September 14, 2011
7Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Two Kinds of SimilarityTurney (2006)
Rel1(X1,Y1) vs. Rel2(X2,Y2)
Attributional Similarity
Correspondence between attributes
X1::X2 (mason :: carpenter)
Y1::Y2 (stone :: wood)
Relational Similarity
Correspondence between relations
Rel1 :: Rel2 (mason:stone :: carpenter:wood)
mason:stone(a) teacher:chalk(b) carpenter:wood(c) soldier:gun(d) photograph:camera(e) book:word
RANLP: Hissar, Bulgaria: September 14, 2011
8Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Measuring Relational Similarity Turney (2006)
Attributional similarity can be used to measure relational similarity, e.g.
Fine for near analogies:
mason:stone(a) teacher:chalk(b) carpenter:wood(c) soldier:gun(d) photograph:camera(e) book:word
Bad for far analogies:
traffic:street(a) ship:gangplank(b) crop:harvest(c) car:garage(d) pedestrians:feet(e) water:riverbed
RANLP: Hissar, Bulgaria: September 14, 2011
9Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Semantic Relation Extraction
Going back to semantic relations…
Similar split between two lines of research: relational: rely on patterns that can connect the arguments
(Hearst,92;Turney,05; Turney&Littman,05; Turney,06; Kim&Baldwin,06; Pantel&Pennacchiotti,06; O’Seaghdha&Copestake,07; Davidov et al.07; Davidov&Rappoport,08; Nakov& Hearst,08; Katrenko et al.,10)
attributional: generalize the arguments using a lexical hierarchy (Rosario&Hearst,01; Rosario et al.,02; Girju et al.,05; Kim&Baldwin,07; O’Seaghdha,09)
RANLP: Hissar, Bulgaria: September 14, 2011
10 Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Pattern-based Relation Extraction
Pattern learning (relational)
for context-dependent, episodic relations?
e.g., CAUSE-EFFECT: My Friday’s exam causes me anxiety.
Argument generalization (attributional)
for context-independent, permanent relations?
e.g., PART-WHOLE: door-car
can benefit from pre-existing resources like WordNet
BUT limited coverage, need for WSD, etc.
RANLP: Hissar, Bulgaria: September 14, 2011
11 Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Our Objectives
Combine relational and attributional similarity
and study the relative importance for different relations
Use no pre-existing lexical resources
in the real-world, these resources have limited coverage
Human-readable, explicit semantic representation
ideally…
12 Nakov & Kozareva: Combining Relational and Attributional Similarity ...
RANLP: Hissar, Bulgaria: September 14, 2011
Method
RANLP: Hissar, Bulgaria: September 14, 2011
13Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Our Web-based Approach at a Glance
• Learn from the Web-
paraphrasing verbs, prepositions, coordinating conjunctions
- hypernyms and co-hyponyms for each argument
• Compute vector similarity for kNN
malaria mosquito
carried bytransmitted via
spreadscauses
insectdisease fly
RANLP: Hissar, Bulgaria: September 14, 2011
14Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Learning Relational Patterns
For “noun1 noun2”, query:
"noun1 THAT? * noun2"
"noun2 THAT? * noun1"
Extract:
V: verbs
P: prepositions
C: coordinating conjunctions
committee -- member
RANLP: Hissar, Bulgaria: September 14, 2011
15 Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Learning Relational Patterns (cont.)
coffee -- guy
RANLP: Hissar, Bulgaria: September 14, 2011
16Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Sample of harvested patterns forcoffee -- guy
Learned Relational Patterns
RANLP: Hissar, Bulgaria: September 14, 2011
17Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Generalizing Arguments
• Doubly-anchored pattern has anchoring through conjunctions or terms
- “ relation <seed> and * ”- “ <seed> and * relation * ”- “ * relation * and <seed> ”
• can mine both hypernyms and co-hyponyms• achieves higher accuracy than (Etzioni,05; Pasca,04)• easy to implement
“ making it vulnerable to predators such as jaguar and puma ”“ big cats such as jaguar and puma. Similar animal ”
“ car brands such as jaguar and land rover were known to be “
RANLP: Hissar, Bulgaria: September 14, 2011
Instantiate the noun arguments of each sentence with the doubly-anchored lexico-syntactic pattern
Submit each pattern as a query to Yahoo! Boss
Harvested hypernyms and co-hyponyms
Build a hypernym, co-hyponym vector
18Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Argument Harvesting Procedure
coffee andsuch as
beveragedrinkfood
teachocolatecocoa
product
RANLP: Hissar, Bulgaria: September 14, 2011
19Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Example: Learned (Co-)Hypernyms
“ * such as coffee and* ”
“* such as guy and * ”
20 Nakov & Kozareva: Combining Relational and Attributional Similarity ...
RANLP: Hissar, Bulgaria: September 14, 2011
Dataset
RANLP: Hissar, Bulgaria: September 14, 2011
21Nakov & Kozareva: Combining Relational and Attributional Similarity ...
SemEval-1 task 4:Classification of Semantic Relations between Nominals
Follow-up: SemEval-2 task 8Multi-Way Classification of Semantic Relations Between Pairs of Nominals
Dataset
RANLP: Hissar, Bulgaria: September 14, 2011
Dataset: SemEval-1 Task#4
22Nakov & Kozareva: Combining Relational and Attributional Similarity ...
RANLP: Hissar, Bulgaria: September 14, 2011
23 Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Experiments and Evaluation
RANLP: Hissar, Bulgaria: September 14, 2011
24Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Classifier, Weights, Similarity
Classifier: 1NN
Weighting functions
Frequencies
TF.IDF
TF.IDF w/ add-one smoothing for IDF
Similarity Measures
cosine
Dice
Lin’s measure
RANLP: Hissar, Bulgaria: September 14, 2011
25Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Features
Relational
verbs
prepositions
coordinating conjunctions
Attributional
nouns (arguments, attributes)
hypernyms
co-hyponyms
RANLP: Hissar, Bulgaria: September 14, 2011
26Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Combining Relational and Attributional Similarity
Linear combination for relational and attributional similarity
Sm: argument 1
Sh: argument 2
Sr: the relation
Weights tuned on the training set with cross-validation
Also tried:(a) Whyp = 1(b) Wcoh = 1
RANLP: Hissar, Bulgaria: September 14, 2011
Results: Overall Micro-averaged
Combining attributional and relational similarity outperforms: all baselines by 0.5-19.5% absolute (statistically significant in 15 out of 21 cases)
the best system at SemEval-1 Task 4 (66.0%): by 5.3% absolute the state-of-the-art (Davidov&Rapport, 2008), who had 70.1%: by 1.2%
When using manually annotated WordNet senses was not allowed.
27Nakov & Kozareva: Combining Relational and Attributional Similarity ...
RANLP: Hissar, Bulgaria: September 14, 2011
Results: Relational Similarity
28Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Instrument-Agency (laser--printer) and Product-Producer(honey--bee) are better characterized by patterns
For Instrument-Agency the head (argument 2) is also important hypernyms are more important than co-hyponyms
RANLP: Hissar, Bulgaria: September 14, 2011
Results: Attributional Similarity
29Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Theme-Tool (copyright--law) and Origin-Entity (olive--oil) are best characterized by the properties of the arguments
For Theme-Tool the head (argument 2) is most critical co-hyponyms are more important than hypernyms
RANLP: Hissar, Bulgaria: September 14, 2011
Results: Both Similarities
30Nakov & Kozareva: Combining Relational and Attributional Similarity ...
For Cause-Effect (growth -- hormone) both the modifier and the relation are important
For Cause-Effect the modifier (argument 1) is most critical again, co-hyponyms are more important than hypernyms
RANLP: Hissar, Bulgaria: September 14, 2011
Results: Comparing to WordNet
For some relations, the results outpeform those at SemEval-1 task 4, even when manual WordNet senses are allowed
Origin-Entity: 77.8% vs. 72.8% (stat. significant)
Theme-Tool: 74.7% vs. 74.6%
31Nakov & Kozareva: Combining Relational and Attributional Similarity ...
RANLP: Hissar, Bulgaria: September 14, 2011
32 Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Conclusion
RANLP: Hissar, Bulgaria: September 14, 2011
33 Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Conclusion
Presented: Web-based approach to relation extraction
Uses no pre-existing lexical resources
Based on human-readable, explicit semantic representationparaphrases: verbs, prepositions, coordinating conjunctions
hypernyms and co-hyponyms
Studied the combination of relational and attributional similarity for various relations
Achieved sizable improvements over a strong baseline
small improvements over the state-of-the-art
RANLP: Hissar, Bulgaria: September 14, 2011
34 Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Future Work
We plan to use other relation inventories, e.g., SemEval-2 task 8
model sentence context
jointly generalize arguments and paraphrases
use bootstrapping to mine more examples
try explicit paraphrases beyond verbs/preps/conj.
RANLP: Hissar, Bulgaria: September 14, 2011
35 Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Paraphrasing Semantics: Beyond Verbs
“onion tears”tears from onions
tears due to cutting onion
tears induced when cutting onions
tears that onions induce
tears that come from chopping onions
tears that sometimes flow when onions are chopped
tears that raw onions give you
SemEval-2013 task 24C. Butnariu, I. Hendrickx, S. N. Kim, Z. Kozareva, P. Nakov, D. Ó Séaghdha, S. Szpakowicz, T. Veale
RANLP: Hissar, Bulgaria: September 14, 2011
36 Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Future Work
We plan to use other relation inventories, e.g., SemEval-2 task 8
model sentence context
jointly generalize arguments and paraphrases
try explicit paraphrases beyond verbs/preps/conj.
use bootstrapping to mine more examples
Thank you!
RANLP: Hissar, Bulgaria: September 14, 2011
37 Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Detailed Results (1)
RANLP: Hissar, Bulgaria: September 14, 2011
38 Nakov & Kozareva: Combining Relational and Attributional Similarity ...
Detailed Results (2)