Combining Relational and Attributional Similarity for Semantic Relation Classification Preslav Nakov, National University of Singapore Zornitsa Kozareva,

Combining Relational and Attributional Similarityfor Semantic Relation Classification

Preslav Nakov, National University of Singapore

Zornitsa Kozareva, University of Southern California

RANLP 2011

RANLP: Hissar, Bulgaria: September 14, 2011

2

What causes tumors to shrink?

The period of tumor shrinkage after radiation therapy is often long and varied.

CAUSE-EFFECT

How do we build a system to classify the relation between two nouns?


3 Nakov & Kozareva: Combining Relational and Attributional Similarity ... 3

Relation Extraction (between Nouns)

• Given a pair of nouns, identify the semantic relation(s) between them

malaria mosquito

effect-cause

content-container

orange basket


4Nakov & Kozareva: Combining Relational and Attributional Similarity ...

Semantic Relations: Applications

Help real applications:

information extraction

document summarization

machine translation

construction of thesauri and semantic networks

Facilitate auxiliary tasks:

word sense disambiguation

language modeling

paraphrasing

recognizing textual entailment


5 Nakov & Kozareva: Combining Relational and Attributional Similarity ...

PART-WHOLE (Winston, Chaffin, and Hermann 1987) Component-Integral object

cup-handle, kitchen-apartment, wing-bird

Member-Collectionsoldier-army, professor-faculty, tree-forest

Portion-Massslice-pie, meter-kilometer

Stuff-Objectsilk-dress, steel-car, and alcohol-wine

Feature-Activitypaying-shopping, chewing-eating

Place-AreaGeographic

oasis-desert, county-state, path-forest

Geometricthe end (of a stick) is part of (that) stickthe surface (of a lake) is part of (that) lakethe side (of a building) is part of (that) building

Semantic Relations:Can Be Heterogeneous



Thus, often handled using instance-based classifiers

We need to measure SIMILARITY…

Semantic Relations:Can Be Heterogeneous



Two Kinds of SimilarityTurney (2006)

Rel1(X1,Y1) vs. Rel2(X2,Y2)

Attributional Similarity

Correspondence between attributes

X1::X2 (mason :: carpenter)

Y1::Y2 (stone :: wood)

Relational Similarity

Correspondence between relations

Rel1 :: Rel2 (mason:stone :: carpenter:wood)

mason:stone(a) teacher:chalk(b) carpenter:wood(c) soldier:gun(d) photograph:camera(e) book:word



Measuring Relational Similarity Turney (2006)

Attributional similarity can be used to measure relational similarity, e.g.

Fine for near analogies:

mason:stone(a) teacher:chalk(b) carpenter:wood(c) soldier:gun(d) photograph:camera(e) book:word

Bad for far analogies:

traffic:street(a) ship:gangplank(b) crop:harvest(c) car:garage(d) pedestrians:feet(e) water:riverbed



Semantic Relation Extraction

Going back to semantic relations…

Similar split between two lines of research: relational: rely on patterns that can connect the arguments

(Hearst,92;Turney,05; Turney&Littman,05; Turney,06; Kim&Baldwin,06; Pantel&Pennacchiotti,06; O’Seaghdha&Copestake,07; Davidov et al.07; Davidov&Rappoport,08; Nakov& Hearst,08; Katrenko et al.,10)

attributional: generalize the arguments using a lexical hierarchy (Rosario&Hearst,01; Rosario et al.,02; Girju et al.,05; Kim&Baldwin,07; O’Seaghdha,09)



Pattern-based Relation Extraction

Pattern learning (relational)

for context-dependent, episodic relations?

e.g., CAUSE-EFFECT: My Friday’s exam causes me anxiety.

Argument generalization (attributional)

for context-independent, permanent relations?

e.g., PART-WHOLE: door-car

can benefit from pre-existing resources like WordNet

BUT limited coverage, need for WSD, etc.



Our Objectives

Combine relational and attributional similarity

and study the relative importance for different relations

Use no pre-existing lexical resources

in the real-world, these resources have limited coverage

Human-readable, explicit semantic representation

ideally…



Method



Our Web-based Approach at a Glance

• Learn from the Web-

paraphrasing verbs, prepositions, coordinating conjunctions

- hypernyms and co-hyponyms for each argument

• Compute vector similarity for kNN

malaria mosquito

carried bytransmitted via

spreadscauses

insectdisease fly



Learning Relational Patterns

For “noun1 noun2”, query:

"noun1 THAT? * noun2"

"noun2 THAT? * noun1"

Extract:

V: verbs

P: prepositions

C: coordinating conjunctions

committee -- member



Learning Relational Patterns (cont.)

coffee -- guy



Sample of harvested patterns forcoffee -- guy

Learned Relational Patterns



Generalizing Arguments

• Doubly-anchored pattern has anchoring through conjunctions or terms

- “ relation <seed> and * ”- “ <seed> and * relation * ”- “ * relation * and <seed> ”

• can mine both hypernyms and co-hyponyms• achieves higher accuracy than (Etzioni,05; Pasca,04)• easy to implement

“ making it vulnerable to predators such as jaguar and puma ”“ big cats such as jaguar and puma. Similar animal ”

“ car brands such as jaguar and land rover were known to be “


Instantiate the noun arguments of each sentence with the doubly-anchored lexico-syntactic pattern

Submit each pattern as a query to Yahoo! Boss

Harvested hypernyms and co-hyponyms

Build a hypernym, co-hyponym vector


Argument Harvesting Procedure

coffee andsuch as

beveragedrinkfood

teachocolatecocoa

product



Example: Learned (Co-)Hypernyms

“ * such as coffee and* ”

“* such as guy and * ”



Dataset



SemEval-1 task 4:Classification of Semantic Relations between Nominals

Follow-up: SemEval-2 task 8Multi-Way Classification of Semantic Relations Between Pairs of Nominals

Dataset


Dataset: SemEval-1 Task#4




Experiments and Evaluation



Classifier, Weights, Similarity

Classifier: 1NN

Weighting functions

Frequencies

TF.IDF

TF.IDF w/ add-one smoothing for IDF

Similarity Measures

cosine

Dice

Lin’s measure



Features

Relational

verbs

prepositions

coordinating conjunctions

Attributional

nouns (arguments, attributes)

hypernyms

co-hyponyms



Combining Relational and Attributional Similarity

Linear combination for relational and attributional similarity

Sm: argument 1

Sh: argument 2

Sr: the relation

Weights tuned on the training set with cross-validation

Also tried:(a) Whyp = 1(b) Wcoh = 1


Results: Overall Micro-averaged

Combining attributional and relational similarity outperforms: all baselines by 0.5-19.5% absolute (statistically significant in 15 out of 21 cases)

the best system at SemEval-1 Task 4 (66.0%): by 5.3% absolute the state-of-the-art (Davidov&Rapport, 2008), who had 70.1%: by 1.2%

When using manually annotated WordNet senses was not allowed.



Results: Relational Similarity


Instrument-Agency (laser--printer) and Product-Producer(honey--bee) are better characterized by patterns

For Instrument-Agency the head (argument 2) is also important hypernyms are more important than co-hyponyms


Results: Attributional Similarity


Theme-Tool (copyright--law) and Origin-Entity (olive--oil) are best characterized by the properties of the arguments

For Theme-Tool the head (argument 2) is most critical co-hyponyms are more important than hypernyms


Results: Both Similarities


For Cause-Effect (growth -- hormone) both the modifier and the relation are important

For Cause-Effect the modifier (argument 1) is most critical again, co-hyponyms are more important than hypernyms


Results: Comparing to WordNet

For some relations, the results outpeform those at SemEval-1 task 4, even when manual WordNet senses are allowed

Origin-Entity: 77.8% vs. 72.8% (stat. significant)

Theme-Tool: 74.7% vs. 74.6%




Conclusion



Conclusion

Presented: Web-based approach to relation extraction

Uses no pre-existing lexical resources

Based on human-readable, explicit semantic representationparaphrases: verbs, prepositions, coordinating conjunctions

hypernyms and co-hyponyms

Studied the combination of relational and attributional similarity for various relations

Achieved sizable improvements over a strong baseline

small improvements over the state-of-the-art



Future Work

We plan to use other relation inventories, e.g., SemEval-2 task 8

model sentence context

jointly generalize arguments and paraphrases

use bootstrapping to mine more examples

try explicit paraphrases beyond verbs/preps/conj.



Paraphrasing Semantics: Beyond Verbs

“onion tears”tears from onions

tears due to cutting onion

tears induced when cutting onions

tears that onions induce

tears that come from chopping onions

tears that sometimes flow when onions are chopped

tears that raw onions give you

SemEval-2013 task 24C. Butnariu, I. Hendrickx, S. N. Kim, Z. Kozareva, P. Nakov, D. Ó Séaghdha, S. Szpakowicz, T. Veale



Future Work

We plan to use other relation inventories, e.g., SemEval-2 task 8

model sentence context

jointly generalize arguments and paraphrases

try explicit paraphrases beyond verbs/preps/conj.

use bootstrapping to mine more examples

Thank you!



Detailed Results (1)







Documents

Combining Relational and Attributional Similarity for Semantic Relation Classification Preslav Nakov, National University of Singapore Zornitsa Kozareva,