Lecture: Word Senses

Seman&c Analysis in Language Technology http://stp.lingfil.uu.se/~santinim/sais/2016/sais_2016.htm

Word Senses

Marina San(ni

[email protected]

Department of Linguis(cs and Philology

Uppsala University, Uppsala, Sweden

Spring 2016

Previous Lecture: Sen$ment Analysis

•  Affec(ve meaning is a kind of connota(onal meaning •  The importance of sen(ment lexicons •  Methods for the automa(c expansion of manually-‐annotated lexicons

•  A baseline algorithm: Naive Bayes •  Prac(cal ac(vity: ML-‐based sen(ment classifier: movie reviews, product reviews, restaurant reviews… •  Results… uhm… somehow biassed (short text vs long texts); never a full posi(ve polarity; etc.

2 Lecture 4: Word Senses

How is *meaning* handled in Seman$c-‐Based LT-‐Applica$ons?

•  Seman(c Role Labelling/Predicate-‐Argument Structure •  Main trend:

•  crea(on of annotated resources (PropBank, FrameNet, etc.); •  use of supervised machine learning: classifiers are trained on annotated resources, such as PropBank and

FrameNet. •  Sen(ment Analysis

•  Main trends: •  iden(fica(on of sen(ment-‐bearing features or iden(fica(on of representa(ve features for the problem •  use of supervervised learning à cf the results of NLTK classifier

•  Word sense disambigua(on (???) •  Informa(on extrac(on (???) •  Ques(on Answering (???) •  Ontologies (???)

Lecture 4: Word Senses 3

Reminder: Glossary Entries

•  Which concepts are the most salient in Lect 3?

•  Update your Glossary…


Previous lecture: end


Word Senses

•  Master Students à NLP course •  Bachelor Students à Seman(cs


From Lect 2: PropBank & Selec$onal Restric$ons

•  PropBank is organized by word senses : word senses are different aspects of meaning of a word

•  Selec(onal restric(ons… we can use seman(c constraints to disambiguate senses. Ex: eat, serve…. bear me some pa(ence…


Acknowledgements Most slides borrowed from:

Dan Jurafsky and James H. Mar(n

Some slides borrowed from D. Jurafsky and C. Manning and D. Radev (Coursera)

J&M(2015, draf): hgps://web.stanford.edu/~jurafsky/slp3/

Outline

•  Word Meaning •  WordNet and Other Lexical Resources •  Selec(onal Restric(ons


Logic: meaning representa$on: uppercase words!

•  Constant •  Variables •  Predicates •  Boolean connec(ves •  Quan(fiers •  Brackets and comma to group the symbols together

•  Ex: A woman crosses Sunset Boulevard Lecture 4: Word Senses 10

Formal Seman(cs

Defini$ons •  Lexical seman$cs is the study of the meaning of words and the systema(c

meaning-‐related connec(ons between words. •  A word sense is the locus of word meaning; defini(ons and meaning rela(ons

are defined at the level of the word sense rather than wordforms. •  Homonymy is the rela(on between unrelated senses that share a form. •  Polysemy is the rela(on between related senses that share a form. •  Synonymy holds between different words with the same meaning. •  Hyponymy and hypernymy rela(ons hold between words that are in a class

inclusion rela(onship. •  Meronymy type of hierarchy that deals with part–whole rela(onships. •  WordNet is a large database of lexical rela(ons for English 11

Lecture 4: Word Senses

Word Meaning and Similarity

Word Senses and Word Rela(ons

Reminder: lemma and wordform

•  A lemma or cita$on form •  Same stem, part of speech, rough seman(cs

•  A wordform •  The “inflected” word as it appears in text

Wordform Lemma banks bank sung sing duermes dormir Lecture 4: Word Senses 13

Cf. token/type ra(o: crude measure of lexical densi(y: If a text is 1,000 words long, it is said to have 1,000 "tokens". But a lot of these words will be repeated, and there may be only say 400 different words in the text. "Types", therefore, are the different words. The ratio between types and tokens in this example would be 40%. (source: wordsmith tools)

Lemmas have senses

•  One lemma “bank” can have many meanings: •  …a bank can hold the investments in a custodial account…!

•  “…as agriculture burgeons on the east bank the river will shrink even more”

•  Sense (or word sense) •  A discrete representa(on of an aspect of a word’s meaning.

•  The lemma bank here has two senses

1!

2!

Sense 1:

Sense 2:


Other examples?


Homonymy

Homonyms: words that share a form but have unrelated, dis(nct meanings:

•  bank1: financial ins(tu(on, bank2: sloping land •  bat1: club for hiqng a ball, bat2: nocturnal flying mammal

1.  Homographs (bank/bank, bat/bat) 2.  Homophones:

1.  Write and right 2.  Piece and peace


Homonymy causes problems for NLP applica$ons

•  Informa(on retrieval •  “bat care”!

•  Machine Transla(on •  bat: murciélago (animal) or bate (for baseball)

•  Text-‐to-‐Speech •  bass (stringed instrument) vs. bass (fish)

•  There would be no ambiguity for Speech to Text: why? Lecture 4: Word Senses 17

Polysemy

•  1. The bank was constructed in 1875 out of local red brick. •  2. I withdrew the money from the bank •  Are those the same sense?

•  Sense 2: “A financial ins(tu(on” •  Sense 1: “The building belonging to a financial ins(tu(on”

•  A polysemous word has related meanings •  Most non-‐rare words have mul(ple meanings


Polysemy


•  Lots of types of polysemy are systema(c •  School, university, hospital!•  All can mean the ins(tu(on or the building.

•  A systema(c rela(onship: •  Building Organiza(on

•  Other such kinds of systema(c polysemy: Author (Jane Austen wrote Emma)

Works of Author (I love Jane Austen) Tree (Plums have beautiful blossoms) ! !Fruit (I ate a preserved plum)!

Metonymy or Systema$c Polysemy: A systema$c rela$onship between senses


How do we know when a word has more than one sense?


How do we know when a word has more than one sense?

•  The “zeugma” test: Two senses of serve? •  Which flights serve breakfast?!•  Does Lufthansa serve Philadelphia?!•  ?Does Lufhansa serve breakfast and San Jose?

•  Since this conjunc(on sounds weird, •  we say that these are two different senses of “serve”


Synonyms

•  Word that have the same meaning in some or all contexts. •  filbert / hazelnut •  couch / sofa •  big / large •  automobile / car •  vomit / throw up •  Water / H20

•  Two lexemes are synonyms •  if they can be subs(tuted for each other in all situa(ons •  If so they have the same proposi$onal meaning


Synonyms

•  But there are few (or no) examples of perfect synonymy. •  Even if many aspects of meaning are iden(cal •  S(ll may not preserve the acceptability based on no(ons of politeness, slang, register, genre, etc.

•  Example: •  Water/H20 •  Big/large •  Brave/courageous high brow: la(nate words


Synonymy is a rela$on between senses rather than words

•  Consider the words big and large •  Are they synonyms?

•  How big is that plane? •  Would I be flying on a large or small plane?

•  How about here: •  Miss Nelson became a kind of big sister to Benjamin. •  ?Miss Nelson became a kind of large sister to Benjamin.

•  Why? •  big has a sense that means being older, or grown up •  large lacks this sense


Synonymy: Summary


Other seman$c rela$ons…


Antonyms

•  Senses that are opposites with respect to one feature of meaning •  Otherwise, they are very similar!

dark/light short/long !fast/slow !rise/fall!hot/cold! up/down! in/out!

•  More formally: antonyms can •  define a binary opposi(on

or be at opposite ends of a scale •  long/short, fast/slow!

•  Be reversives: •  rise/fall, up/down!


Hyponymy and Hypernymy

•  One sense is a hyponym of another if the first sense is more specific, deno(ng a subclass of the other •  car is a hyponym of vehicle •  mango is a hyponym of fruit

•  Conversely hypernym/superordinate (“hyper is super”) •  vehicle is a hypernym of car •  fruit is a hypernym of mango

Superordinate/hyper vehicle fruit furniture

Subordinate/hyponym car mango chair Lecture 4: Word Senses 29

Hyponymy more formally •  Extensional:

•  The class denoted by the superordinate extensionally includes the class denoted by the hyponym

•  Entailment: •  A sense A is a hyponym of sense B if being an A entails being a B

•  Hyponymy is usually transi(ve •  (A hypo B and B hypo C entails A hypo C)

•  Another name: the IS-‐A hierarchy •  A IS-‐A B (or A ISA B) •  B subsumes A Lecture 4: Word Senses 30

Hyponyms and Instances

•  WordNet has both classes and instances. •  An instance is an individual, a proper noun that is a unique en(ty

•  San Francisco is an instance of city!•  But city is a class •  city is a hyponym of municipality...location...!


WordNet and other Online Thesauri

Applica$ons of Thesauri and Ontologies

•  Informa(on Extrac(on •  Informa(on Retrieval •  Ques(on Answering •  Bioinforma(cs and Medical Informa(cs •  Machine Transla(on


WordNet


Synsets


How is “sense” defined in WordNet? •  The synset (synonym set), the set of near-‐synonyms,

instan(ates a sense or concept, with a gloss •  Example: chump as a noun with the gloss:

“a person who is gullible and easy to take advantage of”

•  This sense of “chump” is shared by 9 words: chump1, fool2, gull1, mark9, patsy1, fall guy1, sucker1, soft touch1, mug2!

•  Each of these senses have this same gloss •  (Not every sense; sense 2 of gull is the aqua(c bird)


gullible=naive

Tree-‐like Structure


WordNet: bar 1/6


2/6


3/6


4/6


5/6


6/6


Polysemy


WordNet 3.0

•  A hierarchically organized lexical database •  On-‐line thesaurus + aspects of a dic(onary

•  Some other languages available or under development •  (Arabic, Finnish, German, Portuguese…)

Category Unique Strings Noun 117,798 Verb 11,529 Adjec(ve 22,479 Adverb 4,481


Senses of “bass” in Wordnet


WordNet Hypernym Hierarchy for “bass”


WordNet Noun Rela$ons


WordNet 3.0

•  Where it is: • hgp://wordnetweb.princeton.edu/perl/webwn

•  Libraries • Python: WordNet from NLTK • hgp://www.nltk.org/Home

•  Java: •  JWNL, extJWNL on sourceforge

Lecture 4: Word Senses

Synset

•  MeSH (Medical Subject Headings) •  177,000 entry terms that correspond to 26,142 biomedical “headings”

•  Hemoglobins Entry Terms: Eryhem, Ferrous Hemoglobin, Hemoglobin Defini$on: The oxygen-‐carrying proteins of ERYTHROCYTES. They are found in all vertebrates and some invertebrates. The number of globin subunits in the hemoglobin quaternary structure differs between species. Structures range from monomeric to a variety of mul(meric arrangements

MeSH: Medical Subject Headings thesaurus from the Na$onal Library of Medicine


The MeSH Hierarchy

•  a


Uses of the MeSH Ontology

•  Provide synonyms (“entry terms”) •  E.g., glucose and dextrose

•  Provide hypernyms (from the hierarchy) •  E.g., glucose ISA monosaccharide

•  Indexing in MEDLINE/PubMED database •  NLM’s bibliographic database: •  20 million journal ar(cles •  Each ar(cle hand-‐assigned 10-‐20 MeSH terms Lecture 4: Word Senses 52

Other resources


BabelNet


Selectional Restrictions

Selec$onal Restric$ons

Consider the two interpreta(ons of: I want to eat someplace nearby.

a)  sensible: Eat is intransi(ve and “someplace nearby” is a loca(on adjunct

b)  Speaker is Godzilla: a monster that likes ea(ng buildings!!! Eat is transi(ve and “someplace nearby” is a direct object

How do we know speaker didn’t mean b) ? Because the THEME of ea(ng tends to be something edible


Selec$onal restric$ons are associated with senses

•  The restaurant serves green-‐lipped mussels. •  THEME is some kind of food

•  Which airlines serve Denver? •  THEME is an appropriate loca(on


apply zeugma test

Represen$ng selec$onal restric$ons

12 CHAPTER 22 • SEMANTIC ROLE LABELING

There are two possible parses and semantic interpretations for this sentence. Inthe sensible interpretation, eat is intransitive and the phrase someplace nearby isan adjunct that gives the location of the eating event. In the nonsensical speaker-as-Godzilla interpretation, eat is transitive and the phrase someplace nearby is the directobject and the THEME of the eating, like the NP Malaysian food in the followingsentences:(22.30) I want to eat Malaysian food.

How do we know that someplace nearby isn’t the direct object in this sentence?One useful cue is the semantic fact that the THEME of EATING events tends to besomething that is edible. This restriction placed by the verb eat on the filler of itsTHEME argument is a selectional restriction.

Selectional restrictions are associated with senses, not entire lexemes. We cansee this in the following examples of the lexeme serve:(22.31) The restaurant serves green-lipped mussels.(22.32) Which airlines serve Denver?Example (22.31) illustrates the offering-food sense of serve, which ordinarily re-stricts its THEME to be some kind of food Example (22.32) illustrates the provides acommercial service to sense of serve, which constrains its THEME to be some typeof appropriate location.

Selectional restrictions vary widely in their specificity. The verb imagine, forexample, imposes strict requirements on its AGENT role (restricting it to humansand other animate entities) but places very few semantic requirements on its THEMErole. A verb like diagonalize, on the other hand, places a very specific constrainton the filler of its THEME role: it has to be a matrix, while the arguments of theadjectives odorless are restricted to concepts that could possess an odor:(22.33) In rehearsal, I often ask the musicians to imagine a tennis game.(22.34) Radon is an odorless gas that can’t be detected by human senses.(22.35) To diagonalize a matrix is to find its eigenvalues.

These examples illustrate that the set of concepts we need to represent selectionalrestrictions (being a matrix, being able to possess an odor, etc) is quite open ended.This distinguishes selectional restrictions from other features for representing lexicalknowledge, like parts-of-speech, which are quite limited in number.

22.7.1 Representing Selectional RestrictionsOne way to capture the semantics of selectional restrictions is to use and extend theevent representation of Chapter 14. Recall that the neo-Davidsonian representationof an event consists of a single variable that stands for the event, a predicate denotingthe kind of event, and variables and relations for the event roles. Ignoring the issue ofthe l -structures and using thematic roles rather than deep event roles, the semanticcontribution of a verb like eat might look like the following:

9e,x,y Eating(e)Âgent(e,x)^T heme(e,y)

With this representation, all we know about y, the filler of the THEME role, is thatit is associated with an Eating event through the Theme relation. To stipulate theselectional restriction that y must be something edible, we simply add a new term tothat effect:

9e,x,y Eating(e)Âgent(e,x)^T heme(e,y)ÊdibleT hing(y)58

12 CHAPTER 22 • SEMANTIC ROLE LABELING

There are two possible parses and semantic interpretations for this sentence. Inthe sensible interpretation, eat is intransitive and the phrase someplace nearby isan adjunct that gives the location of the eating event. In the nonsensical speaker-as-Godzilla interpretation, eat is transitive and the phrase someplace nearby is the directobject and the THEME of the eating, like the NP Malaysian food in the followingsentences:(22.30) I want to eat Malaysian food.

How do we know that someplace nearby isn’t the direct object in this sentence?One useful cue is the semantic fact that the THEME of EATING events tends to besomething that is edible. This restriction placed by the verb eat on the filler of itsTHEME argument is a selectional restriction.

Selectional restrictions are associated with senses, not entire lexemes. We cansee this in the following examples of the lexeme serve:(22.31) The restaurant serves green-lipped mussels.(22.32) Which airlines serve Denver?Example (22.31) illustrates the offering-food sense of serve, which ordinarily re-stricts its THEME to be some kind of food Example (22.32) illustrates the provides acommercial service to sense of serve, which constrains its THEME to be some typeof appropriate location.

Selectional restrictions vary widely in their specificity. The verb imagine, forexample, imposes strict requirements on its AGENT role (restricting it to humansand other animate entities) but places very few semantic requirements on its THEMErole. A verb like diagonalize, on the other hand, places a very specific constrainton the filler of its THEME role: it has to be a matrix, while the arguments of theadjectives odorless are restricted to concepts that could possess an odor:(22.33) In rehearsal, I often ask the musicians to imagine a tennis game.(22.34) Radon is an odorless gas that can’t be detected by human senses.(22.35) To diagonalize a matrix is to find its eigenvalues.

These examples illustrate that the set of concepts we need to represent selectionalrestrictions (being a matrix, being able to possess an odor, etc) is quite open ended.This distinguishes selectional restrictions from other features for representing lexicalknowledge, like parts-of-speech, which are quite limited in number.

22.7.1 Representing Selectional RestrictionsOne way to capture the semantics of selectional restrictions is to use and extend theevent representation of Chapter 14. Recall that the neo-Davidsonian representationof an event consists of a single variable that stands for the event, a predicate denotingthe kind of event, and variables and relations for the event roles. Ignoring the issue ofthe l -structures and using thematic roles rather than deep event roles, the semanticcontribution of a verb like eat might look like the following:

9e,x,y Eating(e)Âgent(e,x)^T heme(e,y)

With this representation, all we know about y, the filler of the THEME role, is thatit is associated with an Eating event through the Theme relation. To stipulate theselectional restriction that y must be something edible, we simply add a new term tothat effect:

9e,x,y Eating(e)Âgent(e,x)^T heme(e,y)ÊdibleT hing(y)

22.7 • SELECTIONAL RESTRICTIONS 13

Sense 1hamburger, beefburger --(a fried cake of minced beef served on a bun)=> sandwich=> snack food=> dish=> nutriment, nourishment, nutrition...=> food, nutrient=> substance=> matter=> physical entity=> entity

Figure 22.6 Evidence from WordNet that hamburgers are edible.

When a phrase like ate a hamburger is encountered, a semantic analyzer canform the following kind of representation:

9e,x,y Eating(e)Êater(e,x)^T heme(e,y)ÊdibleT hing(y)^Hamburger(y)

This representation is perfectly reasonable since the membership of y in the categoryHamburger is consistent with its membership in the category EdibleThing, assuminga reasonable set of facts in the knowledge base. Correspondingly, the representationfor a phrase such as ate a takeoff would be ill-formed because membership in anevent-like category such as Takeoff would be inconsistent with membership in thecategory EdibleThing.

While this approach adequately captures the semantics of selectional restrictions,there are two problems with its direct use. First, using FOL to perform the simpletask of enforcing selectional restrictions is overkill. Other, far simpler, formalismscan do the job with far less computational cost. The second problem is that thisapproach presupposes a large, logical knowledge base of facts about the conceptsthat make up selectional restrictions. Unfortunately, although such common-senseknowledge bases are being developed, none currently have the kind of coveragenecessary to the task.

A more practical approach is to state selectional restrictions in terms of WordNetsynsets rather than as logical concepts. Each predicate simply specifies a WordNetsynset as the selectional restriction on each of its arguments. A meaning representa-tion is well-formed if the role filler word is a hyponym (subordinate) of this synset.

For our ate a hamburger example, for instance, we could set the selectionalrestriction on the THEME role of the verb eat to the synset {food, nutrient}, glossedas any substance that can be metabolized by an animal to give energy and buildtissue. Luckily, the chain of hypernyms for hamburger shown in Fig. 22.6 revealsthat hamburgers are indeed food. Again, the filler of a role need not match therestriction synset exactly; it just needs to have the synset as one of its superordinates.

We can apply this approach to the THEME roles of the verbs imagine, lift, and di-agonalize, discussed earlier. Let us restrict imagine’s THEME to the synset {entity},lift’s THEME to {physical entity}, and diagonalize to {matrix}. This arrangementcorrectly permits imagine a hamburger and lift a hamburger, while also correctlyruling out diagonalize a hamburger.

Instead of represen(ng “eat” as:

Just add:

And “eat a hamburger” becomes

But this assumes we have a large knowledge base of facts about edible things and hamburgers and whatnot.

Let’s use WordNet synsets to specify selec$onal restric$ons

•  The THEME of eat must be WordNet synset {food, nutrient} “any substance that can be metabolized by an animal to give energy and build 8ssue”

•  Similarly THEME of imagine: synset {en(ty} THEME of li=: synset {physical en(ty} THEME of diagonalize: synset {matrix}

•  This allows imagine a hamburger and li= a hamburger,

•  Correctly rules out diagonalize a hamburger.


The end

Education

Lecture: Word Senses