11

Discriminating Word Senses Discriminating Word Senses Using Using

McQuitty’s Similarity AnalysisMcQuitty’s Similarity Analysis

Amruta PurandareAmruta PurandareUniversity of Minnesota, DuluthUniversity of Minnesota, Duluth

Advisor : Dr Ted PedersenAdvisor : Dr Ted Pedersen

Research supported by National Science Foundation Research supported by National Science Foundation (NSF)(NSF)

Faculty Early Career Development Award Faculty Early Career Development Award (#0092784)(#0092784)

22

Discriminating “line”Discriminating “line”

They will begin line formation before ceremonyConnect modem to any jack on your line

Quit printing after the last line of each fileYour line will not get tied while you are connected to net

Stand balanced and comfortable during line upLines that do not fit a page are truncated

New line service provides reliable connections Pages are separated by line feed characters They stand far right when in line formation

33

They will begin line formation before ceremonyStand balanced and comfortable during line up

They stand far right when in line formation

Your line will not get tied while you are connected to netConnect modem to any jack on your line

New line service provides reliable connections

Quit printing after the last line of each pageLines that do not fit a page are truncated

Pages are separated by line feed characters

44

IntroductionIntroduction• What is Word Sense Discrimination ?What is Word Sense Discrimination ?

• Unsupervised learning Unsupervised learning

Training

Test

Features

Feature Vectors

ClustersClusters

55

Representing contextRepresenting context

• Features (from training)Features (from training)•Bi grams Bi grams

•Unigrams Unigrams

•Second Order Co-occurrences/SOCs Second Order Co-occurrences/SOCs (Schütze98)(Schütze98)

•MixtureMixture

• Feature vectors (Binary)Feature vectors (Binary)

• Measuring similarity Measuring similarity •CosineCosine

•MatchMatch

66

Feature examplesFeature examples

<features> for line<features> for line

UnigraUnigramm

<blank> <text> <service> <connection> <blank> <text> <service> <connection> <modem><modem>

<paragraph> <jack> <reliable> <circuit> <file><paragraph> <jack> <reliable> <circuit> <file>

Bi gramBi gram <blank, <blank, lineline> <text, > <text, lineline> >

<text, paragraph> <blank, space><text, paragraph> <blank, space>

<<lineline, service> <modem, jack>, service> <modem, jack>

<phone, service> <connection, <phone, service> <connection, lineline>>

<reliable, connection><reliable, connection>

SOCsSOCs <space> <paragraph> <phone> <reliable><space> <paragraph> <phone> <reliable>

77

McQuitty’s methodMcQuitty’s method

• Pedersen & Bruce, Pedersen & Bruce, 19971997

• AgglomerativeAgglomerative

• UPGMA / Average UPGMA / Average LinkLink

• Stopping rules Stopping rules – Number of clustersNumber of clusters– Score cutoffScore cutoff

88

EvaluationEvaluationS1S1 S2S2 S3S3 S4S4

C1C1 1010 00 33 22

C2C2 11 11 77 11

C3C3 22 11 11 66

C4C4 22 1515 11 22

99

EvaluationEvaluationS1S1 S3S3 S4S4 S2S2

C1C1 1010 33 22 00 1515

C2C2 11 77 11 11 1010

C3C3 22 11 66 11 1010

C4C4 22 11 22 1515 2020

1515 1212 1111 1717 5555

1010

Majority Sense ClassifierMajority Sense Classifier

1111

Experimental DataExperimental Data

LineLine Senseval-2Senseval-2

#Senses#Senses 66 VariableVariable

Selected top 5Selected top 5

#instanc#instanceses

41464146

(1200:600)(1200:600)120/word, 73 words120/word, 73 words

(100-150:50-100)(100-150:50-100)

1212

Scope of the experimentsScope of the experiments

• 584 experiments (73 * 4 * 2)584 experiments (73 * 4 * 2)– 73 Words: 72 Senseval-2, LINE73 Words: 72 Senseval-2, LINE– 4 Features: Bi grams, Unigrams, SOCs, Mix4 Features: Bi grams, Unigrams, SOCs, Mix– 2 Similarity Measures: Match, Cosine2 Similarity Measures: Match, Cosine

• Window = 5 Window = 5 – for Bi grams and SOCsfor Bi grams and SOCs

• Frequency cutoff = 2Frequency cutoff = 2

1313

Senseval-2 Results POS wiseSenseval-2 Results POS wise

66 77

55 33

77 88

COSCOS MATMAT

SOSOCC

BIBI

UNIUNI

COCOSS

MAMATT

COCOSS

MAMATT

11 11

00 00

11 00

1111 66

55 55

1313 99

SOSOCC

BIBI

UNIUNI

SOSOCC

BIBI

UNIUNI

No of words of a POS for which experiment obtained

accuracy more than Majority

1414

Senseval-2 Results Feature Senseval-2 Results Feature wisewise

66 77

1111 66

11 11

COSCOS MATMAT

NN

VV

ADJADJ

COCOSS

MAMATT COCO

SSMAMATT

77 88

1313 99

11 00

55 33

55 55

00 00

NN

VV

ADJADJNN

VV

ADJADJ

1515

Senseval-2 Results Measure Senseval-2 Results Measure wisewise

66 55 77

1111 55 1313

11 00 11

SOCSOC BIBI UNIUNI

NN

VV

ADJADJ

SOSOCC

BIBI UNIUNI

77 33 88

66 55 99

11 00 00

NN

VV

ADJADJ

1616

0.250.25 0.230.23

0.190.19 0.180.18

0.210.21 0.200.20

COSCOS MATMAT

SOCSOC

BIBI

UNIUNI

Line Results Line Results

On uniform distribution of 6 senses

1717

Sample Confusion Table Sample Confusion Table (fine.soc.cos)(fine.soc.cos)

5.005.00

11.6711.67

63.3363.33

16.6716.67

3.333.33

11.611.677

8.338.33 5050 23.323.333

6.66.677

22 00 00 11 00

11 00 44 22 00

22 55 2525 22 44

11 00 00 99 00

11 00 11 00 00

S0S0 S1S1 S2S2 S3S3 S4S4

77 55 3030 1414 44

33

77

3838

1010

22

60 S0 = elegantS1 = small grained

S2 = superior S3 = satisfactory

S4 = thin

1818

ConclusionsConclusions• Small set of SOCs was powerfulSmall set of SOCs was powerful

– Half the number of unigrams/bigramsHalf the number of unigrams/bigrams• Scaling done by Cosine helps !Scaling done by Cosine helps !• Need more training data!Need more training data!• Need to improve feature… Need to improve feature…

• Selection (Tests of associations)Selection (Tests of associations)• extraction (Stemming)extraction (Stemming)• matching (Fuzzy matching)matching (Fuzzy matching)

… …strategies for bi grams strategies for bi grams • Explore new featuresExplore new features

• POS POS • CollocationsCollocations

1919

Recent workRecent work

• PDL implementation PDL implementation • Cluto - Clustering Toolkit Cluto - Clustering Toolkit

http://www-users.cs.umn.edu/~karypis/clutohttp://www-users.cs.umn.edu/~karypis/cluto•6 clustering methods, 12 merging criteria6 clustering methods, 12 merging criteria

• PlansPlans– Comparing clustering in Comparing clustering in

similarity space Vs vector space (similarity space Vs vector space (Schütze, Schütze, 19981998))

– Stopping rulesStopping rules

2020

They will begin line formation before ceremonyStand balanced and comfortable during line up

They stand far right when in line formation

Your line will not get tied while you are connected to netConnect modem to any jack on your line

New line service provides reliable connections

Quit printing after the last line of each fileLines that do not fit a page are truncated

Pages are separated by line feed characters

Sense labelingSense labeling

2121

Software PackagesSoftware Packages

• SenseClusters SenseClusters (Our Discrimination Toolkit)(Our Discrimination Toolkit) http://www.d.umn.edu/~tpederse/senseclusters.htmlhttp://www.d.umn.edu/~tpederse/senseclusters.html

• PDL PDL (Used to implement clustering algorithms)(Used to implement clustering algorithms)http://pdl.perl.org/http://pdl.perl.org/

• NSP NSP (Used for extracting features)(Used for extracting features)http://www.d.umn.edu/~tpederse/nsp.htmlhttp://www.d.umn.edu/~tpederse/nsp.html

• SenseTools SenseTools (Used for preprocessing, feature (Used for preprocessing, feature matching)matching)http://www.d.umn.edu/~tpederse/sensetools.htmlhttp://www.d.umn.edu/~tpederse/sensetools.html

• Cluto Cluto (Clustering Toolkit)(Clustering Toolkit)http://www-users.cs.umn.edu/~karypis/clutohttp://www-users.cs.umn.edu/~karypis/cluto