View
219
Download
2
Category
Tags:
Preview:
Citation preview
1
The Long Road from Text to Meaning
Adam KilgarriffLexical Computing LtdLexicography MasterClass LtdUniversities of Leeds and Sussex
Adam Kilgarriff2 May 2007
Overview
Research programme Examples:
Corpus lexicographyWord sketchingCollocationalityThesauruses
The long road
Adam Kilgarriff3 May 2007
What is language?
Adam Kilgarriff4 May 2007
What is language? In our heads
Adam Kilgarriff5 May 2007
What is language? In our heads In texts and sound signals
Adam Kilgarriff6 May 2007
What is language? In our heads In texts and sound signals Both
Adam Kilgarriff7 May 2007
Methodology
Study language in our headsCompetenceChomsky“rationalist” (Descartes, Leibniz)
Adam Kilgarriff8 May 2007
Methodology
Study language in our headsCompetenceChomsky“rationalist” (Descartes, Leibniz)Odd method for objective sciencePractical problems: coverage,
arbitrariness
Adam Kilgarriff9 May 2007
Methodology
Study text“empiricist” (Locke, Hume)
Physics: forces, matterChemistry: chemicals, bondsLanguage: text, speech signals
Adam Kilgarriff10 May 2007
It goes against the grain
What is important about a sentence?its meaning
Corpus methodology:Throw away individual sentence
meaningFind patterns
Adam Kilgarriff11 May 2007
Computer power
Corpora bigger and bigger data sets
Language technology toolslemmatizers, POS-taggers, parsersMachine learning, pattern-finding
18 years of rapid ascent
Adam Kilgarriff12 May 2007
A virtuous circle
Pattern finding
Linguisticprocessing
Corpus
Lexicon•Part-of-speech tagging
•Parsing
•Lemmatizing
More data →
gets richer eachtime round
Adam Kilgarriff13 May 2007
Example: corpus lexicography
- four ages
Adam Kilgarriff14 May 2007
Age 1:Pre-computer
Oxford English Dictionary:• 20 million index cards
Adam Kilgarriff15 May 2007
Age 2: KWIC Concordances
From 1980 Computerised
Adam Kilgarriff16 May 2007
Age 2: KWIC Concordance1 arity, which will be used to take a party of under-privileged children to D 2 from outside. You are invited to a party and after a couple of drinks you d 3 tion, we believe politicians of all parties will listen to our views. &equo 4 ould be reaching agreement with all parties concerned, as to which events, 5 lack people. I have certainly been party to one or two discussions amongst 6 . These should be discussed by both parties before entering into the relatio 7 presents They had hosted a cocktail party at Kensington palace, for example 8 akes. By midnight the end-of-course party is in full swing, but most cadet 9 e should be a right for the injured party to terminate the contract. A mana 10 by the Safran Peoples ' Liberation Party. This presents the powerful neigh 11 s. Ahead I could see the rest of my party plodding towards the final slope t 12 cial ethic. The two main political parties - the Tories and the Liberals - 13 ritish successes in Perth The small party of British players competing in th 14 to help control. One member of the party went to summon the rescue team and 15 rket society fashion magazine. The party was held at his flat which was a l 16 security and secrecy than any Tory Party Conference : it seems that bootleg
Adam Kilgarriff17 May 2007
Age 2: KWIC Concordances
From 1980 Computerised COBUILD project was innovator the coloured-pens method
Adam Kilgarriff18 May 2007
1 political association 4 person in an agreement/dispute 2 social event 5 to be party to something...3 group of people
1 arity, which will be used to take a party of under-privileged children to D2 from outside. You are invited to a party and after a couple of drinks you d3 tion, we believe politicians of all parties will listen to our views. &equo4 ould be reaching agreement with all parties concerned, as to which events,5 lack people. I have certainly been party to one or two discussions amongst6 . These should be discussed by both parties before entering into the relatio7 presents They had hosted a cocktail party at Kensington palace, for example8 akes. By midnight the end-of-course party is in full swing, but most cadet9 e should be a right for the injured party to terminate the contract. A mana10 by the Safran Peoples ' Liberation Party. This presents the powerful neigh11 s. Ahead I could see the rest of my party plodding towards the final slope t12 cial ethic. The two main political parties - the Tories and the Liberals -13 ritish successes in Perth The small party of British players competing in th14 to help control. One member of the party went to summon the rescue team and15 rket society fashion magazine. The party was held at his flat which was a l16 security and secrecy than any Tory Party Conference : it seems that bootleg
The coloured pens method
Adam Kilgarriff19 May 2007
Age 2: limitations
as corpora get bigger:too much data
• 50 lines for a word: read all • 500 lines: could read all, takes a long
time• 5000 lines: no
Adam Kilgarriff20 May 2007
Age 3: Collocation statistics
Problem:too much data - how to summarise?
Solution:list of words occurring in neighbourhood of headword, with frequencies
Sorted by salience
Adam Kilgarriff21 May 2007
Collocation listing For collocates of save (>5 hits), window 1-5 words to right of nodeword
word word
forests life
$1.2 dollars
lives costs
enormous thousands
annually face
jobs estimated
money your
Adam Kilgarriff22 May 2007
Age 4: The word sketch
A corpus-derived one-page summary of a word’s grammatical and collocational behaviour
Adam Kilgarriff23 May 2007
Age 4: The word sketch
Large well-balanced corpus Parse to find
subjects, objects, heads, modifiers etc
One list for each grammatical relation
Statistics to sort each list, as before
Adam Kilgarriff24 May 2007
Macmillan English DictionaryFor Advanced Learners
Ed: Rundell, 2002
Adam Kilgarriff25 May 2007
Euralex 2002
Adam Kilgarriff26 May 2007
Euralex 2002
Can I have them for my language please
Adam Kilgarriff27 May 2007
The Sketch Engine Input:
any corpus, any language• Lemmatised, part-of-speech tagged
specification of grammatical relations
Word sketches integrated with corpus query system
Developer: Pavel Rychly, Brno
Adam Kilgarriff28 May 2007
Users: Dictionary publishers
• Oxford UP, Collins, Chambers, Macmillan
Universities • Teaching, research• Framenet
Language teaching http://www.sketchengine.co.uk/
• Self-registration for free trial account
Adam Kilgarriff29 May 2007
Collocationality
Which words are most ‘collocational’ Dictionary publishers
Where to put ‘collocation boxes’ Language learners
Adam Kilgarriff30 May 2007
Verb Freq MLE Prob x log = entropy
Take 2084 -.469
Gain 131 -.169
Offer 117 -.157
See 110 -.150
Enjoy 67 -.104
… … …
Clarify 1 -0.0031
… … …
Total 3730 -3.909
Calculation of entropy for advantage (object relation).
Adam Kilgarriff31 May 2007
Adam Kilgarriff32 May 2007
place (17881), attention (8476), door (8426), care (4884), step (4277), advantage (3730), rise (3334), attempt (2825), impression (2596), notice (2462), chapter (2318), mistake (2205), breath (2140), hold (1949), birth (1016), living (953), indication (812), tribute (720), debut (714), button (661), eyebrow (649), anniversary (637), mention (615), glimpse (531), suicide (486), toll (472), refuge (470), spokesman (453), sigh (436), birthday (429), wicket (412), appendix (410), pardon (399), precaution (396), temptation (374), goodbye (372), fuss (366), resemblance (350), goodness (288), precedence (285), havoc (270), tennis (266), comeback (260), farewell (228), prominence (228), go-ahead (202), sip (198),
Adam Kilgarriff33 May 2007
Thesaurus
a resource that groups words according to similarity
Adam Kilgarriff34 May 2007
Manual and automatic
Manual Roget, WordNets, many publishers
Automatic Sparck Jones (1960s), Lin (1998) aka distributional two words are similar if they occur in same
contexts Are they comparable?
Adam Kilgarriff35 May 2007
Thesauruses in NLP sparse data
Adam Kilgarriff36 May 2007
Thesauruses in NLP sparse data
does x go with y? don’t know, they have never been seen
together New question:
does x+friends go with y+friends indirect evidence for x and y thesaurus tells us who friends are “backing off”
Adam Kilgarriff37 May 2007
Relevant in: Parsing
PP-attachment conjunction scope
Bridging anaphors Text cohesion Word sense disambiguation (WSD) Speech understanding Spelling correction
Adam Kilgarriff38 May 2007
Conjunction scope Compare
old boots and shoesold boots and apples
Adam Kilgarriff39 May 2007
Conjunction scope Compare
old boots and shoesold boots and apples
Are the shoes old?
Adam Kilgarriff40 May 2007
Conjunction scope Compare
old boots and shoesold boots and apples
Are the shoes old? Are the apples old?
Adam Kilgarriff41 May 2007
Conjunction scope Compare
old boots and shoesold boots and apples
Are the shoes old? Are the apples old? Hypothesis:
wide scope only when words are similar
Adam Kilgarriff42 May 2007
(demo)
Adam Kilgarriff43 May 2007
Words and word senses automatic thesauruses
words
Adam Kilgarriff44 May 2007
Words and word senses automatic thesauruses
words manual thesauruses
simple hierarchy is appealinghomonyms
Adam Kilgarriff45 May 2007
Words and word senses automatic thesauruses
words manual thesauruses
simple hierarchy is appealinghomonyms“aha! objects must be word senses”
Adam Kilgarriff46 May 2007
Problems
Theoretical Practical
Adam Kilgarriff47 May 2007
Theoretical
Adam Kilgarriff48 May 2007
Adam Kilgarriff49 May 2007
Adam Kilgarriff50 May 2007
Wittgenstein
Don’t ask for the meaning, ask for the use
Adam Kilgarriff51 May 2007
Practical
Adam Kilgarriff52 May 2007
Problems Practical
a thesaurus is a toolif the tool organises words senses
you must do WSD before you can use it
WSD: state of the art, optimal conditions: 80%
Adam Kilgarriff53 May 2007
Problems
“To use this tool, first replace one fifth of your input with junk”
Adam Kilgarriff54 May 2007
Avoid word senses
Adam Kilgarriff55 May 2007
Avoid word senses This word has three meanings/senses
Adam Kilgarriff56 May 2007
Avoid word senses This word has three meanings/senses This word has three kinds of use
well foundedempiricalwe can build on it
Adam Kilgarriff57 May 2007
Avoid word senses This word has three meanings/senses This word has three kinds of use
well foundedempiricalwe can build on it
Adam Kilgarriff58 May 2007
sorry, roget
Adam Kilgarriff59 May 2007
sorry, AI
Adam Kilgarriff60 May 2007
sorry, AI AI model for NLP:
NLP turns text into meanings AI reasons over meanings word meanings are concepts in an
ontology a Roget-like thesaurus is (to a good
approximation) an ontology Guarino: “cleansing” WordNet
If a thesaurus groups words in their various uses (not meanings) not the sort of thing AI can reason over
Adam Kilgarriff61 May 2007
“The Future of Search”Berkeley, 4 May 2007
NLP panelPell, Hodjat, RussellThe AI dream for 50 years
• Interpret• Search on meanings
Implicit: word senses
Adam Kilgarriff62 May 2007
Quine
“No entity without identity”
Foundations for sound science Identity conditions for
word senses? Noword uses? Yes
Adam Kilgarriff63 May 2007
“linguistics expressions prompt for meanings rather than express meanings”Fauconnier and Turner 2003
It would be nice if … But …
Adam Kilgarriff64 May 2007
A virtuous circle
Pattern finding
Linguisticprocessing
Corpus
Lexicon•Part-of-speech tagging
•Parsing
•Lemmatizing
More data →
gets richer eachtime round
Word sketching
Thesaurus
Adam Kilgarriff65 May 2007
The long journey from text towards meaning
Raw text
Pure meaning
Rationalists
Empiricists
Adam Kilgarriff66 May 2007
The long journey from text towards meaning
Raw text
Pure meaning
Rationalists
Empiricistslemmatizer
POS-taggerparser
thesaurusthematic relations/frame elements
Adam Kilgarriff67 May 2007
Thank you
http://www.sketchengine.co.uk
Recommended