Click here to load reader

Automatic Acquisition of Paradigmatic Relations using Iterated Co-occurrences Chris Biemann, Stefan Bordag, Uwe Quasthoff University of Leipzig, NLP Department

  • View

  • Download

Embed Size (px)

Text of Automatic Acquisition of Paradigmatic Relations using Iterated Co-occurrences Chris Biemann, Stefan...

  • Slide 1

Automatic Acquisition of Paradigmatic Relations using Iterated Co-occurrences Chris Biemann, Stefan Bordag, Uwe Quasthoff University of Leipzig, NLP Department LREC 2004, Learning & Acquisition (II), 27th of May 2004 Slide 2 Chris Biemann 2 Sets of Words Our goal is the automatic extension of homogenous word sets, i.e. WordNet synsets or small subtrees of some hierarchy We collect methods and apply them, eventually in combination Mind experiment: the computer as associator: Input: some example concepts - Detection of the relation - Output of additional instances This can be done semi-supervised Necessary: - very large text corpus - features - methods Slide 3 Chris Biemann 3 Statistical Co-occurrences occurrence of two or more words within a well-defined unit of information (sentence, nearest neighbors) Significant Co-occurrences reflect relations between words Significance Measure (log-likelihood): - k is the number of sentences containing a and b together - ab is (number of sentences with a)*(number of sentences with b) - n is total number of sentences in corpus Slide 4 Chris Biemann 4 Iterating Co-occurrences (sentence-based) co-ocurrences of first order: words that co-occur significantly often together in sentences co-occurrences of second order: words that co-occur significantly often in collocation sets of first order co-occurrences of n-th order: words that co-occur significantly often in collocation sets of (n-1)th order When calculating a higher order, the significance values of the preceding order are not relevant. A co-occurrence set consists of the N highest ranked co-occurrences of a word. Slide 5 Chris Biemann 5 Constructed Example I Ord 1dogterriercatmousebarkingbiteyelp dog---XxX terrier---xxX cat--x-x- mouse--X-x- barkingXX---- biteXXxx-- yelpxx---- Ord 2dogterriercatmousebarkingbiteyelp dog311--- terrier311--- cat111--- mouse111-1- barking----22 bite---122 yelp----22 Slide 6 Chris Biemann 6 Constructed Example II Ord 3dogterriercatmousebarkingbiteyelp dog------ terrier----- cat------ mouse------ barking----11 bite----11 yelp----11 Ord 2dogterriercatmousebarkingbiteyelp dogx----- terrierx----- cat------ mouse------ barking----xx bite----xx yelp----xx Slide 7 Chris Biemann 7 Properties of Iterated Co-occurrences after some iterations the sets remain more or less stable the sets are somewhat semantically homogeneous sometimes, they have to do nothing with the reference word calculations performed until 10th order. Example for TOP 20 NB-collocations of 10th order forerklrte [explained]: sagte, schwrmte, lobt, schimpfte, meinte, jubelte, lobte, resmierte, schwrmt, Reinhard He, rgerte, kommentierte, urteilte, analysierte, bilanzierte, freute, freute sich, Bundestrainer, freut,gefreut [said, enthused, praises, grumbled, meant, was jubilant, praised, summarized, dreamt, Reinhard Hess, annoyed, commentated, judged, analyzed, balanced, made happy, was pleased, coach of the national team, is pleased, been pleased] Slide 8 Chris Biemann 8 Mapping co-occurrences to graphs For all words having co-occurrences, form nodes in a graph. Connect them all by edges, initialize edge weight with 0 For every co-occurrence of two words in a sentence, increase edge weight by significance Slide 9 Chris Biemann 9 First Iteration Step The two black nodes A and B get connected in the step if there are many nodes C which are connected to both A and B The more Cs, the higher the weight of the new edge new connection existing connection Slide 10 Chris Biemann 10 Second Iteration Step The two black nodes A and B get connected in the step if there are many (dark grey) nodes Ds which are connected to both A and B. The connections between the nodes Ds and the nodes A and B were constructed because of (light gray) nodes Es and Fs, respectively new connection former connection existing connection A B Ds Es Fs Slide 11 Chris Biemann 11 Collapsing bridging nodes Upper bound for path length in iteration n is 2 n. However, some of the bridging nodes collapse, giving rise to self-keeping clusters of arbitrary path length, which are invariant under iteration. Upper 5 nodes: invariant cluster A, B are being absorbed by this cluster Slide 12 Chris Biemann 12 Examples of Iterated Co-occurrences OrderReference word TOP-10 collocations N2winewines, champagne, beer, water, tea, coffee, Wine, alcoholic, beers, cider S10winewines, grape, sauvignon, chardonnay, noir, pinot, cabernet, spicy, bottle, grapes S1ringingphone, bells, phones, hook, bell, endorsement, distinctive, ears, alarm, telephone S2ringingrung, Centrex, rang, phone, sounded, bell, ring, FaxxMaster, sound, tolled S4ringingsounded, rung, rang, tolled, tolling, sound, tone, toll, ring, doorbell S10pressingCtrl, Shift, press, keypad, keys, key, keyboard, you, cursor, menu, PgDn, keyboards, numeric, Alt, Caps, CapsLock, NUMLOCK, NumLock, Scroll Slide 13 Chris Biemann 13 Intersection of Co-occurrence Sets: resolving ambiguity Herz- Bube Stich Becker Achtelfinale - Aufschlag - Boris Becker - Daviscup - Doppel - DTB Edberg - Finale - Graf - Haas - Halbfinale - Match - Pilic - Runde - Sampras - Satz - Tennis - Turnier - Viertelfinale - Weltrangliste - Wimbledon Alleinspieler - Herz - Herz-Dame - Herz- Knig - Hinterhand - Karo - Karo-As - Karo- Bube - Kreuz-As - Kreuz-Bube - Pik-As - Pik-Bube - Pik-Knig - Vorhand - Becker - Courier - Einzel - Elmshorn - French Open - Herz-As - ins - Kafelnikow - Karbacher - Krajicek - Kreuz-As - Kreuz-Bube - Michael Stich - Mittelhand - Pik-As - Pik-Bube - Pik-Knig bedient - folgenden - gereizt - Karo-Buben - Karo-Dame - Karo- Knig - Karte - Karten - Kreuz-Ass - Kreuz-Dame - Kreuz-Hand - Kreuz-Knig - legt - Mittelhand - Null ouvert - Pik - Pik-Ass - Pik- Dame - schmiert - Skat - spielt - Spielverlauf - sticht - bernimmt - zieht - Agassi - Australian Open - Bindewald - Boris - Break - Chang - Dickhaut - - gewann - Ivanisevic - Kafelnikow - Kiefer - Komljenovic - Leimen - Matchball - Michael Stich - Monte Carlo - Prinosil - Sieg - Spiel - spielen - Steeb - Teamchef Stich Slide 14 Chris Biemann 14 Example: NB-collocations of 2nd order warm, k hl, kalt Disjunction and filtering for adjectives of collocation sets for warm, khl, kalt [warm, cool, cold] results in: abgekhlt, aufgeheizt, eingefroren, erhitzt, erwrmt, gebrannt, gelagert, hei, heruntergekhlt, verbrannt, wrmer [cooled down, heated, frozen, heated up, warms up, burned, stored, hot, down- cooled, burned, more warmly] emotional reading abweisend [repelling] for khl, kalt is eliminated Slide 15 Chris Biemann 15 Detection of X-onyms synonyms, antonyms, (co)-hyponyms... Idea: Intersection of co-occurrence sets of two X-onyms as reference words should contain X-onyms lexical ambiguity of one reference word does not deteriorate the result set Method: - Detect word class for reference words - calculate co-occurrences for reference words - filter co-occurrences w.r.t the word class of the reference words (by means of POS tags) - perform disjunction of the co-occurrence sets - output result ranking can be realized over significance values of the co-occurrences Slide 16 Chris Biemann 16 Mini-Evaluation Experiments for different data sources, NB-collocations of 2nd and 3rd order fraction of X-onyms in TOP 5 higher than in TOP 10 ranking method makes sense disjunction of 2nd-order and 3rd-order collocations almost always empty different orders exhibit different relations satisfactory quantity, more through larger corpora quality: for unsupervised extension not precise enough Slide 17 Chris Biemann 17 Word Sets for Thesaurus Expansion Application: thesaurus expansion start set: [warm, kalt] [warm, cold] result set: [hei, wrmer, klter, erwrmt, gut, heier, hoch, hher, niedriger, schlecht, frei] [hot, warmer, colder, warmed, good, hotter, high, higher, lower, bad, free] start set: [gelb, rot] [yellow, red] result set: [blau, grn, schwarz, grau, bunt, leuchtend, rtlich, braun, dunkel, rotbraun, wei] [blue, green, black, grey, colorful, bright, reddish, brown, dark, red-brown, white] start set: [Mrder, Killer] [murderer, killer] result set: [Tter, Straftter, Verbrecher, Kriegsverbrecher, Ruber, Terroristen, Mann, Mitglieder, Mnner, Attentter] [offender, delinquent, criminal, war criminal, robber, terrorists, man, members, men, assassin Slide 18 Chris Biemann 18 More Examples in English Intersection of N2-Order collocation sets Slide 19 Chris Biemann 19 Questions? THANK YOU !

Search related