Advanced Natural Language Processing Lecture 21 Discourse ... · Advanced Natural Language Processing Lecture 21 Discourse and its Structures (abbreviated version) ... primarily by

Advanced Natural Language ProcessingLecture 21

Discourse and its Structures(abbreviated version)

Bonnie Webber

9 November 2012

Webber ANLP Lecture 21 9 November 2012

1

What are discourse structures?Discourse structures are the patterns one sees in multi-sentence (multi-clausal)texts.

Recognizing these pattern(s) and what they convey is useful for deriving intendedinformation from the text.

Researchers in Language Technology (LT) are beginning to be able to recognizeand exploit these patterns for useful ends.


2

What kind of patterns?

• Topic patterns, each topic about a set of set of entities and what’s being saidabout them, as in text books, encyclopedias, and other expository text.

• Functional patterns, each element serving a particular purpose with respect tothe discourse as a whole or some other segment of discourse, as in essays, legalarguments, and scientific research papers;

• Patterns of eventualities – Events and states, and their spatio-temporalrelations, both of which are an essential part of narratives.

At a lower level, discourse shows patterns of coherence relations – aka discourserelations – between abstract objects, which is what a discourse segment isinterpreted as.


3

Elements of discourse structure

The slides in anlp21-full.pdf discuss all these types of patterns.

For lack of time, we focus here on coherence relations, since Assignment 3focusses on a possible link between anaphor resolution and coherence relations.


4

Coherence RelationsCoherence relations are binary relations that hold between pairs of segments in atext — primarily by virtue of their meaning.

Lexicalized means that there is lexico-syntactic evidence for the existence of acoherence relation, though possibly ambiguous with respect to sense.

The largest manually annotated corpus of lexicalized coherence relations is thePenn Discourse TreeBank (PDTB), over the 1m-word Penn Wall Street JournalCorpus.

N.B. Similar corpora are being developed for other languages (e.g., Turkish,Hindi, Chinese, Modern Standard Arabic) and genres (biomedical journal papers,dialogue).


5

Lexicalized Approach to Coherence RelationsTwo primary sources of lexico-syntactic evidence for a coherence relation are:

• explicit connectives that express a relation between clauses (e.g. coordinatingand subordinating conjunctions, discourse adverbials);

• implicit connectives between otherwise unmarked adjacent sentences, if one ormore explicit connectives can be inferred that express the relation(s) betweenthem.

The latter have the same status as the implicit relations between noun-nounmodifiers in English:

(1) container ship crane operator courses(courses for operators of cranes for ships carrying containers)


6

PDTB Annotation of Coherence RelationsAll coherence relations identified to date are binary, with two and only twoarguments.

In the PDTB, the arg syntactically attached to the connective is called arg2, andthe other, arg1.

(2) By most measures, the nation’s industrial sector is now growing very slowly –if at all. Factory payrolls fell in September. So did the Federal Reserve Board’sindustrial-production index. Yet many economists aren’t predicting that theeconomy is about to slip into recession. [wsj 0036]

Annotators first annotated all explicit connectives in the PDTB and their twoarguments.


7

PDTB Annotation of Coherence RelationsFor coherence relations between adjacent sentences, if annotators could infer ≥1connective(s) that express the relation(s) between the sentences, they insert theconnective(s), with all or part of the first sentence as arg1, and all, part of orpossibly more than the second, as arg2.

(3) Mr. Lane’s final purpose isn’t to glamorize the Artist’s vagabond existence.He has a point he wants to make, and he makes it, with a

great deal of force. [wsj 0039]

These are called implicit connectives.


8

PDTB Annotation of Coherence RelationsFor coherence relations between adjacent sentences, if annotators could infer ≥1connective(s) that express the relation(s) between the sentences, they insert theconnective(s), with all or part of the first sentence as arg1, and all, part of orpossibly more than the second, as arg2.

(3) Mr. Lane’s final purpose isn’t to glamorize the Artist’s vagabond existence.Implicit=rather He has a point he wants to make, and he makes it, witha great deal of force. [wsj 0039]

These are called implicit connectives.

If sentence 2 contained any explicit inter-S connective, it was taken to holdbetween the pair, so no further annotation was added.


9

Other lexicalizations of discourse relationsIf annotators felt the relation was already expressed, they were asked to annotatethe lexico-syntactic evidence for it as an alternative lexicalization or AltLex:

(4) The two companies each produce market pulp, containerboard and whitepaper. That means goods could be manufactured closer to customers, savingshipping costs, he said. [wsj 0317]

(5) The new structure would be similar to a recapitalization in whichholders get a special dividend yet retain a controlling ownership interest.The difference is that current holders wouldn’t retain majority ownership orcontrol. [wsj 1531]


10

Other types of relations

When an adjacent sentence appeared only related to its predecessor throughentity-based coherence, this was labelled EntRel.

(6) Hale Milgrim, 41 years old, senior vice president, marketing at ElecktraEntertainment Inc., was named president of Capitol Records Inc., a unit ofthis entertainment concern. EntRel Mr. Milgrim succeeds David Berman, whoresigned last month. [wsj 0945]


11

No coherence relation

If no relation could be perceived between a successive pair of sentences, NoRelwas explicitly marked between them.

(7) Dodge reported an 8% increase in construction contracts awarded inSeptember. NoRel The goverment counts money as it is spent [wsj 0036]


12

Total relation annotation in the PDTB

PDTB Relations No. of tokens

Explicit 18459Implicit 16224AltLex 624EntRel 5210NoRel 254

Total 40600

Many strings annotated as AltLex in the PDTB 2.0, will be included as explicitconnectives in the next version of the corpus.


13

PDTB Sense Hierarchy


14

PDTB Sense Hierarchy: Temporal

• Synchronous

• Asynchronous (precedence, succession)


15

PDTB Sense Hierarchy: Comparison

• Contrast (juxtaposition, opposition)

• Concession (expectation, contra-expectation)

• Pragmatic Contrast

• Pragmatic Concession

(2) By most measures, the nation’s industrial sector is now growing very slowly– if at all. . . . Yet [Comp.Concession.contra-expectation] manyeconomists aren’t predicting that the economy is about to slip into recession.[wsj 0036]


16

PDTB Sense Hierarchy: Contingency

• Cause (reason, result)

• Condition

• Pragmatic cause (justification)

• Pragmatic condition

(4) The two companies each produce market pulp, containerboard and whitepaper. [Cont.Cause.result] That means goods could be manufacturedcloser to customers, saving shipping costs, he said. [wsj 0317]


17

PDTB Sense Hierarchy: Expansion

• Conjunction

• Instantiation

• Restatement

• Alternative (conjunctive, disjunctive, chosen alternative)

• Exception

• List

(3) Mr. Lane’s final purpose isn’t to glamorize the Artist’s vagabond existence.[Exp.Alternative.chosen alternative] He has a point he wants tomake, and he makes it, with a great deal of force. [wsj 0039]


18

Automatically recognizing coherence relationsTask involves:

• Identifying the evidence for the discourse relation – ie, evidence for the“discourse predicate”;

• Identifying the arguments related by that predicate;

• Identifying the sense of the relation.

[Elwell & Baldridge, 2008; Lin et al, 2010; Pitler & Nenkova, 2009; Prasad et al.2008; Prasad, Joshi & Webber, 2010; Wellner & Pustejovsky, 2007]


19

Classifying sense relationsEvidence for the sense relation between discourse elements may be explicit or onlyimplicit.

Some explicit discourse connectives are unambiguous:

Conn sense Conn sense

accordingly result (5/5) in addition conjunction (165/165)

additionally conjunction (7/7) moreover conjunction (100/101)

afterward precedence (11/11) so result (262/263)

as a result result (78/78) thus result (112/112)

consequently result (10/10) till precedence (3/3)

for instance instantiation (98/98) unless disjunctive (94/95)


20

Classifying sense relationsBut several common connectives can express ≥1 sense:

• since: contingency.cause.reason (94), temporal.succession (78)

• as: temporal.synchrony (387), contingency.cause.reason (166)

• and: contingency.cause.result (38),expansion.conjunction (2543),both of these simultaneously (138)

Using lexical and syntactic features, [pit09b] train a simple Naive Bayes classifierto 94.15% accuracy in disambiguating between explicit connectives expressingcontingency vs. temporal vs. comparison vs. expansion.

I haven’t seen similar figures for the more useful level-2 classes.


21

Resources annotated with Coherence Relations

• Modern Standard Arabic [Al-Saif and Markert, 2011]

• Chinese [Zhou and Xue, 2012]

• Czech [Mladova et al, 2008]

• Dutch [van der Vliet et al, 2011]

• German [Stede, 2004]

• Hindi [Oza et al, 2009]

• Turkish [Zeyrek et al, 2010]

• specific genres, such as biomedical text [Prasad et al, 2011] and discoursefound in dialogue [Tonelli et al, 2010]


22

References◦ Amal Al-Saif and Katya Markert (2011). Modelling discourse relations for Arabic. Proceedings, Empirical Methodsin Natural Language Processing, pp. 736–747.

◦ Robert Elwell and Jason Baldridge (2008). Discourse connective argument identication with connective specicrankers. Proc. IEEE Conference on Semantic Computing (ICSC-08), Santa Clara CA.

◦ Ziheng Lin, Hwee Tou Ng and Min-Yen Kan (2010). A PDTB-Styled End-to-End Discourse Parser. TechnicalReport TRB8/10. School of Computing, National University of Singapore.

◦ Lucie Mladova, Sarka Zikanova, and Eva Hajicova (2008). From sentence to discourse. Proc. 6th Int’l Conf. onLanguage Resources and Evaluation.

◦ Umangi Oza, Rashmi Prasad, Sudheer Kolachina, Dipti Misra Sharma and Aravind Joshi (2009). The HindiDiscourse Relation Bank. Proc. 3rd Linguistic Annotation Workshop (LAW III). Singapore.

◦ Emily Pitler and Ani Nenkova (2009). Using syntax to disambiguate explicit discourse connectives in text.Proceedings, 47th Meeting of the Association for Computational Linguistics, pp. 13–16.

◦ Rashmi Prasad, Aravind Joshi, and Bonnie Webber (2010). Exploiting scope for shallow discourse parsing.Proceedings, 7th Int’l Conference on Language Resources and Evaluation (LREC), pp. 2076–2083.


23

◦ Rashmi Prasad, Susan McRoy, Nadya Frid, Aravind Joshi and Hong Yu (2011). The Biomedical Discourse RelationBank. BMC Bioinformatics, 12(188), 18 pages. (http://www.biomedcentral.com/1471-2015/12/188)

◦ Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber(2008). The Penn Discourse TreeBank 2.0. Proc. 6th Int’l Conference on Language Resources and Evaluation.

◦ Manfred Stede (2004). The Potsdam Commentary Corpus. ACL Workshop on Discourse Annotation, Barcelona.

◦ Sara Tonelli, Guiseppe Riccardi, Rashmi Prasad and Aravind Joshi (2010). Annotation of Discourse Relations forConversational Spoken Dialogs. Proceedings, 7th International Conference on Language Resources and Evaluation.

◦ Nynke van der Vliet, Ildiko Berzlanovich, Gosse Bouma, Markus Egg and Gisela Redeker (2011). Building adiscourse-annotated Dutch text corpus. Beyond Semantics, Gottingen, Germany.

◦ Ben Wellner and James Pustejovsky (2007). Automatically identifying the arguments of discourse connectives.Proceedings, Conference on Empirical Methods in Natural Language Processing (EMNLP-07).

◦ Deniz Zeyrek, Isin Demirsahin, A. Sevdik-Callı, et al (2010). The annotation scheme of the Turkish Discourse Bankand an evaluation of inconsistent annotations. Proceedings of the 4th Linguistic Annotation Workshop (LAW III).

◦ Yuping Zhou and Nianwen Xue (2012). PDTB-style discourse annotation of chinese text. Proc. 50th AnnualMeeting of the ACL, Jeju Island, Korea.


Documents

Advanced Natural Language Processing Lecture 21 Discourse ... · Advanced Natural Language Processing Lecture 21 Discourse and its Structures (abbreviated version) ... primarily by