Upload
samara
View
70
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Seminar by Nikhil: 06005004 Adith: 06005005 Prachur: 06D05011. Prosody and NLP. We have a presentation this Friday ? We have a presentation this Friday ? We have a presentation this Friday ?. Abstract. - PowerPoint PPT Presentation
Citation preview
Prosody and NLP
Seminar by Nikhil : 06005004Adith : 06005005
Prachur : 06D05011
We have a presentation this Friday ?We have a presentation this Friday ?We have a presentation this Friday ?
Abstract
Speech Processing and Natural Language Processing share a common area of study: Language. However, over time, they have grown to have little in common regarding theoretical models or methods of analysis. NLP takes written text as the starting point for it's analysis, however, a lot of valuable information is lost in encoding speech as merely text. It is commonly accepted that intonational features of spoken language can greatly aid NLP tasks (like adjective scope resolution).
We explore the foundations of the study of Prosody and observe some approaches that use prosodic cues to aid NLP.
Motivation
• Language is not text driven but speech driven.
• NLP currently has written text as the starting point for it's analysis (primarily due to the abundance of such data).
• A lot of information is lost on ignoring spoken features and just looking at the text.
A Way Out ?
• Utilize spoken features for NLP tasks
• NLP needs all the help it can get.
Dealing at the pragmatics or discourse level is extremely untenable
Prosodic cues carry useful pragmatic information
What is Prosody, exactly?
• Comes from Poetry, prosody refers to the study of poetic meters[1] (rhythms)
• Written text treats words as the basic building blocks of language.
• Spoken language treats syllables as the basic building block.
• Syllables : Consisting of Onset, Nucleus and Coda
• Wikipedia has this to say : Prosody is the rhythm, stress, and intonation of connected speech (as opposed to smaller elements like syllables or words).
Prosody may reflect : features of the speaker, emotional state of a speaker, features of the utterance, ironic or sarcastic, emphasis, contrast, and focus
What is Prosody, exactly? (contd.)
Intonation ?• Conveys paralinguistic information, emphasis and
contrast.
• Intonation on a particular word could differentiate between sentence moods.
– You are finISHED (interrogative)– You are FINIshed (imperative)
Image courtesy Google Image Search
• Stress is applied on Content Words in spoken utterances.
cOntent - Noun. "I really liked their presentation's content.“
contEnt - Verb. "I have done my best. I am content.“
• Stress on a pair of words distinguishes between the syntactic role played by each word in the pair.
• tight<pause>rope : A rope that is held taut.• tight-rope : A circus-act uses this contraption :)
[2]
And Stress!
Courtesy
tom The Dancing Bug
Prosodic cues
Prosodic functions important for linguistics are – Marking of boundaries (syntactic, semantic or
dialogue units.)
– Relative duration of phonetic segments
– At syllable level : Energy, intensity, duration and intonation of syllable.
We shall see two approaches of using these features in tasks central to NLP.
• Cue Used : Relative duration of phonetic segments
• Aim : To improve the parsing of ambiguous
sentences.
• Method : Augmenting the syntax grammar with a few
non-terminals and rules.
Prosody-Augmented Syntax Grammars[3]
• Concept of “Word Break Indices” used to show prosodic decoupling between neighboring words.
• E.g. - • Andrea 1 moved 1 the 0 bottle 3 under 0 the 0
bridge.
• Andrea 1 moved 3 the 0 bottle 1 under 0 the 0 bridge.
Break indices were generated by analysing the coda that have a pause.Coda is the final sounded consonant of a word E.g. – cup , milk
Grammar Modification
• Original grammar rules like S -> NP VP etc. are changed to S -> NP link1 VP.
• “Link” non-terminals are used for the word-break indices.
• The Parse Trees for the aforementioned example become…
Example 1
S
• Andrea 1 moved 1 the 0 bottle 3 under 0 the 0 bridge.
PP Vmoved NPN
Punder
NP
Nbottle
VP
Dthe
N bridge
NP
Dthe
Andrea
Link1
1
Link2
1 Link4
3
Link3
0
Link5
0
Link6
0
Example 2• Andrea 1 moved 3 the 0 bottle 1 under 0 the 0 bridge.
S
PP
Vmoved
NPN
Punder
NP
Nbottle
VP
Dthe
N bridge
NP
Dthe
Andrea
Link1
1
Link2
3
Link4
1
Link3
0
Link5
0
Link6
0
Results
• The incorporation of prosody resulted in a reduction of about 25% in the number of parses found .
• Parse times increase about 37%.
• Extremely common cases of syntactic ambiguity can be resolved with prosodic information, and that grammars can be modified to take advantage of prosodic information for improved parsing
Using Prosodic Features in Language Models[4]
• The outlined approach uses syllable-based prosodic cues, namely– Duration of the syllable– Average energy (intensity)– The average F0 (fundamental frequency of the
syllable) contour– The slope of the F0 contour (visualised as
intonation-rising or falling/flat)
Recognition of Prosodic Features
Courtesy [4]
Prosody in Language Model
• We want to measure P(wn | wn-1,wn-2,…,F)
• Naively modelled by linear interpolation : – Assumption : prosody features independent of
previous words (not true!!). P(wn | wn-1,wn-2,…,F) = αP(wn | wn-1,wn-2,…) + (1- α)P(wn|F)
• We want something better
Factored Language Model
• Instead of a word W we will deal with a set of word-factors F={f1,f2..fk} (Factors may include the word itself)
• Here, F is chosen as {W, prosodic features}
• The four prosodic features are encoded as binary numbers(s0 to s15).
• These numbers are assigned to each syllable of the word .
• For e.g. the prosodic representation for the word “Actually” can be either ‘s10s12s6’ or ‘s10s15s6’ .
Conclusion
• Prosodic Cues can play an important role as a heuristic for many NLP tasks.
• All is not one way traffic though. POS tagging (since its relatively accurate) is used to aid speech synthesis tasks which conventionally used only prosodic cues[5]
Future Work
• Handling prosodic information is a first step towards integration of Speech Processing and NLP
Courtesy ZITS
References1. Fromkin, Rodman and Hyams, An Introduction to
Language, 7th Ed, Thomson and Wadsworth,20022. En.wikipedia.org : Prosody, Austrian School of
Thought, Intonation3. John Bear and Patti Price (1990), “Prosody,
Syntax and Parsing”, in proceedings of the 28th annual meeting of the ACL
4. Songfang Huang and Steve Renals (2007), “Using Prosodic Features in Language Models for Meetings”, IRTG annual meeting
5. http://speech.iiit.net/~raghavendra/Webpage/ppprts.pdf