Upload
brasen
View
54
Download
0
Embed Size (px)
DESCRIPTION
Martin J. Pickering, Simon Garrod , Behavioral and Brain Sciences, 2012. An Integrated Theory of Language Production and Comprehension. Computer Science & Engineering 2012-20835 Sang-Woo Lee. Background - Aphasia. Broca’s Aphasia. Expressive aphasia Agrammatic aphasia - PowerPoint PPT Presentation
Citation preview
Martin J. Pickering, Simon Garrod, Behavioral and Brain Sciences, 2012.
An Integrated Theory of Language Production and Comprehension
Computer Science & Engineering2012-20835
Sang-Woo Lee
Background - Aphasia
3
Expressive aphasia Agrammatic aphasia Understand what other people say, but cannot speak the sen-
tence well. Caused by damage to, or developmental issues in the anterior
regions of the brain Including (but not limited to) the Broca’s area
Broca’s Aphasia
4
Also known as Receptive Aphasia Fluent apahsia, or sensory aphasia Speak the sentence fluently, but not well-organized sense in
their speech Traditionally associated with neurological damage to Wernicke’s
area in the brain (Actually it is not just simply associated to Wernicke’s area in current exper-
imental result, but anyway…)
Wernicke’s Aphasia
5
There are modules which specify some function
The “classical Lichtheim-Broca-Wernicke” Model
perceptionaction
6
Background - Aphasia Traditional independence of production and comprehension Interweaving in action and action perception
Perception process in action Predict next action of other Joint Action
Interweaving in Production and Comprehension Comprehension process in Production Predict next speech of other Interactive Language
Professor’s Question
Contents
Traditional independence of
production and comprehension
8
Traditional model of communication Discrete stages
A produces, B comprehends B produces, A comprehends
9
Assumes “horizontal split” between pro-duction and comprehension Arrows-within-arrows indicate feedback (in in-
teractive accounts) But this feedback is internal to production or
comprehension It may involve “general knowledge” But production does not involve comprehension pro-
cesses And comprehension does not involve production pro-
cesses
Horizontal Split
10
Interlocutors are not static, as the traditional model as-sumes, but are “moving targets” performing a joint activitiy (Garrod & Pickering, 2009)
Example of predict in Behavioral Instance
11
Example of predict in Neuroscience
Big(neuter)
Painting (neuter)
Big (common)Bookcase (com-
mon)
(Pickering & Garrod, 2007)
große Gemälde großen Bücherschrank
12
Also, many experiments demonstrate ef -fects of one on the other Picture-word interference (Schriefers et al.,
1990) Word identification affected by externally con-
trolled cheek movement (Ito et al., 2009) And strongly overlapping neural circuits
for production and comprehension (e.g., Pulvermüller & Fadiga, 2010; Scott et al., 2009)
Other Counterexample
13
Prediction process could be naturally understood with pro-duction module
When they comprehend the utterance, they also use pro-duction model internally.
Result
- There is forward modelTo predict perception caused by their own ut-terance- Fast alert when you say something wrong
Interweaving in action and action perception
Perception process in action Predict next action of other Joint Action
Interweaving in action and action perception
• Close links between action and action percep-tion, e.g.– participants’ arm movements affected by observing
another person’s arm movements (Kilner et al., 2003)– And making hand movements can facilitate concur-
rent visual discrimination of deviant hand postures (Miall et al., 2006)
• Such links could have various purposes – Supporting overt imitation– facilitating memory or understanding (“postdictively”)
• But authors propose that they aid prediction of own and others’ actions, by use of a forward model– Based on computational neuroscience (Wolpert, 1997;
see Grush, 2004)
Forward modelling in action In our terms, the action command causes the action im-
plementer to move the hand and the perceptual imple-menter to construct the percept
And the efference copy causes the forward action model to generate the predicted hand movement and the forward perceptual model to construct the predicted percept
Forward Modeling in Action- Just act- Feel involved per-
cepts of your own acte.g. Own coordination infoFeeling of wind blow-ing to your armGravity info …
18
Efference Copy
Forward Modeling in Action- Predict perception caused by their own action
e.g. Own coordination infoFeeling of wind blowing to your armGravity info …
Prediction-by-Simulation
- Predict perception of other’s next actionby seeing other’s current action
Joint Action• People are highly adept at joint activities
(Sebanz et al., 2006). – ballroom dancing, playing a duet, carrying a
large object together– Precise timing is crucial
• To succeed, A predicts B’s action and B predicts B’s action
22
Joint Action
Interweaving in Production and Comprehension
Comprehension process in Production Predict next speech of other Interactive Language
Forward modeling in language production
• Action implementer production imple-menter
• Perceptual implementer comprehension implementer
• Action command production command– Drives the production implementer– Efference copy drives the forward models
• Comparator monitor – compares the utterance percept and the pre-
dicted utterance percept
Unifying production and comprehension
• Production and comprehension are inter-woven– Tight coupling in dialogue (Clark, 1996; Picker-
ing & Garrod, 2004)– Behavioural experiments show effects of com-
prehension processes on production and vice versa (e.g., Schriefers et al., 1990)
– Overlap of brain circuits for production and comprehension (e.g., Pulvermuller & Fadiga, 2010)
• Such interweaving facilitates prediction of self and other’s utterances
26
Classical modeling in language production
production command ti
Production implementer
Efference copy
Utterance
tphonsynsemp ,,
Forward production model
Comprehension implementer
Utterance percept tphonsynsemc ,,
Forward comprehension model
monitor
Predicted utterance
tphonsynsemp ,,ˆ
Predicted utterance percept tphonsynsemc ,,ˆ
- Just say utterance,- Listen what you say.
27
Forward modeling in language production
production command ti
Production implementer
Efference copy
Utterance
tphonsynsemp ,,
Forward production model
Comprehension implementer
Utterance percept tphonsynsemc ,,
Forward comprehension model
monitor
Predicted utterance
tphonsynsemp ,,ˆ
Predicted utterance percept tphonsynsemc ,,ˆ
- Predict perception caused by their own ut-terance- Fast alert when you say something wrong
Self-monitoring
Speaker wishes to say kite In the past, she has always constructed
the kite-concept and then uttered /k/ She therefore constructs forward model
p^[phon](t) = /k/ If she then incorrectly constructs p[phon]
= /g/, the monitor notices the mismatch If she believes the forward model, she will
detect an error (and perhaps reformulate) Otherwise, she will alter her forward model
29
Prediction-by-simulation
Efference copy
Forward production model
Comprehension implementer
Inverse model + context
Comprehension implementer
Forward comprehension model
monitor
Covert imitation
Person B
Person A
B’s utterance 1,, tphonsynsemp B
Derived production command
tiB
Predicted utterance
1,,ˆ tphonsynsemp B
Predicted utterance percept 1,.ˆ tphonsynsemc B
Utterance percept tphonsynsemc B,,
Utterance percept 1,, tphonsynsemc B
Overt Responses
Derived production command
1tiB
B’s utterance tphonsynsemp B,,
Derived intentional act of communication iA(t+1)
- Predict perception of other’s next utteranceby listening other’s cur-rent utterance
Big(neuter)
Painting (neuter)
30
Interactive Language
Joint action involves combining accounts of action and action perception
Similarly, interactive language involves combining ac-counts of production and comprehension Facilitates coordination (e.g., short intervals between speak-
ers; Wilson & Wilson, 2005) Facilitates alignment (developing same representations; Pick-
ering & Garrod, 2004) Alignment in turn facilitates comprehension (better prediction
of others)
31
Interactive Language
Conclusion We propose that language production and com-
prehension are interwoven It assumes a central role to prediction in produc-
tion, comprehension, and dialogue Speakers construct forward models to predict as-
pects of their upcoming utterances Listeners covertly imitate speakers and use for-
ward models to predict the speakers Our account helps explain the efficiency of pro-
duction and comprehension and the remarkable fluidity of dialogue
Thank you
Professor’s Question
Comprehension process in Production Predict next speech of other Interactive Language
35
Q1: Give the evidence for how language production and comprehension are tightly interwoven. How does this relate to the perception-action cycle theory of cognitive systems?
Question 1
36
Q2: Explain and give the evidence for how action, action perception, and joint action are interwoven. Explain how the authors use this to develop accounts of production, com-prehension, and interactive language.
Question 2
Action implementer pro-duction implementer
Perceptual implementer comprehension implementer
Action command produc-tion command
Comparator monitor
participants’ arm move-ments affected by ob-serving another person’s arm movements (Kilner et al., 2003)
And making hand move-ments can facilitate con-current visual discrimina-tion of deviant hand pos-tures (Miall et al., 2006)
37
Q3: Give examples of what behavioral and neuroscientific data on language processing can be explained by the integrated theory of language production and comprehension explains, while modular theory does not.
Question 3
- Behavioral data
- Neuroscientific data (Pick-ering & Garrod, 2007)