Embed Size (px)
Citation preview
Indicating verbs as ‘multimodal’ constructions
Adam Schembri Kearsy Cormier Jordan Fenlon
Overview
® Recent findings about indicating verbs from corpus-based studies
® Implications ® Theoretical framework for understanding
these constructions
Indicating verbs
® A class of verbs that move in space between, or are directed towards, locations associated with, for example, agent and/or patient arguments
® Found in vast majority of sign languages studied to date ® In the work reported here, we focus on a subset of indicating
verbs by excluding those that involve a locative argument (the subset also known as ‘agreement verbs’)
BSL GIVE3>1 ‘She gives me’
Indicating verbs
® Directional verbs: a class of indicating verbs that move in space between locations associated with, for example, agent and patient arguments
® Found in vast majority of sign languages studied to date ® In the work reported here, we focus on a subset of indicating
verbs by excluding those that involve a locative argument (the subset also known as ‘agreement verbs’)
BSL GIVE3>1 ‘She gives me’
How to analyse these
verbs? A number of different
views
Our view: Fusion of lexical signs and pointing
® These verbs point to their present referents or locations associated with their absent referents (Liddell, 2003; de Beuzeville et al. 2009), as if the referents were present
® Directionality thus controlled by language external factors, such as the current location of a referent
® Grammatical system, but modification not obligatory
Modification of indicating verbs ® Previously assumed that if an indicating verb can be modified, it
must (i.e. obligatory “agreement”) ® E.g. John established on left, GIVE must be directed to left
® But studies based on large datasets have shown such modification is clearly NOT obligatory ® de Beuzeville et al. (2009) for Auslan ® Fenlon, Schembri & Cormier (in prep) for BSL
® Both Auslan and BSL studies additionally found that verbs modified for patient/object were more likely to be produced with constructed action than without ® Cormier, Fenlon & Schembri (2015)
BSL Corpus
1680 tokens of indications verbs in free conversation from 100 signers (4 regions) in the
BSL Corpus (Schembri et al., 2010)
Coded for
® Coded for agent and patient modification according to: ® Person, number, animacy and co-reference ® Person to person modification combinations (1st to 2nd,
3rd to 3rd etc.) ® Presence/absence of constructed action ® Path of movement (e.g., body-sagittal/-diagonal, side-
to-side) ® Verb position in the clause ® Lexical frequency (using objective frequency measures
from Fenlon et al., 2014) ® Social factors: gender, age, region, language
background, ethnicity ® Included lexical items and participants as a
random effect in a mixed effect model
Rate of modification
differs from citation form looks like citation form but matches spatial location of referent (i.e., example of
syncretism)
Does not differ from citation form (typically first to second person)
UNMODIFIED MODIFIED CONGRUENT
Patient
Rate of modification
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Agent (1220 tokens) Patient (1470 tokens)
Ambiguous Unmodified Congruent Modified
29% 50%
37%
32% 26%
14%
Rate of modification
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Ambiguous Unmodified Congruent Modified
29% 50%
37%
32% 26%
14%
Modification appears to
be optional for both
Factors underlying actor modification
® Verb position in the clause: verb only (81%) and verb final position (82%) favours modification over other positions (61%; p=<.0001)
® Person: 1st strongly favours modification over 3rd and 2nd person referents (93% over 54%, 52% respectively: p=<.0001)
® Person to person: Non-1st to 1st favoured modification over 1st to non-1st and non-1st to non-1st (p=<.0001)
® Signer’s language background: signers with hearing parents use modification for agent more often than those with deaf parents (75% over 66%; p=<.001)
Factors underlying patient modification
® Constructed action ® Overt CA is more likely to occur with modification than no CA (p= <.001)
® Suggests that signers are pointing to imagined referents
® Person ® Second and first person occurs with modification
more than third person (p =<.01) ® Reflects a distinction between present and non-present
referents
Factors underlying patient modification
® Animacy ® Animate arguments occurs with more modification
than inanimate arguments (p=<.01) ® Coreference
® Coreference with a noun, null argument, pronoun occurs with modification more than no coreference (p=<.001) ® Suggests a reference tracking system
Factors underlying patient modification
® Person to Person ® Non-1st to 1st favoured modification over 1st to
non-1st and non-1st to non-1st (p=<.0001) ® Verb position in the clause
® verb only and verb final position favours modification over other positions (p=<.001)
Implications
✓ Directionality and modification as a pointing-based reference tracking system, drawing on mental space blends as discussed by Liddell (2003) ✗ Agreement analysis
® But how are we to understand these results?
On indicating verbs as ‘multimodal’ constructions
® Critics of the Liddellian account: how do we account for specific lexical and grammatical properties that are unexpected if the sign is a fusion of a morpheme and pointing (e.g., Meier, 2002)?
® Specific idiosyncratic properties of some indicating verbs – e.g. ® GIVE2>1 in BSL towards the signer’s chest ® REMIND2>1 touches the shoulder ® LOOK2>1 towards the face
Our proposal
® Indicating verbs represent a kind of composite construction of lexical sign and pointing (Liddell, 2003)
® Analogous to speech+gesture multimodal motion constructions in Zima (2014)
® Similar to proposals involving composite utterances of speech and gesture (Enfield, 2009) and their application to sign languages (Johnston, 2013)
Construction grammar
® Constructions are symbolic units or signs, that is, a pairing of form and meaning.
® A construction is the only unit of grammatical representation.
® There is a continuum from schematic complex constructions (corresponding to syntactic rules in other theories) to substantive atomic constructions, that is, words (corresponding to the lexicon in other theories).
Construction Grammar
® Constructions are organized in a network, chiefly by taxonomic relations and part-whole relations.
® In usage-based Construction Grammar, mental representation of a construction is determined not only by the (non)predictability of the constructional properties, but also by token and type frequency (Bybee 1985, 1995).
Speech+gesture multimodal constructions
® Zima (2014) examined four specific motion constructions and co-speech gestures accompanying them in Red Hen Lab Corpus ® 60.4% of 202 instances of [V(motion) in circles] co-
occurred with a gesture ® For other constructions, the co-occurrence with gesture
was even higher: ® [N spin around] 72% ® [zigzag] 71% ® [all the way from X PREP Y] 80.4%
Speech+gesture multimodal constructions
® Construction Grammar theory proposes two important factors that reflect both individual entrenchment and socio-cultural conventionalisation of constructions: 1. Recurrence 2. Idiosyncracy
Indicating verbs and multimodal constructions
® Recurrence ® Frequency of usage leads to perception of co-
occurrences as a relatively fixed combination of form and meaning which is stored in the memory as a unit
® Spoken English motion constructions from Zima (2014) occur with gesture 60-80% of the time in the corpus
® Indicating verbs occur with pointing less often (54% for patient arguments, 66% for agent) but are still the majority of uses
® Idiosyncracy ® Specific formal and/or semantic/pragmatic properties
come to be associated with constructions ® Sometimes not due to the properties of its components. ® Specific semantic uses of English motion constructions –
e.g. [all the way from X PREP Y] to refer to actual distance – further encourage use of co-speech gesture (86% of all instances) whereas temporal or metaphoric uses ranged from 56% down to 33.3%
® Indicating verbs in BSL also show idiosyncratic behaviour, with signs like PUSH the least likely to be modified (43%, 6/14) and PAY (100%, 12/12) the most likely (note we excluded signs that showed no modification at all from the analysis).
Indicating verbs and multimodal constructions
® Frequent combination of pointing and lexical sign in indicating verbs reflect: ® entrenchment in the minds of individual signers ® conventionalisation of these combinations in
signing communities ® Idiosyncratic properties of individual indicating
verbs (cf., Liddell, 2003) ® Together these properties match what would
be predicted in a Construction Grammar account
Indicating verbs and multimodal constructions
Indicating verbs
® [GIVEx>y] ® Handshape, orientation, movement: lexically
specified ® Initial and final location: variable
BSL GIVE3>1 ‘She gives me’
Conclusion
® In this comparison with spoken language multimodal constructions, the crucial difference for sign languages is that indicating verbs are unimodal (rather than multimodal) fusions of lexical items and pointing, making them a typologically unique linguistic phenomenon.
Ryan & Corrine: we need to talk!
Methodology: BSL Corpus Project
® To create an on-line, open-access collection of BSL digital video data ® 249 participants were filmed in 8 regions across the UK, balanced as much as
possible for gender, age, region, language background, ethnicity:
Directional verb study: Coding
® Each token coded for agent and patient modification according to: ® Person, person-to-person marking, number, animacy and co-
reference ® Verb position in clause, serial/double verb construction ® Presence/absence of constructed action ® Direction of movement ® Lexical frequency (using objective frequency measures from
Fenlon et al., 2014) ® Social factors: gender, age, region, language background,
ethnicity ® Included lexical items and participants as a random effect in a
mixed effect model
Factors underlying patient modification (1034 tokens)
Factor group
Factors Tokens % Log odds Weights
Constructed action
CA No CA
656 378
78% 59%
0.454 –0.454
0.61 0.39
Co-reference
Co-ref No co-ref
411 628
81% 65%
0.387 –0.387
0.60 0.40
Person/number
2sg/pl 1sg/pl 3sg/pl
57 242 735
81% 81% 70%
0.389 0.062 –0.451
0.60 0.52 0.40
Animacy Humans Other animates Inanimates
693 122 219
76% 71% 54%
0.326 0.173 –0.451
0.58 0.54 0.38
How did we decide modification?
Does the verb look like citation form?
Has spatial reference been set up?
Has spatial reference been set up?
Does the verb match the spatial location?
Does the verb match the spatial location?
YES
NO
YES YES
YES
NO
NO
NO
NO
YES
Modified
Unmodified without spatial reference
Congruent
Unmodified with spatial reference
Modified congruent
Modified incongruent
How did we decide presence/absence of CA?
Is gaze directed towards the initial or final location of the verb?
Are other non manuals active?
Is there CA?
Is there CA?
YES
NO
YES
YES
YES
NO
NO
NO
Eye gaze only
No CA
CA without eye gaze
CA with eye gaze and other articulators
No CA with eye gaze and linguistic non manuals
