Brain + Multilingualism / Language Learning

Factors that matter

in Second Language Acquisition –

a Cognitive Neuroscience Approach to SLA

Ass. Prof. Susanne Reiterer, PhD

Dr.rer.nat. (PhD in Psychology) – Univ of Vienna, Austria

Mag.phil.(MA in Linguistics) - Univ of Vienna

University of Vienna & Tübingen (formerly)

Centre for Language Learning Learning and Teaching Research (FDZ ) – at

the Faculty for Philology and Cultural Studies, University Vienna

Language is for everyone ..

Factors for growth & development in L2

L1

age

..exposure

use..

soil

Plant type

dandelion (engl)

“pissenlit“ (fr)

Löwenzahn (ger)

wind

rain

sun

Individual variation !

Where do these abilities stem from? - theories

Mezzofanti

Kató Lomb

• God

• Brain structure &

function

• DNA/ Gene

(80% genes in linguistic functions

compared to g. IQ (K.

Stromswold

FOXP2, KE family;

JR family –semantics,

Briscoe2012 )

• “they work harder“

• Motivation

• exercise

• having much time

• everybody can do it

• musicality

• working memory / hearing

• personality (extraversion)

• empathy

Behaviorism Nativism

handedness

Individual Differences (Factors)

Brain structure

Attention

Extraversion (personality)

Openness

Conscientiousness

Emotional Intelligence / Empathy

Anxiety

Attitudes

Teaching method

Nature (genes, brain) aptitude? giftedness

Modifying factors Nurture (environment) practice

Second Language Learning Process

Language Aptitude

Motivation

Working Memory

DNA sex/gender

testosterone level

Exposure Time

Quality of training

Typology of language (linguistic similarity/distance)

Learning strategy

Purpose of language use

Age of Onset of L2

Age of Onset of L2

Age of acquisition versus proficiency level ?

Auditory input

Motor output

Bilingual (L2) Brain representation?

the big fight !

Factors for L1 and L2 attainment/proficiency

Brain (brain structure, lateralization, maturation)

DNA genetic equipment („developmental language disorders“…)

Sex (female linguistic advantage?)

Handedness (lateralization and language, left handers)

Age (Age of Onset) (as a consequence of brain maturation, deterioration) Bio

log

y

Intelligence/ Verb Intell (biol? social?)

Aptitude, language learning talent (biol? psychol? social?)

Domain general cognition / Executive Functions / Cognitive Control

Affective factors (Anxiety..)

Memory / Working Memory (biol?)

Learning strategies

Learner‘s Type (inborn? acquired?)

Personality (biol? cultural? psychol?)

Psych

olo

gy/

Co

gn

itiv

e

So

cia

l

Acquisition / Teaching Method

Quality of Input

Purpose of Language Use/ linguistic environment

Exposure Time

Language Attitudes

Polyglottism (L2, L3, L4… number of)

Bidialectalism (early experience of)

Amount / Quality of Training

Language typology Lin

gu

isti

c /

so

cio

lin

gu

isti

c

Language Learning Aptitude

• Definitions

• prediction of how well, relative to other individuals, an individual can

learn a foreign language in a given amount of time (speed and

accuracy are important) [John B Carroll, Charles Stansfield]

• As with many measures of aptitude, language learning aptitude is

thought to be relatively stable throughout an individual’s lifetime and

per definition there is a certain “given”, pre-existing element of

giftedness to it. [Niklas Abrahamsson]

what is this?

http://www.youtube.com/watch?v=FOiXtWcQ8GI 16 year old American boy, 20 languages

Language and its many faces (levels)

Grammar (syntax,

morphology) Phonetics/phonology

Pragmatics

Speech imitation aptitude: a large scale study Linguistic part Psychological part Brain Imaging part

Linguistic description (~10

tests)

Proficiency & Talent measures

• Language aptitude test (MLAT by Carroll 1960), • detailed phonetic language

recordings to determine

pronunciation fluency and aptitude

(own tests)

* 2 TOEFL (list comp, grammar)

• language learning questionnaires (history)

• psycholing. Questions (learn style, type of instruction, contacts in L2)

N=138

Psychological testing

(~15 Tests)

• Phonological working memory

• 2 Empathy tests

• 2 Personality questionnaires (Big

Five and Inhibition-Activation Sys)

* Verbal & non-verbal IQ (Raven)

• 2 Musicality (Gordon’s AMMA)

plus questions abt music hobbies

• Mental Flexibility (Simon task)

N~116

Brain Imaging

(3 types of measures taken)

• structure

MR anatomy – (mdeft)

VBM (voxel based morphometry – white / grey matter density)

• DTI (diffusion tensor imaging) (diff 12 dir)

• function fMRI (EPI, TR 2s; +Tr 12s)

N~70

How were the recordings done?

1. part: phonetic-

linguistic testing:

138 participants were recorded in (Uni Tübingen&Stuttgart) a soundproof room on 1

h speech production and 1 h speech perception.

Production tasks: English (L2): read texts, tell story, free speech, imitate sentences.

German (L1): imitate sentences (prosody), speak foreign-accented sentences in L1.

Hindi (L0): imitate sentences and words (no prior experience).

Perception tasks: English and German: disriminate sentences in diff prosody, Toefl test, listening comp,

detect foreign accented speech …

2 online native speaker ratings:

1. For English (L2): www.susannereiterer.eu/eval we need ~ 250 natives from all world (mainly US & GB) to make short online ratings (30 min) – work in progress!

2. For Hindi (L0): www.susannereiterer.eu/eval_hindi we had ~ 30 natives from all India (mainly Hindi speakers) to make long online ratings (2 h) – work completed!

Hindi score

for “pure“

imitation talent

without the

influence of

experience

English score

for “experience

plus ability“

- fluency (perform)

- pronunciation (Tal)

Sample in database:

138 Germans +

18 Native Indians

(to confuse the raters)

Sample in database:

138 Germans +

14 Native English

(to confuse the raters)

How to rate speech output? Imitation talent

native model (10) 7.2 / 6.4 / 4.6 / 2.2 / 0.6 / 0.1

native range 8-10

Rating

system:

10 =

highest score

0 =

lowest score

Highest score for German speakers for this sample: 7.2 138 German subjects+18 natives speakers=156 /

Sentence1: 8.8; Sentence3: 8.8 ! 30 (different) native raters (15f/ 15m).

Males were rated sig. better than females (by males and females)

(males mean: 4,9 points females 4,4; t-test: sig. 0.005

Ratings in internet were

performed for

3 Hindi sentences

and 1 word

Example here: sentence 2

(most difficult)

HINDI

Behavioral results: Hindi imitation scores

T – test f independent samples:

Males: 4.9 (SD±1.0)

Females: 4.4 (SD±0.9)

Sig: p= .005

normal distributions & linguistic theory

• Reconciling models/ theory

How to rate speech output? Pronunciation (ability)

Task 2: Reading of “The North Wind and the Sun”

The North Wind and the Sun

The North Wind and the Sun were disputing which was the stronger, when

a traveller came along wrapped in a warm cloak. They agreed that the

one who first succeeded in making the

traveller take his cloak off should be considered stronger than the other.

Then the North Wind blew as hard as he could, but the more he blew

the more closely did the traveler fold his cloak around him; and at last

the North Wind gave up the attempt.

Then the Sun shined out warmly, and immediately the traveller took off

his cloak. And so the North Wind was obliged to confess that the Sun

was the stronger of the two.

jbh10 / bfem10 / kgl5 / 6 / hbau2,2 / ahp 2

Rating system

10 = highest score

0 = lowest score

• 8 % (=11 subjects) of German speakers could

„cheat“ the native ears.

• They were all in the range between 8-10 points

• males rated significantly better than females by

American „raters“ (females).

• normally distributed

First results of native speaker rating English

Task 2: Reading of “The North Wind and the Sun”

How did we do the brain scans?

3. part

magnetic

resonance

imaging

MR earphones MR microphone

Button-press-device Drive-in starts!

First results - fMRI: high low ability (Hindi) groups

Preliminary results: fMRI: sentence imitation

Group versus group contrasts:

Nontalents versus Talents = Low Talent Group

First results - fMRI: sentence + word imitation (Hindi)

Results – structural MRI (VBM) – anatomy only (Hindi)

Voxel based morphometry measurement: gray matter volume

Reiterer SM, Hu X, Erb M, Rota G, Nardo D, Grodd W, Winkler S and Ackermann H. (2011) Individual differences in audio-

vocal speech imitation aptitude in late bilinguals: functional neuro-imaging and brain morphology. Frontiers in Psychology ,

2:271. (free access journal)

Hu X, Erb M, Ackermann H, Martin JA, Grodd W, Reiterer SM (2011) Voxel-Based Morphometry Studies of Personality: Issue

of Statistical Model Specification - Effect of Nuisance Covariates. NeuroImage, 54:1994-2005.

Dogil, G. and SM Reiterer (eds) (2009) Language Talent and Brain Activity. Mouton de Gruyter: Berlin, New York.

Talents

Non Talents

0

50

100

150

200

250

300

350

400

450

500

read_ger read_eng read_acc

Are

a u

pto

10

0 H

z

Talents

Non-Talents

Temporal Modulations (Hz)

Spectr

al M

odula

tions (

Cycle

s/K

Hz)

* p=0.002 * p=0.002 * p= <0.001

Results – acoustic spectrograms: sentence reading, faking accents

Read German (A) Read English (B) German with fake Engl Accent ( C)

r=0.7**

Reiterer S, Singh NC, Winkler S. (2012) Predicting speech imitation ability biometrically.(In: Empirical Approaches to Linguistic Theory

Studies in Meaning and Structure. Edited by Stolterfoht, Britta and Featherston, Sam. Mouton de Gruyter: Berlin, New York).

Results - fMRI: sentence reading and faking accents

Reading German with foreign accent

Reading tasks L1, L2, L1Acc

Increase BOLD activity in peak activation area

High ability group Low ability group

High ability group Low ability group

Low - High

L1 Germ

Low - High

L2 Engl

Low - High

L1 + ACC

N=9 (2 groups, high ability l low ability)

p<0.05, corrected at cluster level

L1 + Accent:

Increasing activation the lower

the scores

Differences low versus high ability group:

significant correlations with low ability

C

C

C

A

B

A B C C B A

Reiterer S, Singh NC, Winkler S. (2012) Predicting speech imitation ability biometrically.(In: Empirical Approaches to Linguistic Theory

Studies in Meaning and Structure. Edited by Stolterfoht, Britta and Featherston, Sam. Mouton de Gruyter: Berlin, New York).

Results – phonetic measurements: vowel duration measures

agreed high ability

low ability

Words: along, alike, agree, alive, asleep, …

r=0.7**

the shorter the vowel – the better the imitation score

High ability L0 (Hindi) speech imitators:

• better Work Memory (digit, nonword) 0.37***

• higher scores MLAT 0.34***

• English imitation 0.33***

• can “sing“ better 0.3**

• higher fun-seekingness (BAS_fun) 0.22*

• less conscientiousness -0.28*

preliminary behavioral results: correlations with language and

psychological measures (work in progress)

Individuals with high pronunciation aptitude in L2

(English) are also good in:

• higher scores MLAT 0.6***

• like “acting“ more 0.36***

• higher “openness to experience“ 0.33***

• more school years in L2 0.34***

• time spent abroad 0.33***

• speak more dialects in L1 0.3**

• higher musicality (Gordon test) 0.29**

• higher empathy 0.28*

• like singing more 0.28*

Hindi score English score

*** = .0001 / ** = .001 / *= .01

uncorrected for multiple comparisons

0.33***

• Subjects: n=113, age = 26 ± 6, from 19 to 40 years, onset =10

• Dependent Variable: Hindi Pronunciation Aptitude (imitation of Hindi sentences)

• Independent Variables: Verbal IQ, Non-Verbal IQ, Vocabulary Short-Term Memory, Non-Verbal Short-Term Memory (WAIS), Phonetic Coding Ability, Musicality, Neuroticism, Extraversion, Openness to Experience, Agreeableness, Conscientiousness, Empathy

• Results: Only Working Memory (digit span and nonword repetition) together “singing” (“like singing and can sing”) significantly predicted the Hindi Pronunciation Aptitude (Hindi score).

behavioural results (Hindi) – multiple regression

• Subjects: n=113, age = 26 ± 6, from 19 to 40 years, onset =10

• Dependent Variable: English Pronunciation Aptitude (imitation of Hindi sentences)

• Independent Variables: Verbal IQ, Non-Verbal IQ, Vocabulary Short-Term Memory, Non-Verbal Short-Term Memory (WAIS), Phonetic Coding Ability, Musicality, Neuroticism, Extraversion, Openness to Experience, Agreeableness, Conscientiousness, Empathy

Results: Only Phonetic Coding Ability (MLAT) and Empathy and Singing

together significantly predicted the English Pronunciation Aptitude. No significant interaction between phonetic coding ability and empathy.

=> No working memory influence in the late stage.

behavioural results (English) – multiple regression

• Xiaochen Hu, Hermann Ackermann, Jason Martin, Michael Erb, Susanne Winkler and Susanne M Reiterer. (2013)

Language Aptitude for Pronunciation in Advanced Second Language (L2) Learners: Behavioural Predictors and

Neural Substrates. Brain and Language: in press.

musicality (singing) and language learning?

sound learning / articulate sound imitation / flexibility in sound imitation

Original song (composition) lower score singer high score singer

Ongoing diploma-thesis Markus Christiner

(www.christiner.at), English Department, University of Vienna

(supervisors Prof. E. Keizer, Ass Prof. S. Reiterer)

“Happy Birthday“ (creativity)

Preliminary results:

high correlations between singing and pronouncing

(imitating) - and working memory,

lower with general musicality and

not with instrument playing , not with amount of singing

lessons

Acknowledgements

My sincere thanks to present and past collaboration partners, students and supervisors:

Ackermann, Hermann – University of Tübingen, Hertie Institute, Germany

Anus, Arev (project student) – University of Stuttgart, Germany

Baumotte, Henrike (PhD student) – University of Stuttgart, Germany

Bhattacharya, Joydeep - Goldsmith College, London; Academy of Sciences, Vienna

Christiner, Markus – (diploma student) – University of Vienna

Erb, Michael – University of Tübingen, CIN, Germany

Dogil, Grzegorz - University of Stuttgart, Germany

Grodd, Wolfgang – University of Tübingen, Germany

Hu, Xiaochen (PhD student) – University of Tübingen and Bonn Germany

Klose, Uwe – University of Tübingen, Germany

Kumar, Vinod – University of Tübingen, Germany

Nardo, Davide (project student) – Santa Lucia Foundation, Rome, Italy

Meier, Jens – University of Tübingen, Germany

Rappelsberger, Peter - Medical University of Vienna, Austria

Rota, Giuseppina (project student)- University of Pisa, Italy & Stuttgart, Germany

Singh, Nandini, Chatterjee – National Brain Research Centre (NBRC), Gurgaon, INDIA

Wildgruber, Dirk – University of Tübingen, Germany

Winkler, Susanne – University of Tübingen, Germany

book and papers

Reiterer, S. (2009). Brain and Language Talent: A Synopsis. In: Dogil, G. & Reiterer,

S. (eds.) Language Talent and Brain Activity. Trends in applied linguistics 1, Mouton

de Gruyter, Berlin-New York: 155-191.

• Hu X and Reiterer S (2009). Personality and pronunciation talent in second

language acquisition. In: Dogil, G. & Reiterer, S. (eds.) Language Talent and Brain

Activity. Trends in applied linguistics 1, Mouton de Gruyter, Berlin-New York: 97-129.

• Rota G and Reiterer S (2009). Cognitive aspects of pronunciation talent: how

empathy, mental flexibility, working memory and intelligence interact with phonetic

talent. In: Dogil, G.& Reiterer, S. (eds.) Language Talent and Brain Activity. Trends in

applied linguistics 1,

Mouton de Gruyter, Berlin-New York: 67-96.

• Nardo D and Reiterer S (2009). Musicality and phonetic language aptitude. In:

Dogil, G. & Reiterer, S. (eds.) Language Talent and Brain Activity. Trends in applied

linguistics 1, Mouton de Gruyter, Berlin-New York: 213-255.

• Xiaochen Hu, Hermann Ackermann, Jason Martin, Michael Erb, Susanne Winkler and Susanne M Reiterer.

(2013) Language Aptitude for Pronunciation in Advanced Second Language (L2) Learners: Behavioural

Predictors and Neural Substrates. Brain and Language: in press.

• Reiterer S, Singh NC, Winkler S. (2012) Predicting speech imitation ability biometrically.(In: Empirical

Approaches to Linguistic Theory Studies in Meaning and Structure. Edited by Stolterfoht, Britta and

Featherston, Sam. Mouton de Gruyter: Berlin, New York).

• Reiterer S, Hu X, Erb M, Rota G, Nardo D, Grodd W, Winkler S, Ackermann H. (2011)

Individual differences in audio-vocal speech imitation aptitude in late bilinguals: functional neuro-imaging and

brain morphology. Frontiers in Psychology (Language Sciences) 2:271, 1-12.

• Hu X, Erb M, Ackermann H, Martin JA, Grodd W, Reiterer SM (2011)

Voxel-Based Morphometry Studies of Personality: Issue of Statistical Model Specification - Effect of Nuisance

Covariates. NeuroImage, 54: 1994-2005.

Thank You!

Prelim. Behavioral Results1: Correlations with Lang measures

Females

Like

dancing! sig

Pronunciation Talents are also good in:

• proficiency measures (grammar, etc.)

• can speak more dialects in their L1!

• like to imitate dialects in L1

• like singing more! (and can sing better)

• have more contacts with natives

• time spent abroad (~ only MLAT)

• no of L2s spoken ( ~ only MLAT syntax)

Prelim. Behavioral Results2: Correlations with Psych tests

Byproduct:

sig. other Correlations with Sex:

• Females - higher empathy scores

• Females - higher neuroticism scores

• Females – higher agreeableness sc.

• Females – higher conscientiousness

• Females – higher inhibition scores !

Acc our tal measures, pron. talents

have:

• better Work Memory (digit, nonword)

• better Musicality (Rhythm)

• would be more open, fun-seeking

and less conscientious!

• have slightly more empathy

• more nonverbal IQ (acc. to the MLAT

only ! (beware of known correlations

between aptitude tests and IQ tests)

How to rate pronunciation (ability) – the

„foreign accent test“

(B) Mr. Richardson led his dog to a tree. (L2)

(A) Der Bauer erntete jetzt viel Getreide. (L1)

Task (3 conditions): Read sentences in A German/ B English/ C German with fake foreign accent

(C) Der Professor präsentiert das Resultat. (L1Acc)

(C) Der Bademeister rettete den Jungen.

(C) Der Beamte genießt seine Ferien.

High ability speakers:

MB

(A)

(B)

(C)

Low ability speakers:

AK

mid-range speakers:

SK

fMRI Paradigm What do the people do in the scanner? fMRI paradigm

Different tasks have to be completed inside the scanner. Speech production was recorded online during scanning.

A. Speech production (articulation, microphone recorded) (3 Tasks, 20 / 20 / 30 min)

1. WORD IMITATION (listen to and repeat WORDS in TAMIL (L0), English (Brit+American) (L2).

Ex: distributor, legislature, synthesizer, transparency / Tamil: pustagangal, patirigei, arasamgam.

2. SENTENCE IMITATION (listen to and repeat Sentences in English (Br+Am) (L2) and German (L1).

Ex: The teacher welcomed the student to her class./ Am: The engineer constructed the bridge with care.

Der Rettungsdienst brachte ihn ins Krankenhaus. Der Winter überraschte die Eichhörnchen.

3. SENTENCE READING (read sentences in (A) L1-German (B) L2-English and (C) German with fake English foreign

accent (LAcc)

Ex.: (A) Der Bauer erntete jetzt viel Getreide (B) Mr. Richardson led his dog to a tree (C) Der Professor präsentiert das

Resultat

First results - fMRI: sentence imitation (English score)

Main Effect Group: High > Low Aptitude Main Effect Language: English > German

Sentence Imitation

IFG STG, Ins

MR

SFG PC, SMA

BG

CB

STG HG IFG

Ins IFG Ins

SMA, ACC MCC

CB

BG

Comparison: High> Low Aptitude

A1: Sentence Imitation

STG

SFG

IFG

MR

Imitation

A2

Imitation

A3

Imitation

A4

First results - fMRI: sentence imitation (English score)

First results - fMRI: sentence + word imitation (Hindi)

• age range: 20-40 (mean: 25.9, ± 5.2)

balanced for:

• age of Onset English (AOL): around 10 years (in school) (late learners)

• L1: 138 German native speakers, studying in Tübingen or Stuttgart

• education: All completed „a-levels“, university students, young graduates/academics

• linguistic experience~ 50% language students (mostly English students)

(distributed over all „talent“ groups: high-middle-low)

• 90% right handed (in MR experiments – 100% right handed)

• No cases of neurological disorders

Who were the participants in our study?