Development of grammatical carefulness paal2014

Development ofGrammatical CarefulnessGrammatical Carefulnessin English as a Foreign

Language:A Comparison among University, High School, and

Junior High School Students in Japan

at PAAL20142014/8/19

Waseda University, Japan

Outline

• What is Grammatical Carefulness?

• Grammatical Carefulness Scale • Grammatical Carefulness Scale (GCS)

• Research Questions

• The Present Study

• Results & Discussion• Results & Discussion

• Conclusion

Kunihiro KUSANAGI

Junya FUKUTAGraduate School, Nagoya University

JSPS Research Fellow

Contact to:[email protected]

Yusaku KAWAGUCHI

Yu TAMURA

Aki GOTO

Akari KURITAAkari KURITA

Daisuke MUROTAGraduate School, Nagoya University

What isGrammatical Carefulness?

What is Grammatical Carefulness?

• L2 learners’ grammatical performanceperformance– Variant

– Inconsistent

– Complex

– Situation-dependent

• Depends on What?• Depends on What?– Proficiency

– Task-related factors

– Individual differences


• Speed-Accuracy Trade-Off (Dennis & Evans, 1996; Goldhammer & Kroehne, 2014; van der Linden, (Dennis & Evans, 1996; Goldhammer & Kroehne, 2014; van der Linden, 2007, 2009）

– Accurate when one can take enough time

– Less accurate when one is speeded

→→→→This may explain a large part of the variance!

Speed Accuracy Trade-Off and Individual Differences (Kusanagi et al, 2014)A

ccur

acy

The slower, the more accurate

Speed

Acc

urac

y

Acc

urac

y

Less accurate when

speeded

Speed

Acc

urac

y

SpeedThe functions change


• Inter-Learner Variance in Speed-Accuracy Trade-OffSpeed-Accuracy Trade-Off– Steady within a learner

– May be strongly linked to some psychological, behavioral, and meta-cognitive factors

• Types of motivation, aptitude, beliefs, strategies, • Types of motivation, aptitude, beliefs, strategies, confidence and anxiety…

(cf. Karcheva & Amaar, 2014; Kormos, 1996, Loewen et al., 2009)


• Grammatical Carefulness–Proposed by Kusanagi et al. (2014)–Proposed by Kusanagi et al. (2014)

–A construct in a psychometric sense

• by definition refers to a behavioral, psychological, and meta-cognitive characteristic of language learners, characteristic of language learners, which typically entails highly controlled, cautious, analytical, and sometimes time-consuming language use.

Grammatical Carefulness Scale


• GCS, Japanese Version (GCSJ)

• A questionnaire• A questionnaire

– Kusanagi et al. (2014)

– Composed of 14 items

– Written in Japanese

– Validated using factor analyses with – Validated using factor analyses with a large sample size of Japanese English learners (n < 2000)


• Pragmatic Carefulness

– k = 5– k = 5– e.g., I always think about the appropriateness of the expression.

• Lexical-Syntactic Carefulness

– k = 4– e.g., I always notice the mistakes of the spelling of words.

• Phonological Carefulness

– k = 5– e.g., I always listen carefully to the pronunciations of others.


• Factorial validity supported– By both exploratory and confirmatory FAs– By both exploratory and confirmatory FAs

• Showed high reliability (almost α = .90) for each subscale

• Content validity supported– In total 10 linguists judged that the items actually

referred to the aspects (pragmatic, lexical-syntactic, and phonological) in linguistic terms

• Criterion-based validity supported• Criterion-based validity supported– Correlated to “analytical beliefs” (Tanaka & Ellis, 2003) more

strongly than “empirical beliefs”

– All the subscales successfully predicted the accuracy of GJTs and a C-test and time to complete the C-test

Research Research QuestionsQuestions

Research Questions

• The present study attempts to reveal…reveal…– The developmental processes of grammatical

carefulness of Japanese learners of English as a foreign language

– How is grammatical carefulness of learners different among junior high school, high-school, different among junior high school, high-school, and university students?

– What types of learners are there in terms of their grammatical carefulness?

The Present The Present StudyStudy

The Present Study

• Data correction

– Participants（n = 2,269）– Participants（n = 2,269）– Shared with Kusanagi et al. (2014)

• 11 universities（n = 850）

• 2 high schools（n = 1,158）

• 2 junior high schools （n = 261）

– Composition of the questionnaire– Composition of the questionnaire

• Face Sheet

• The items of GJSJ (k = 15)

– Seven-point Rikert scales

The Present Study

• Data analysis– Basically, we compared the mean scale – Basically, we compared the mean scale

points of three types of carefulness among junior high, high school, university students

• Descriptive statistics for the scale points of individuals

• Visualizations • Standardized mean differences (d)• Standardized mean differences (d)• Confidence interval estimations for means and

ds• Didn’t employ statistical hypothesis testing

– Cluster Analyses

Results &Results &DiscussionDiscussion

Before Comparisons…

Reliability Pragmatic Lexical-Syntactic Phonological

Junior High 0.87 0.86 0.91High 0.85 0.88 0.93Univ. 0.87 0.89 0.92Univ. 0.87 0.89 0.92

P

1 3 5 7

0.74

13

57

0.72

57

LS0.66

Junior High School Students

P

1 3 5 7

0.76

13

57

0.68

57

LS0.64

High School Students

P

1 3 5

0.76

13

57

0.64

5 LS0.59

University Students

13 0.66

1 3 5 7 1 3 5 7

13

57

PH

13 0.64

1 3 5 7 1 3 5 7

13

57

PH

13 0.59

1 3 5 7 1 3 5 7

13

57

PH




P

1 3 5 7

0.74

13

57

0.72

57

LS0.66


P

1 3 5 7

0.76

13

57

0.68

57

LS0.64


P

1 3 5

0.76

13

57

0.64

5 LS0.59

University Students

13 0.66

1 3 5 7 1 3 5 7

13

57

PH

13 0.64

1 3 5 7 1 3 5 7

13

57

PH

13 0.59

1 3 5 7 1 3 5 7

13

57

PH




P

1 3 5 7

0.74

13

57

0.72

57

LS0.66


P

1 3 5 7

0.76

13

57

0.68

57

LS0.64


P

1 3 5

0.76

13

57

0.64

5 LS0.59

University Students

13 0.66

1 3 5 7 1 3 5 7

13

57

PH

13 0.64

1 3 5 7 1 3 5 7

13

57

PH

13 0.59

1 3 5 7 1 3 5 7

13

57

PH

Comparisons

PLS

PH

Junior High School Students (n = 261)

Sub

scal

es

6

7

Pragmatic

Lexical-Syntactic

Phonological1 2 3 4 5 6 7

Score

PLS

PH

1 2 3 4 5 6 7

High School Students (n = 1158)

Sub

scal

es

1

2

3

4

5

Sca

le P

oin

t

Phonological

Score

PLS

PH

1 2 3 4 5 6 7

University Students (n = 850)

Score

Sub

scal

es

1

Junior

High

High Univ. Junior

High

High Univ. Junior

High

High Univ.

Groups

Comparisons

PLS

PH


Sub

scal

es

6

7

Pragmatic

Lexical-Syntactic

Phonological1 2 3 4 5 6 7

Score

PLS

PH

1 2 3 4 5 6 7


Sub

scal

es

1

2

3

4

5

Sca

le P

oin

t

Phonological

Score

PLS

PH

1 2 3 4 5 6 7


Score

Sub

scal

es

1

Junior

High

High Univ. Junior

High

High Univ. Junior

High

High Univ.

Groups

Comparisons

d [CI] Pragmatic Lexical-Syntactic Phonological

Junior High - High 0.21 [0.07, 0.35] 0.25 [0.11, 0.39] 0.29 [0.15, 0.43]

Junior High - Univ. 0.44 [0.30, 0.58] 0.48 [0.34, 0.62] 0.42 [0.28, 0.56]

High - Univ. 0.26 [0.17, 0.35] 0.26 [0.17, 0.35] 0.16 [0.07, 0.25]High - Univ. 0.26 [0.17, 0.35] 0.26 [0.17, 0.35] 0.16 [0.07, 0.25]

0.0

0.2

0.4

Pragmatic

y HighJunior High Univ.

0.0

0.2

0.4

Lexical-Syntacticy

Univ.High

Junior High. 0.0

0.2

0.4

Phonological

y

Univ.Junior High.

-0.4 -0.2 0.0 0.2 0.4

-0.4

-0.2

x

Junior High Univ.

-0.4 -0.2 0.0 0.2 0.4

-0.4

-0.2

x

High

-0.4 -0.2 0.0 0.2 0.4

-0.4

-0.2

x

Univ.High

Junior High.

Clustering

6

7

Junior High 1

2

3

4

5

Me

an

Sco

re

Junior High 1

Junior High 2

Junior High 3

High 1

High 2

High 3

Univ. 1

Univ. 2

1

2

P LS PH P LS PH P LS PH

Subscale

Univ. 2

Univ. 3

Results and Discussion

• Summary– Japanese EFL learners tend to become – Japanese EFL learners tend to become

less careful for grammar

– Rather, learners are careful for grammar at the beginning of their language acquisition

– There may be three types of the learners: – There may be three types of the learners:

• Very careful, especially in phonology

• Middle level of carefulness

• Generally Less careful

Results and Discussion

• Grammatical Carefulness, Speed-Accuracy Trade-Off, and L2 Accuracy Trade-Off, and L2 Development– Grammatical carefulness is one of the inter-

learner variance

– However, generally, there is a tendency of learners to become less careful, parallel to the learners to become less careful, parallel to the language development

• Skill-Acquisition theory

• Explicit and implicit knowledge

• Automatization and resrtucturing

Model

Acc

urac

y

Speed

Conclusion

• Limitations and Future Studies

– This study was cross-sectional, – This study was cross-sectional, longitudinal ones will be strongly desired

– More on relationships between performance and grammatical carefulnesscarefulness

– Further validations of the scale

– Networking with other learners’ psychological and behavioral factors

References --1

• Dennis, I., & Evans, J. St. B. T. (1996). The speed-error trade-off problem in

psychometric testing. British Journal of Psychology, 87, 105–129.

• Goldhammer, F., & Kroehne, U. (2014). Controlling individuals’ time spent

on task in speeded performance measures: Experimental time limits,

posterior time limits, and response time Modeling. Applied Psychological

Measurement, 38, 255–267.

• Kartchava, E., & Ammar, A. (2014). Learners' Beliefs as Mediators of What Is

Noticed and Learned in the Language Classroom. TESOL Quarterly, 48, 86-

109.

Kormos, J. (1999). The effect of speaker variables on the self-correction • Kormos, J. (1999). The effect of speaker variables on the self-correction

behaviour of L2 learners. System, 27, 207–221.

• Loewen, S., Li, S., Fei, F., Thompson, A., Nakatsukasa, K., Ahn, S., & Chen, X.

(2009). Second language learners’ beliefs about grammar instruction and

error correction. Modern Language Journal, 93, 91–104.

References --2

• Tanaka, K., & Ellis, R. (2003). Study abroad, language proficiency, and learner

beliefs about language learning. JALT journal, 25, 63-85.

• van der Linden, W. J. (2007). A hierarchical framework for modeling speed

and accuracy on test items. Psychometrika, 73, 287–308.

• van der Linden, W. J. (2009). Conceptual issues in response-time modeling.

Journal of Educational Measurement, 46, 247–272.

Development ofGrammatical Carefulness

in English as a Foreign Language:A Comparison among University, High School, and Junior High School Students in Japan

Contact to:

Kunihiro KUSANAGI

Junya FUKUTAGraduate School, Nagoya University

JSPS Research Fellow

Contact to:[email protected]

PLS

PH

1 2 3 4 5 6 7


Score

Sub

scal

es

4

5

6

7

Me

an

Sco

re

Junior High 1

Junior High 2

Junior High 3

High 1

High 2

Yusaku KAWAGUCHI

Yu TAMURA

Aki GOTO

Akari KURITA

Daisuke MUROTAGraduate School, Nagoya University

PLS

PH

1 2 3 4 5 6 7


Score

Sub

scal

es

PLS

PH

1 2 3 4 5 6 7


Score

Sub

scal

es

1

2

3

P LS PH P LS PH P LS PH

Me

an

Sco

re

Subscale

High 3

Univ. 1

Univ. 2

Univ. 3

Speed

Acc

ura

cy

-0.4 -0.2 0.0 0.2 0.4

-0.4

-0.2

0.0

0.2

0.4

Lexical-Syntactic

x

y

Univ.High

Junior High.