Profiling lexical diversity in college level writing

Profiling Lexical Diversity in College-Level Writing

Melanie C. González. Ph.D.

SSLW 2017

Bangkok, Thailand

July 2, 2017

The intersection of writing and vocabulary

Writing draws heavily on authors’ productivelexical faculties and has the potential to expose gaps in vocabulary knowledge

Lexical diversity is a strong indicator of writing proficiency and sophisticated word use (Crossley & Mcnamara, 2009; Friginal, Li, & Weigle, 2014; Johnson, Acevedo, & Mercado, 2013; Yu, 2009)

Lexical diversity: is it a question of teaching more words? Which words?

Lexical frequency profiles can guide instructors and learners towards more efficient vocabulary learning (Crossley, Cobb, & McNamara, 2013; Gardner & Davies, 2013; Laufer & Nation, 1995; Nation & Waring, 1997)

Lexical frequency profiles may reveal possible gaps in productive lexicon that impact lexical diversity (Laufer, 1994)

Scarce but conflicting evidence that that lexical frequency does not always correlate well to lexical diversity (Johnson, Mercado, & Acevedo, 2013; Laufer, 1994)

The intersection of writing and vocabulary

Purpose: To explore lexical frequency in relation to lexical diversity in college writing

RESEARCH QUESTION 1:• How do the lexical frequency profiles of advanced

ML writers’ academic compositions compare to those of their MES peers?

RESEARCH QUESTION 2:• What frequency level(s) is a significant

contributor to lexical diversity in academic compositions?

Research questions

119 final drafts of narrative essays collected: 2universities (both public universities, one in western U.S., other in the northeastern U.S.)

Mean token count = 618.78

MES (n=65); ML (n=54)

L1s: Amharic (n=1); Arabic (n=1); Chinese (any variety) (n=21); English (n=64); French (n=1); Hmong (n=3); Hindi (any variety) (n=2); Indonesian (n=1); Japanese (n=2); Kannada (n=1); Korean (n=2); Portuguese (n=1); Russian (n=1); Spanish (n=10); Tagalog (n=1); Vietnamese (n=6)

Methods: Corpus

Lexical frequency: BNC-COCA 25 (Lextutor; Cobb, n.d.)

Lexical frequency categories (Schmitt & Schmitt, 2014):• 1K-3K = high-frequency terms• 4K-8K = mid-frequency terms• 9K+ = low-frequency terms

Lexical diversity: MTLD (McCarthy & Jarvis, 2010; Coh Metrix; Graesser et al., 2004)

Methods: Variables

Desig. M SD

High-freq. MES 468.55 8.26

(1K-3K) ML 487.17 8.29

Mid-freq. MES 14.45 6.01

(4K-8K) ML 7.09 6.34

Low-freq. MES 3.20 2.14

(9K+) ML 2.40 3.66

MTLD MES 79.95 17.34

ML 69.54 12.89

Results

ML writers used more high-frequency terms (F2,117=54.13, p<.00)

MES writers used more mid-frequency words (F2,117=15.12, p<.00)

No statistical difference found in terms of either groups’ use of low-frequency terms

MES writers texts exhibited greater lexical diversity (F2,117=5.06, p<.05)

Results:RQ 1

The total variance explained by the model as a whole was 26.8% (F2,117= 4.75, p < .05)

Regression model correctly classified 105 of the 119 essays

Mid-frequency vocabulary was the only significant predictor of lexical diversity

As lexical diversity increases, there was also an increase in the use of mid-frequency terms (beta = .93, p < .05)

Results:RQ 2

The total variance explained by the model as a whole was 26.8% (F2,117= 4.75, p < .05)

Regression model correctly classified 105 of the 119 essays

Mid-frequency vocabulary was the only significant predictor of lexical diversity

As lexical diversity increases, there was also an increase in the use of mid-frequency terms (beta = .93, p < .05)

Discussion

Excerpt 1: MES writer

Writing is an important tool to have in life and can be applicable to various occupations. Writing is a way of formulating your thoughts and observations on paper with no requirement on the subject matter. In the past I have been exposed to multiple writing experiences that required that I be clear and concise in my diction. I have had some encouraging and frustrating experiences, but overall I am determined to master the art of composition.

MTLD: 95.46

Discussion

Excerpt 2: ML writer

I believe I have the potential of being a good writer. I could not speak or understand Englishwhen I entered school in the US, but I was determined to become fluent in this new language. I was previously nervous about writing, but as I had many writing opportunities, I continued to improve each time. Who I am as a person describes who I am as a writer. I have always been a determined type and that can show in my writing.

MTLD: 87.95

The finding that ML writers use more high-frequency terms than their MES peers and that there is no difference between their use of low-frequency items is in line with precedent research (see Crossley & McNamara, 2009; 2012; Meara & Bell, 2001 )

MES writers employed double the number of mid-frequency terms than ML writers - a possible gap in ML writers’ lexicons?

During editing and peer-editing exercises, have students identify overly repeated words and phrases

Teach how to use tools like a thesaurus, synonym feature in MS Word, etc.

Target mid-frequency vocabulary terms (synonyms, hypernyms) or phrases that add variety to the text

Conclusions &

applications

• Small-scale study (119 essays) – need more, longer essays for examination; variety of academic genres

• Genre was limited to narrative papers and topic of “Myself as a Writer”; first-year writing

• BNC-COCA corpus (word family vs. lemma)

• Look toward other indices that may impact lexical diversity such as measures of cohesion, tease out “lexical sophistication”

Limitations

• Include independent measures of general receptive and productive vocabulary size

• Explore more detailed measures relating to synonymy, hypernymy, polysemy, and lexical density - control for function words? Proper nouns?

• Include qualitative data: instructor perceptions; student writer interviews/think alouds

Next steps

References

Booth, P. (2014). The variance of lexical diversity profiles and its relationship to learning style. International Review of Applied Linguistics in Language Teaching, 52(4), 357-375.

Cobb, T. (n.d.). Compleat Lexical Tutor [Computer Program]. Retrieved from http://www.lextutor.caCrossley, S. & Cobb, T. & McNamara, D. (2013). Comparing count-based and band-based indices of word

frequency: Implications for active vocabulary research and pedagogical applications. System, 41(4), 965-981.

Crossley, S.A., & McNamara, D.S. (2009). Computational assessment of lexical differences in L1 and L2 writing. Journal of Second Language Writing, 18(2), 119-135.

Friginal, E., Li, M., Weigle, S.C. (2014). Revisiting multiple profiles of learner compositions: A comparison of highly rated NS and NNS essays. Journal of Second Language Writing, 23(1), 1-16.

Gardner, D., & Davies, M. (2014). A new academic vocabulary list. Applied Linguistics, 35(3), 305-327. doi:10.1093/applin/amt015

Graesser, A. C., McNamara, D. S., Louwerse, M. M.,&Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavioral Research Methods, Instruments, and Computers, 36, 193–202.

Johnson, M., Acevedo, A., & Mercado, L. (2013). What vocabulary should we teach?: Lexical frequency profiles and lexical diversity in second language writing. Writing & Pedagogy, 5(1), 82-103. doi: 10.1558/wap.v4i5.1

Laufer, B., & Nation, P. (1995). Vocabulary size and use: Lexical richness in L2 written production. Applied Linguistics, 16(3), 307-322.

McCarthy, P.M., & Jarvis, S. (2010). MTLD, voc-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42, 381-392. doi: 10.3758/BRM.42.2.381

Schmitt, N., & Schmitt, D. (2012). A reassessment of frequency and vocabulary size in L2 vocabulary teaching. Language Teaching, 47(4), 484-503.

Yu, G. (2009). Lexical diversity in writing and speaking task performances. Applied Linguistics, 31(2), 236-259.

Thank you!Melanie C. González

mgonzalez@salemstate.edu

This research and presentation was funded in part by Salem State University's School of Graduate Studies and the Emilio and Mary DiFeliceEndowment for Research in Education.