A study of Task-related writing A study of Task-related writing performance performance

and its measurementsand its measurements

Chen, Huiyuan Chen, Huiyuan

Yunnan University Yunnan University of Finance and Economics, of Finance and Economics,

Yunnan, ChinaYunnan, China<hychenew@gmail.com><hychenew@gmail.com>

The guiding research questionsThe guiding research questions

Whether different types of L2 writing tasks and conditions Whether different types of L2 writing tasks and conditions will affect the linguistic performance of Chinese EFL will affect the linguistic performance of Chinese EFL learners. And if so, in what way and why.learners. And if so, in what way and why.

What linguistic features and tendency can we find in What linguistic features and tendency can we find in learners’ task related performance?learners’ task related performance?

Can qualitative and quantitative method lend support to Can qualitative and quantitative method lend support to each other?each other?

The tasks to be studiedThe tasks to be studied

1.1. Topic writing (TW)Topic writing (TW)

2.2. Picture writing (PW)Picture writing (PW)

3.3. Graph writing (GW) Graph writing (GW)

4.4. Summary writing (SW) Summary writing (SW)

The control of time conditionThe control of time condition

30 minutes -- group A30 minutes -- group A 50 minutes -- group B50 minutes -- group B

Accuracy measuresAccuracy measures

The ratio of error-free s-nodes to the total number The ratio of error-free s-nodes to the total number of s-nodes of s-nodes (short for EFS/S or just EFS). (short for EFS/S or just EFS).

The number of errors in each T-unitThe number of errors in each T-unit (E/T). (E/T).

The percentage of the total number of errors to The percentage of the total number of errors to the total number of words in the written textthe total number of words in the written text (E/W). (E/W).

Complexity measuresComplexity measures

Syntactic complexitySyntactic complexity Ratio of S-nodes per T-unitRatio of S-nodes per T-unit (S/T) (S/T) The number of words per T-unitThe number of words per T-unit (W/T). (W/T).

Lexical VariationLexical Variation The ratio of different types of words to the total numbThe ratio of different types of words to the total numb

er of words or tokenser of words or tokens (type/token or t/t). (type/token or t/t). The number of different types of word divided by the sThe number of different types of word divided by the s

quare root of twice the total number of wordsquare root of twice the total number of words (t/√2 (t/√2W).W).

Fluency measuresFluency measures

The amount of revision to the total number The amount of revision to the total number of wordsof words (R/W). (R/W).

The number of words per minuteThe number of words per minute (W/M) (W/M)

The total number of words produced in a The total number of words produced in a written textwritten text (Length) (Length)

Statistical analysesStatistical analyses

A 3-way multivariate tests with the design of 4 ×2 ×A 3-way multivariate tests with the design of 4 ×2 ×2 on 10 measures: to get a general picture of the over2 on 10 measures: to get a general picture of the overall effect. The results in Table 1. all effect. The results in Table 1.

One-way ANOVA to find out the effect of task types aOne-way ANOVA to find out the effect of task types a

nd the results in Table 4.nd the results in Table 4.

Table 1. Summary of effect of all factors (SPSS)Table 1. Summary of effect of all factors (SPSS)

Effect

Value

(Wilks’ Lambda)

F-value

p. <

Main effect

Proficiency .944 1.32 .227

Time .745 7.62 .000

Tasks .108 24.74 .000

2-way interaction

Proficiency by Time .904 2.37 .014

Proficiency by Tasks .831 1.42 .079

Time by Tasks .829 1.43 .072

3-way interaction

Proficiency by Time by Tasks .919 .718 .852

Table 3. Time effect on the nine dependent variables Table 3. Time effect on the nine dependent variables (t-test, Statistica)(t-test, Statistica)

Variables Time 1

(Mean)

Time 2

(Mean) t-value df p. <

Accuracy

EFS .653 .699 -2.686 222 .008**

E/T .600 .536 1.971 222 .050*

E/W .046 .042 2.046 222 .042*

Complexity

S/T 1.755 1.781 -0.526 222 .600

W/T 13.151 12.882 0.651 222 .516

t/t .460 .435 2.177 222 .031*

t/√ 2W 4.781 4.925 -1.189 222 .236

Fluency

R/W .053 .070 -3.221 222 .001**

W/M

Length 228.321 269.125 -5.349 222 .000**

Table 4. Task effect on measures of accuracy, Table 4. Task effect on measures of accuracy, complexity and fluencycomplexity and fluency

(one-way ANOVA, SPSS) (one-way ANOVA, SPSS)

TW

(N=56)

means

PW

(N=56)

means

GW

(N=56)

means

SW

(N=56)

means

F-values

p. <

Accuracy

EFS/S 0.662 0.709 0.562 0.770 39.472 .000

E/T 0.645 0.524 0.627 0.474 6.953 .000

E/W 0.053 0.047 0.044 0.033 16.302 .000

Complexity

S/T 1.782 1.694 1.555 2.042 21.900 .000

W/T 12.170 11.174 14.548 14.173 19.205 .000

t/t .471 .477 .339 .503 104.578 .000

t/√ 2W 5.342 5.411 3.588 5.071 156.327 .000

Fluency

R/W .058 .060 .076 .052 3.705 .01

W/M 7.0497 7.0554 6.3097 5.5717 9.076 .000

Length 272.571 268.732 241.482 212.107 16.621 .000

Table 5. Means of the 10 measures across four tasks Table 5. Means of the 10 measures across four tasks

(SPSS)(SPSS)

TW

PW

GW

SW

EFS/S .6652 .7086 .5611 .7705

E/T .6453 .5243 .627 .4743

E/W .0529 .0469 .0437 .0332

S/T 1.781 1.6935 1.5547 2.0417

W/T 12.1701 11.1733 14.5473 14.1730

t/t .471 .477 .339 .504

t/√ 2W 5.342 5.412 3.588 5.071

R/W .0575 .0589 .0757 .0516

W/M 7.0497 7.0554 6.3097 5.5717

Length 272.571 268.732 241.482 212.107

Note: The red indicates the highest means; the green indicate the lowest. However, the tasks

belonging to the same subset by the Duncan’s test will have the same color.

Summary of the quantitative resultsSummary of the quantitative results

For the aspect of accuracy, SW and PW were ranked higher For the aspect of accuracy, SW and PW were ranked higher than the other two tasksthan the other two tasks

For the aspect of syntactic complexity ( in terms of S/T), SW For the aspect of syntactic complexity ( in terms of S/T), SW ranked first and GW the leastranked first and GW the least

For the aspect of lexical complexity, PW and TW ranked For the aspect of lexical complexity, PW and TW ranked higher than SW and GWhigher than SW and GW

For the aspect of fluency (in terms of length and the For the aspect of fluency (in terms of length and the production of words in time), TW and PW ranked higher production of words in time), TW and PW ranked higher than GW and SW. than GW and SW.

In terms of the amount of revision, GW ranked the highestIn terms of the amount of revision, GW ranked the highest

Linguistic features typical of each taskLinguistic features typical of each task

TWTWmore unclear, ambiguous or more unclear, ambiguous or

awkward structures and awkward structures and expressions expressions

more “global errors”more “global errors”

1. The local government used not to care for it, and now make it be serious. (Awkward and

unclear) (LA S5 TWi)

2. IF we can live safely in the world, we can’t do other things very well. (Unclear) (LB S4 TW)

3. My meaning is that water from all kinds of chemical factories should be made no dirty again

and again. (Awkward and ambiguous) (LA S7 TW) (the writer is aware that the expression

may not be clear, so “My meaning is …” (very strong trace of NL)

4. Only did we pay more attention to the pollution and escape pollution do our best. (Awkward)

(LA S8 TW)

5. I suggest anyone who find any factory is illegal to tell the government. (Awkward) (LB S3

TW).

6. … the pollution have been kept up with it, … (awkward) (HB S4 TW) i The abbreviation “L” stands for "Lower proficiency”, and “H” stands for higher proficiency. The

Letter “A” indicates time constrained group (i.e. time A or time A1 group). Similarly, B means less

time constrained group (time B or time 2 group)". ‘S” indicate the subject, so "S14" signifies the

student who was numbered 14 in that group. Thus, for example: abbreviations “HA S4 TW”

means this abstract was taken from a participant in the "higher proficiency", "time constrained"

group, subject number 14’s TW writing. And, so LA S5 TW means it is taken from Lower

proficiency, time constrained group, subject number 5’ TW writing.

PWPW

Oral /speech formOral /speech form

Exclamation as means of Exclamation as means of transiting ideastransiting ideas

PersonificationPersonification

5. Why? Human is the producer (of pollution) (LA S14 PW)

6. What is he doing? He is dragging a big “Jin” out of the sea. (LA S12 PW)

7. (At the beginning of the essay) Have you noticed that the natural environment around us

become more and more terrible? (LB S10 PW)

8. What happened to the man? Because he ate unhealthy fish which was got from the polluted

river. (LB S4 PW)

9. Oh, terrible, when the fish is landing, people is standing at the edge of the land and say, “No,

you fish, come back to your family …” (HA S9 PW)

GWGW13 he investigators measured the content of cadmium in water in three districts in a city.

They have found that in the south district of the city the content of cadmium in the

drinking water is 0.003 ppm and in the water for irrigation is 0.33 ppm. In this district,

there are 38.71% children whose physical build and growth are normal. 33.71%

children are thin and 22.58% children are stout. In the north district, the content of

cadmium in water is less than that in the south district. In drinking water it is only

0.0115 ppm. There are 46.69% children who are at normal state. Only 11.36% children

are short and fat. Campared with the south and north districts, the content of cadmium

in water in the central district is the lowest. In drinking water, it is only 0.0071 ppm.

However, there are more children whose physical build and growth are normal.

10.27% children are stout, less than that in the south and north districts. … (HB S2

GW)

SW: use of strategies:SW: use of strategies:

CombinationCombination

13 in the United States, some rivers are polluted by manufacturing industries water with

chemicals and with heat. (Combination or piecing together the content of three sentences in

the original text. See Appendix I for the original text. Paragraph 1) (LA S1)

14 In the United States, manufacturing industries cause much of the water pollution. (Combining

the original two sentences.) (LA S5)

Conclusion Conclusion

Task difference can give rise to different Task difference can give rise to different performance in different ways, as a results of task performance in different ways, as a results of task requirement and use of strategies pertinent to requirement and use of strategies pertinent to taskstasks

Both quantitative and qualitative approaches are Both quantitative and qualitative approaches are necessary and can be used to add validity to the necessary and can be used to add validity to the research results as the results of one method research results as the results of one method may find support in the other may find support in the other

ReferencesReferences Arthur, B. (1979). Short-term changes in EFL composition skills. In C. Yorio, K. PerkArthur, B. (1979). Short-term changes in EFL composition skills. In C. Yorio, K. Perk

ins, & J. Schachter (Eds.), ins, & J. Schachter (Eds.), On TESOL ’79: The learners in focusOn TESOL ’79: The learners in focus (pp. 330-342). Was (pp. 330-342). Washington, D.C.: TESOL.hington, D.C.: TESOL.

Bardovi-Harlig, K. (1992). A second look at T-unit analysis: Reconstructing the senBardovi-Harlig, K. (1992). A second look at T-unit analysis: Reconstructing the sentence. tence. TESOL QuarterlyTESOL Quarterly, 26/2: 390-395., 26/2: 390-395.

Carroll, J. B. (1967). On sampling from a lognormal model of word-frequency distrCarroll, J. B. (1967). On sampling from a lognormal model of word-frequency distribution. In H. Hucera & W. N. Francis (Eds.), ibution. In H. Hucera & W. N. Francis (Eds.), Computational Analysis of Present-daComputational Analysis of Present-day American Englishy American English (pp. 406-424). Providence, RI: Brown University. (pp. 406-424). Providence, RI: Brown University.

Crookes, G. (1990). The utterance, and other basic units for second language discCrookes, G. (1990). The utterance, and other basic units for second language discourse analysis. ourse analysis. Applied Linguistics, Applied Linguistics, 11/2: 183-199. 11/2: 183-199.

Cumming, A. & Mellow, D. (1996). An investigation into the validity of written indicCumming, A. & Mellow, D. (1996). An investigation into the validity of written indicators second language proficiency. In A. Cumming & R. Berwick (Eds.), ators second language proficiency. In A. Cumming & R. Berwick (Eds.), Validation iValidation in Language Testingn Language Testing (pp. 72-93). Clevedon, England: Multilingual Matters. (pp. 72-93). Clevedon, England: Multilingual Matters.

Flahive, D. E. & Snow, B. G. (1980). Measures of syntactic complexity in evaluating Flahive, D. E. & Snow, B. G. (1980). Measures of syntactic complexity in evaluating ESL compositions. In J. W. Oller & K. Perkins (Eds.), ESL compositions. In J. W. Oller & K. Perkins (Eds.), Research in Language TestingResearch in Language Testing (pp. 171-176). Rowley, MA: Newbury House. (pp. 171-176). Rowley, MA: Newbury House.

Gass, S.; Mackey, A.; Alvarez-Torres, M. J. & Fernadez-Garcia, M. (1999). The effect Gass, S.; Mackey, A.; Alvarez-Torres, M. J. & Fernadez-Garcia, M. (1999). The effect of task repetition on language output. of task repetition on language output. Language LearningLanguage Learning, 49/4: 549-581., 49/4: 549-581.

References (cont.)References (cont.) Homburg, T. J. (1984). Holistic evaluation of ESL compositions: Can it be validated objectively? Homburg, T. J. (1984). Holistic evaluation of ESL compositions: Can it be validated objectively? TESOL QuartTESOL Quart

erly,erly, 18: 87-107. 18: 87-107. Hunt, K. (1965). Hunt, K. (1965). Grammatical Structures Written at Three Grade Levels.Grammatical Structures Written at Three Grade Levels. Champaign, IL: The National Council Champaign, IL: The National Council

of Teachers of English. of Teachers of English. Johns, A. M. & Mayes, P. (1990). An analysis of summary protocols of university ESL students. Johns, A. M. & Mayes, P. (1990). An analysis of summary protocols of university ESL students. Applied LinguisApplied Linguis

ticstics, 11/3: 253-271., 11/3: 253-271. Lennon, P. (1991). Error: Some problems of definition, identification, and distinction. Lennon, P. (1991). Error: Some problems of definition, identification, and distinction. Applied LinguisticsApplied Linguistics, 12/, 12/

2: 180-196.2: 180-196. Mehnert, U. (1998). The effects of different lengths of time for planning on second language performance. Mehnert, U. (1998). The effects of different lengths of time for planning on second language performance. StuStu

dies in Second Language Acquisitiondies in Second Language Acquisition, 20/1: 83-108., 20/1: 83-108. Ojima, M. (2006). Concept mapping as pre-task planning: A case study of three Japanese ESL writers. Ojima, M. (2006). Concept mapping as pre-task planning: A case study of three Japanese ESL writers. SystemSystem, ,

34/4: 566-585.34/4: 566-585. Ortega, L. (1999). Planning and focus on form in L2 oral performance. Ortega, L. (1999). Planning and focus on form in L2 oral performance. Studies in Second Acquisition,Studies in Second Acquisition, 21: 109- 21: 109-

148.148. Ortega, L. (1999). Planning and focus on form in L2 oral performance. Ortega, L. (1999). Planning and focus on form in L2 oral performance. Studies in Second Acquisition,Studies in Second Acquisition, 21: 109- 21: 109-

148.148. Reid, J. (1990). Responding to different topic types: A quantitative analysis from a contrastive rhetoric perspReid, J. (1990). Responding to different topic types: A quantitative analysis from a contrastive rhetoric persp

ective. In B. Kroll (Ed.), ective. In B. Kroll (Ed.), Second Language Writing: Research Insights for the ClassroomSecond Language Writing: Research Insights for the Classroom (pp. 191-210). Cambri (pp. 191-210). Cambridge: Cambridge University Press.dge: Cambridge University Press.

Rifkin, B. & Roberts, F. D. (1995). Error gravity: A critical review of research design. Rifkin, B. & Roberts, F. D. (1995). Error gravity: A critical review of research design. Language Learning, Language Learning, 45/3: 545/3: 511-537.11-537.

Robinson, P. (1995). Task complexity and second language narrative discourse. Robinson, P. (1995). Task complexity and second language narrative discourse. Language LearningLanguage Learning, 45/1: 99-, 45/1: 99-140.140.

References (cont.)References (cont.) Sasaki, M. & Hirose, K. (1996). Explanatory variables for EFL students’ expository Sasaki, M. & Hirose, K. (1996). Explanatory variables for EFL students’ expository

writing. writing. Language LearningLanguage Learning, 46/1: 137-174., 46/1: 137-174. Skehan, P. (1996). A framework for the implementation of Task-based Instruction. Skehan, P. (1996). A framework for the implementation of Task-based Instruction.

Applied Linguistics,Applied Linguistics, 17/1: 38-62. 17/1: 38-62. Skehan, P. (1998). Task-based instruction. Skehan, P. (1998). Task-based instruction. Annual Review of Applied LinguisticsAnnual Review of Applied Linguistics, 1, 1

8:268-286.8:268-286. Taylor, G. (1986). Errors and explanations. Taylor, G. (1986). Errors and explanations. Applied Linguistics,Applied Linguistics, 7/2: 144-166. 7/2: 144-166. Wigglesworth, G. (1997). An investigation of planning time and proficiency level oWigglesworth, G. (1997). An investigation of planning time and proficiency level o

n oral test discourse. n oral test discourse. Language TestingLanguage Testing, 14: 85-106., 14: 85-106. Wolfe-Quintero, K.; Inagaki, S. & Kim, H. (1998). Wolfe-Quintero, K.; Inagaki, S. & Kim, H. (1998). Second Language Development in Second Language Development in

Writing: Measures of Fluency, Accuracy & ComplexityWriting: Measures of Fluency, Accuracy & Complexity. Technical Report #17. Unive. Technical Report #17. University of Hawai’i at Manroa: Second Language Teaching & Curriculum Center.rsity of Hawai’i at Manroa: Second Language Teaching & Curriculum Center.

Wu, H. (2006). Investigating the effect of time restraint on EFL writing. Wu, H. (2006). Investigating the effect of time restraint on EFL writing. Foreign LanForeign Language Teaching and Researchguage Teaching and Research, 38/1: 37-43. (Beijing, China) , 38/1: 37-43. (Beijing, China)

Xu, Youzhi. (2005). Xu, Youzhi. (2005). English StylisticsEnglish Stylistics. Beijing: Higher Education Press.. Beijing: Higher Education Press. Zhang, S. (1987). Cognitive complexity and written production in English as a secoZhang, S. (1987). Cognitive complexity and written production in English as a seco

nd language. nd language. Language Learning, Language Learning, 37/4: 469-481.37/4: 469-481.