37
Watching for Washback: Observing the Influence of the International English Language Testing System Academic Writing Test in the Classroom Anthony Green University of Cambridge ESOL Examinations Previous studies of washback (the influence of a test on teaching and learning) have provided insights into the complexity of educational systems and test use, especially in relation to the role of the teacher, but have given insufficient attention to the rela- tionship between observed practices and test design features. In this article a wash- back model is proposed that incorporates both test design and participant characteris- tics. The model is used to predict behaviour on preparation courses directed toward the Academic Writing component of the International English Language Testing System (IELTS) test. 197 learners and 20 teachers were observed over 51 classroom hours. These encompassed 22 IELTS preparation classes and, for comparison, 13 classes of English for academic purposes (EAP). Evidence was found for substantial areas of common practice between IELTS and other forms of EAP but also for some narrowing of focus in IELTS preparation classes that could be traced to test design features. Before offering a place to an international student, most universities in Eng- lish-speaking countries will require evidence of the student’s language ability. As increasing numbers of students choose to travel to access global educational op- portunities, there has been rapid growth in the use of language tests for this pur- pose. In the United Kingdom the most widely recognised test of English for aca- demic purposes (EAP) is the International English Language Testing System (IELTS). Between 1995 and 2005 the number of candidates rose from under 50,000 to over half a million per year (International English Language Testing LANGUAGE ASSESSMENT QUARTERLY, 3(4), 333–368 Copyright © 2006, Lawrence Erlbaum Associates, Inc. Correspondence should be addressed to Anthony Green, Validation Unit, University of Cambridge ESOL, 1 Hills Road, Cambridge, CB1 2EU, UK. E-mail: [email protected]

Watching for Washback_Article

Embed Size (px)

Citation preview

Page 1: Watching for Washback_Article

Watching for Washback: Observing theInfluence of the International EnglishLanguage Testing System Academic

Writing Test in the Classroom

Anthony GreenUniversity of Cambridge ESOL Examinations

Previous studies of washback (the influence of a test on teaching and learning) haveprovided insights into the complexity of educational systems and test use, especiallyin relation to the role of the teacher, but have given insufficient attention to the rela-tionship between observed practices and test design features. In this article a wash-back model is proposed that incorporates both test design and participant characteris-tics. The model is used to predict behaviour on preparation courses directed towardthe Academic Writing component of the International English Language TestingSystem (IELTS) test. 197 learners and 20 teachers were observed over 51 classroomhours. These encompassed 22 IELTS preparation classes and, for comparison, 13classes of English for academic purposes (EAP). Evidence was found for substantialareas of common practice between IELTS and other forms of EAP but also for somenarrowing of focus in IELTS preparation classes that could be traced to test designfeatures.

Before offering a place to an international student, most universities in Eng-lish-speaking countries will require evidence of the student’s language ability. Asincreasing numbers of students choose to travel to access global educational op-portunities, there has been rapid growth in the use of language tests for this pur-pose. In the United Kingdom the most widely recognised test of English for aca-demic purposes (EAP) is the International English Language Testing System(IELTS). Between 1995 and 2005 the number of candidates rose from under50,000 to over half a million per year (International English Language Testing

LANGUAGE ASSESSMENT QUARTERLY, 3(4), 333–368Copyright © 2006, Lawrence Erlbaum Associates, Inc.

Correspondence should be addressed to Anthony Green, Validation Unit, University of CambridgeESOL, 1 Hills Road, Cambridge, CB1 2EU, UK. E-mail: [email protected]

Page 2: Watching for Washback_Article

System, 2005). The rapid expansion of the test has brought with it increased de-mand for test preparation books and courses.

IELTS is used as a means of determining whether candidates should be ac-cepted into English-medium courses and whether they will require further lan-guage support. However, concern has been expressed that preparation for tests likeIELTS, because of the limitations on what can realistically and equitably be testedin a few hours, may not develop the full range of skills required for successful uni-versity study, particularly in the area of academic writing (Deakin, 1997; Read &Hirsh, 2005). J. Turner (2004), for example, argues that “what the IELTS test or theTOEFL test delivers underspecifies the complexity of language issues in the aca-demic context” (p. 98). Her concern is that education in academic literacy is beingsupplanted by training in test taking. But what influence does the IELTS writingtest really have on teaching, and how different are writing classes in IELTS prepa-ration courses from other forms of EAP?

To investigate these questions, this article compares the practices observed inwriting classes of two types: IELTS preparation classes directed at success on thetest and, as a suitable point of comparison, presessional EAP writing classes pro-vided by universities to prepare learners for academic study.

LITERATURE REVIEW

Washback studies, investigating the effects of tests on the teaching and learning di-rected toward them, have often involved direct observation of the behaviour ofteachers and learners in the classroom. The inclusion of an observational elementin such studies has been recommended as a means of contextualising, corroborat-ing, or correcting data from surveys and interviews (Alderson & Wall, 1993; C.Turner, 2001; Wall, 1996; Watanabe, 2004).

Table 1 summarises the methods and findings of recent case study investiga-tions of washback in language education that have included an observational ele-ment. These studies covered a wide range of educational contexts, with observa-tion either focussing on a small number of participants observed intensively over asustained period (Alderson & Hamp-Lyons, 1996; Read & Hayes, 2003) or on alighter sampling of classes to allow for observation of larger numbers of teachersand a broader perspective (Hawkey, 2006; Wall, 2005).

With the exception of Burrows (1998, 2004) and Hawkey (2006), all includedcomparisons between different types of class. Wall (1996, 2005) and Cheng (2005)focused on changes over time as a new test is introduced. This approach also in-formed work relating to the recent update of the Test of English as a Foreign Lan-guage (TOEFL; Hamp-Lyons, 2005; Wall & Horák, 2004). Alderson andHamp-Lyons (1996) and Watanabe (1996, 2004) compared the practices of teach-

334 GREEN

Page 3: Watching for Washback_Article

335

TAB

LE1

Stu

dies

ofW

ashb

ack

inLa

ngua

geE

duca

tion

Tha

tInc

lude

Obs

erva

tion

and

Inte

rvie

wD

ata

Stud

yE

xam

Inst

itut

ions

Obs

erva

tion

sN

and

Fre

quen

cyIn

terv

iew

sK

eyF

indi

ngs

Ald

erso

nan

dH

amp-

Lyo

ns(1

996)

Test

ofE

nglis

has

aFo

reig

nL

angu

age

(TO

EFL

)

Spec

ialis

edla

ngua

gein

stitu

tein

Uni

ted

Stat

es

Purp

ose-

desi

gned

inst

rum

ent,

2te

ache

rs

16cl

asse

s:8

gene

ralE

nglis

h,8

TO

EFL

prep

arat

ion

over

1w

eek

3fo

cus

grou

pin

terv

iew

sw

ith3–

12st

uden

ts;

9te

ache

rsin

grou

pan

din

divi

dual

sess

ions

TO

EFL

was

hbac

km

aybe

gene

rate

dm

ore

byte

ache

rs,m

ater

ials

,wri

ters

,an

dad

min

istr

ator

sth

anby

test

.“A

mou

nt”

and

“typ

e”of

was

hbac

kva

ryac

cord

ing

tote

stst

atus

,re

latio

nof

test

tono

ntes

tpr

actic

e,de

gree

tow

hich

teac

hers

and

mat

eria

lw

rite

rsth

ink

abou

tap

prop

riat

em

etho

dsan

dth

eir

will

ingn

ess

toin

nova

te.

Bro

wn

(199

8)In

tern

atio

nal

Eng

lish

Lan

guag

eTe

stin

gSy

stem

(IE

LTS)

Uni

vers

ity-b

ased

lang

uage

inst

itute

Rec

ords

ofin

stru

ctio

nal

focu

san

dm

ater

ials

,2

teac

hers

All

clas

ses

desi

gnat

edfo

rw

ritin

gov

er10

wee

ks;7

0ho

urs

inIE

LTS

and

30ho

urs

inno

n-IE

LTS

EA

Pco

urse

Not

repo

rted

IELT

Spr

epar

atio

nin

stru

ctio

ncl

osel

yfo

cuss

edon

the

test

with

regu

lar

timed

prac

tice

ofIE

LTS

task

s.IE

LTS

prep

arat

ion

was

mor

esu

cces

sful

than

EA

Pco

urse

inim

prov

ing

IELT

Ssc

ores

from

entr

yto

exit,

buts

mal

lsam

ples

(9IE

LTS,

5E

AP)

limit

inte

rpre

tatio

n.

(con

tinue

d)

Page 4: Watching for Washback_Article

336

Bur

row

s(1

998;

2004

)C

ertif

icat

esin

Spok

enan

dW

ritte

nE

nglis

h(C

SWE

)

Adu

ltE

nglis

hM

igra

ntE

nglis

hPr

ogra

mm

e(A

ME

P)in

Aus

tral

ia

CO

LT,4

teac

hers

Two

4-ho

urle

sson

spe

rte

ache

r30

teac

her

inte

rvie

ws—

incl

udin

gth

ose

obse

rved

(con

duct

edpr

ior

toth

eob

serv

atio

nph

ase)

Teac

hers

vary

inre

spon

ses

toin

nova

tions

inas

sess

men

tin

line

with

indi

vidu

aldi

ffer

ence

san

dex

peri

ence

.W

ithC

heng

(200

3)an

dW

all(

2005

),lin

ksw

ashb

ack

toth

eori

esof

chan

ge.

Che

ng(2

005)

Hon

gK

ong

Cer

tific

ate

Exa

min

atio

nsin

Eng

lish

(HK

CE

E)

3H

Kse

cond

ary

scho

ols

(mai

nst

udy)

Ada

ptat

ion

ofC

OLT

,3

teac

hers

3170

-min

clas

ses

c.6

mon

ths

befo

reex

amov

er2

year

s

Follo

w-u

pin

terv

iew

sw

ithob

serv

edte

ache

rs

Follo

win

gin

nova

tion,

teac

hers

enga

ged

inex

am-l

ike

activ

ities

e.g.

,ro

le-p

lays

,and

used

(con

serv

ativ

e)ex

ampr

epm

ater

ials

incl

ass,

but

belie

fsan

dat

titud

esto

lear

ning

did

notc

hang

e.D

iffe

renc

esin

how

teac

hers

cope

with

chan

ge.

Ext

ensi

vesu

ppor

treq

uire

din

impl

emen

tatio

nof

inno

vatio

ns.

Haw

key

(200

6)IE

LTS

10la

ngua

gesc

hool

sin

UK

,Ja

pan,

Cam

bodi

a

Cam

brid

geII

Sin

stru

men

t,10

teac

hers

10IE

LTS

prep

arat

ion

clas

ses

120

stud

ents

,21

teac

hers

,and

15re

ceiv

ing

inst

itutio

nad

min

istr

ator

sin

focu

sgr

oups

Teac

her

pref

eren

cefo

rta

sk-b

ased

,oft

enin

terr

elat

edm

acro

skill

sac

tiviti

es,i

nvol

ving

mic

rosk

ills

rele

vant

toIE

LTS.

Focu

son

IELT

Sbu

twill

ingn

ess

tous

ea

rang

eof

mat

eria

lsan

dm

etho

ds.

TAB

LE1

(Con

tinue

d)

Stud

yE

xam

Inst

itut

ions

Obs

erva

tion

sN

and

Fre

quen

cyIn

terv

iew

sK

eyF

indi

ngs

Page 5: Watching for Washback_Article

337

Rea

dan

dH

ayes

(200

3);H

ayes

and

Rea

d(2

004)

IELT

S2

lang

uage

scho

ols

inN

ewZ

eala

nd

CO

LTan

dC

ambr

idge

IIS

inst

rum

ent,

2te

ache

rs

22ho

urs

of32

-hou

rco

urse

and

28ho

urs

of32

0-ho

ur(8

-mon

th)

cour

se

23te

ache

rin

terv

iew

s;w

eekl

yin

terv

iew

sw

ithth

ete

ache

rsob

serv

ed

Dif

fere

ntia

ltea

cher

prac

tices

atle

astp

artia

llylin

ked

toin

stitu

tiona

lcon

text

.G

reat

erpr

essu

reto

“tea

chto

the

test

”in

priv

ate

lang

uage

scho

ols.

Wal

l(20

05)

SriL

anka

n‘O

’le

vel

c.50

seco

ndar

ysc

hool

sin

11ar

eas

ofSr

iL

anka

Purp

ose-

desi

gned

obse

rvat

ion

chec

klis

t

5le

sson

sob

serv

edov

er2

year

s(3

9–64

diff

eren

tcl

asse

sob

serv

edpe

rro

und)

atva

ryin

gpe

riod

sah

ead

ofex

am

64te

ache

rsin

focu

sgr

oups

;fo

llow

-up

inte

rvie

ws

with

each

clas

sob

serv

atio

n

Teac

hers

used

text

book

cont

entb

utfa

iled

tofo

llow

sugg

este

dm

etho

ds,e

.g.,

read

ing

for

gist

.Pr

opor

tion

ofcl

asse

sw

asde

dica

ted

toex

ampr

epar

atio

n.N

egle

ctof

spea

king

skill

str

acea

ble

toex

amco

nten

t.W

atan

abe

(199

6;20

04)

Var

ious

Japa

nese

univ

ersi

tyen

tran

ceex

amin

atio

ns

3hi

ghsc

hool

sin

Japa

nA

dapt

atio

nof

CO

LT,5

teac

hers

964

min

ofex

ampr

epar

atio

nan

d83

3m

inof

regu

lar

clas

ses

over

6m

onth

s

Teac

her

follo

w-u

pin

terv

iew

sw

ithea

chob

serv

atio

n

Teac

hers

vary

inth

eir

appr

oach

esto

exam

prep

arat

ion—

info

rmed

byat

titud

esto

war

dsth

eex

am.

Scho

olcu

lture

anim

port

antf

acto

r.M

ater

ial

desi

gned

for

exam

prep

arat

ion

may

som

etim

esbe

used

for

othe

rpu

rpos

es.

Page 6: Watching for Washback_Article

ers in test preparation and non-test-preparation classes, while Read and Hayes(2003; Hayes & Read, 2004) compared two approaches to test preparation.

Three of the observational washback studies listed in Table 1 investigatedIELTS preparation classrooms. Brown (1998) compared practices in two coursesprovided by the same institution: IELTS preparation and non-IELTS EAP. Readand Hayes (2003) compared two IELTS preparation courses at different institu-tions: one an intensive preparation course, the other combining IELTS preparationwith other forms of EAP. The ongoing IELTS Impact Study (IIS; Hawkey, 2006)also includes observational data of IELTS preparation classes.

These studies all found IELTS to affect behaviour. Brown (1998) found that stu-dents in the 10-week IELTS preparation course used IELTS preparation textbooks,completed one Task 1 and one Task 2 essay each week (and no other writing), per-formed three timed practice examinations, were informed about IELTS scoringcriteria, received feedback on the accuracy of their written work, and were in-structed in strategies for writing under timed conditions. In contrast, students in theEAP course worked on a 1,000-word project, did no timed writing, were instructedin strategies for writing in academic contexts, and were encouraged to develop re-search skills. Read and Hayes (2003) combined a broad survey with targeted class-room observation of two teachers to provide mutual corroboration of findings. Thebriefer and more intensive of the two IELTS classes they observed was morenarrowly concerned with the test and included more test practice under timedconditions.

In common with other washback studies, Hawkey (2006) found variation be-tween the 10 teachers he observed in how they conducted their classes, notably inthe number of opportunities they provided for learners to communicate together inEnglish. Teachers were willing to employ a variety of teaching methods and to usematerial both within and beyond the textbook. However, both the institutions pro-viding the courses and the students, who were motivated to succeed on the test, ap-peared to constrain teachers to focus their instruction on IELTS. In class, teachersshowed a preference for task-based activities, targeting microskills they believedto be relevant to the test.

Data from observational studies have informed insights into the complexity ofeducational systems and test use, especially in relation to the role of the teacher.However, a shortcoming identified by Bachman (2005) is the lack of a coherent ev-idential link between test design characteristics and the practices observed or re-ported in the classroom. In studies that involve predicting the effects of a test on in-struction or learning, appeal is more often made to the views of educationalauthorities (Cheng, 2005; Ferman, 2004; Qi, 2004), teachers (Banerjee, 1996), orwidely held public perceptions (Watanabe, 1996) than directly to the design of thetest instrument (as evidenced by available test materials or test specifications).How might the design of the IELTS Academic Writing component be expected toinfluence instruction?

338 GREEN

Page 7: Watching for Washback_Article

Watanabe(2004)suggested twosourcesofevidence thatmaybeused torelateob-servedpractices to the influenceofa test.One is evidence that testdesign featuresarereflected in teachingor learning.Theother is theabsenceof such features in teachingor learning not directed toward the test (or directed toward an alternative test). Of thethree IELTS-related studies, only Brown (1998), in common with Alderson andHamp-Lyons’s (1996) study of TOEFL preparation, incorporated a comparisonwith courses that were not directed toward the test. The inclusion of just two classesin each of the Australian studies limits their generalisability, while the lack of anontest comparison in Hawkey (2006) makes it difficult to disentangle test influencefrom teacher variables. This study involves observation of a larger number of teach-ers and learners than Brown (1998) and Read and Hayes (2003) but includes thecomparison with nontest EAP classes missing from Hawkey (2006).

In considering the mechanisms of washback, a growing body of theory relatestest design, test use, and classroom behaviours, although as Wall (2005) argued,too little is sufficiently informed by empirical evidence. Most of this work takesthe form of recommendations to test developers. Chapman and Snyder (2000) pro-vided a framework for relating tests to educational practices, and Brown (2000)cited Hughes (1989), Heyneman and Ransom (1990), Kellaghan and Greaney(1992), Bailey (1996), and Wall (1996) in identifying features of a test that may bemanipulated in efforts to improve instruction. These embrace both contexts for testuse and technical qualities of the test instrument.

Drawing together these two elements in washback theory, Green (2003) pro-posed the predictive model of test washback set out in Figure 1. The model startsfrom test design characteristics and related validity issues of construct representa-tion identified with washback by Messick (1996) and encapsulated in Resnick andResnick’s (1992) formulation of overlap, or the extent of congruence between testdesign and skills developed by a curriculum or required in a target language usedomain. Test design issues are most closely identified with the direction ofwashback—whether effects are likely to be judged beneficial or damaging toteaching and learning.

The model relates design issues to contexts of test use, including the extent towhich participants (including material writers, teachers, learners, and course pro-viders) are aware of and are equipped to address the demands of the test and arewilling to embrace beliefs about learning embodied therein. These features aremost closely related to washback variability (differences between participants inhow they are affected by a test) and washback intensity. Washback will be most in-tense—have the most powerful effects on teaching and learning behaviours—where participants see the test as challenging and the results as important (perhapsbecause they are associated with high stakes decisions, such as university entrance;Bailey, 1999; Cheng, 2005; Hughes, 1993; Watanabe, 2001).

A survey of IELTS preparation courses conducted in tandem with this study(Green, 2003) indicated that the test was regarded both as important and challeng-

WATCHING FOR WASHBACK 339

Page 8: Watching for Washback_Article

ing by a majority of learners in the IELTS preparation courses involved (70% ratedsuccess on the test as “very important”). It also indicated that almost all the learn-ers were taking the test because they intended to enter higher education in theUnited Kingdom. Most demonstrated at least a basic knowledge of the test formatand viewed the direct test of writing as a positive feature. In short, the conditionsfor intense washback to a majority of participants would seem to be in place.

Beyond a general prediction that the direct testing of writing in IELTS wouldencourage the teaching of writing in test preparation programs, the washbackmodel suggests that features of task design will impact on the nature of that in-struction. The IELTS Academic Writing component (AWC) is one of fourskill-based subtests in the IELTS battery intended to assess “the language ability ofcandidates who intend to study or work where English is used as the language ofcommunication” (International English Language Testing System, 2005). On theAWC, candidates are required to compose two short essays in 1 hour. The first

340 GREEN

FIGURE 1 A model of washback direction, variability, and intensity adapted from Green(2003).

Page 9: Watching for Washback_Article

(Task 1) is a description of a diagram or table, and the second (Task 2) is a discur-sive essay written in response to a point of view, argument, or problem.

Drawing both on the available literature (Banerjee, 2000; Chalhoub-Deville &Turner, 2000; Coffin, 2004; Douglas, 2000; Hale et al., 1996; Mickan & Slater,2003; Moore & Morton, 1999; Thorp & Kennedy, 2003) and on the views ofcourse providers, a framework developed by Weigle (2002) was used to relate thedesign of the IELTS AWC to theories of academic literacy. Information about thetest was derived from sample test materials published by the IELTS partners andfrom unpublished specifications for item writers obtained from Cambridge Eng-lish for Speakers of Other Languages (Cambridge Assessment, Cambridge, UK)under a research agreement. The review highlighted the similarities between thetwo IELTS tasks and paradigmatic reports or essays assigned by participant teach-ers in universities but also highlighted differences between the design of the IELTSAWC and the broader concerns of EAP. Briefly, the following areas emerged as thekey differences between the two.

IELTS tasks involve

• abstract and impersonal topics, but these are not targeted at learners’ chosenacademic subjects

• a limited range of text types (descriptions of iconic data and five-paragraphpersuasive essays)

• composition of texts based on personal opinions about how the world shouldbe.

They do not involve

• literature reviews, summaries, and other genres of relevance to the academiccontext

• arguments based in reading and research (as university assignmentsoften do).

In short, IELTS may imply an approach to instruction that passes over featuresof canonical EAP, such as the integration of source material in learners’ writing,learning of subject- specific vocabulary and text types, and strategies for copingwith the length of university-level written assignments. Preparing for IELTS mayinvolve learning how to shape texts to meet the expectations of examiners (as ex-pressed through the scoring criteria) rather than those of university staff. The scor-ing criteria may imply a focus in the classroom on grammar, vocabulary, and or-ganisation of text, with limited attention given to quality of content. The timedconditions may lead to practice in composing short texts under timed conditions,perhaps in response to past (or imitation) examination papers.

WATCHING FOR WASHBACK 341

Page 10: Watching for Washback_Article

It should be noted that the IELTS AWC has undergone continual modificationand revision since its inception in 1989. The test has changed in certain respectssince this research was conducted. Where these changes are of relevance to thestudy, this is indicated in the text. Regular updates on IELTS developments can befound on the IELTS Web site (http://www.ielts.org).

METHODS

Methodology

The literature review has pointed to specific features that might indicate the influ-ence of IELTS AWC test design on instruction. To investigate whether these pre-dictions about the likely impact of IELTS would be borne out in this context and toexplore how instruction varied between IELTS preparation and EAP classes in ac-ademic writing, a series of classroom observations were carried out at selectedU.K. centres. The observations provided evidence of how preparation for theIELTS AWC was conducted in practice and how this compared with other forms ofEAP writing provision. The courses included in the study involved IELTS prepara-tion in combination with varying proportions of EAP and general English classes.However, comparisons were made at the narrower level of the writing class: be-tween classes directed toward the IELTS AWC and EAP classes directed at prepar-ing learners for writing in English at U.K. universities.

Instrumentation

One observation instrument that has been widely used in washback studies (Bur-rows, 1998; Cheng, 2005; Read & Hayes, 2003; Watanabe, 1996) is the Communi-cative Orientation to Language Teaching (COLT) observation schedule (Spada &Fröhlich, 1995). The scheme is designed to be used flexibly and in real time, de-scribing classroom events at the level of activities and their constituent episodes.These are described qualitatively and recorded quantitatively under a series ofheadings to build a picture of the balance of patterns of classroom organisation,content, and student modality. Spada and Fröhlich (1995) explain the terms thus:“Separate activities include such things as a drill, a translation task, a discussion ora game. Three episodes of one activity would be: teacher introduces dialogue,teacher reads dialogue aloud, individual students read parts of dialogue aloud”(p. 14).

As Read and Hayes (2003) discovered, the COLT schedule could not, withoutadaptation, identify features of direct test preparation, such as learning test-takingstrategies. Read and Hayes (2003) chose to supplement the COLT with the draftIIS observation schedule, an instrument developed by the IELTS partners in col-laboration with researchers from the University of Lancaster at the inception of theIIS (Saville, 2000). This schedule includes lists of text types and activities antici-

342 GREEN

Page 11: Watching for Washback_Article

pated to occur in preparation classes and is specifically designed to record in-stances of IELTS preparation. To address the needs of this study, but avoiding du-plication, elements of the IIS instrument were incorporated into the COLTschedule, and the resulting instrument was further refined through piloting. Theobservation schedule appears in the Appendix.

Among the modifications to the COLT, a Test References section was includedwith three categories: IELTS, Other, and Test Strategies. The IELTS section wasused to record mentions of the IELTS test, a separate note being made of the con-text. The Other section was used to record mention of tests or assessments otherthan IELTS (such as course exit tests). The Test Strategies section was used as a re-cord of test-taking strategy instruction, and the specific strategies were also noted.Copies of all materials used in class were collected for later analysis, and detailswere recorded separately on a second page of the observation form (see the Appen-dix). Under the heading “content,” the distinction made in the COLT between per-sonal or broad topics was extended to further differentiate broad from academic.An academic topic was indicated where teachers and students treated the topic asacademic subject matter: The primary focus would be learning about the topic,rather than exploiting the topic to learn about language. Additional notes weremade of aspects of instruction, such as homework assignments, that were of inter-est but not captured by the schedule.

The adapted schedule was piloted with five classes, including 240 min ofpresessional EAP and 150 min of IELTS preparation (60 in an IELTS intensivecourse and 90 in a course combining IELTS preparation with EAP). Observationswere recorded in real time at intervals of 1 min, and the time in hours and minuteswas entered at each episode boundary.

To provide an estimate of internal consistency, two classes observed at the be-ginning of the data collection exercise were video recorded and reanalyzed in realtime 3 months later. The 82.5% level of agreement between the two sets of obser-vations suggests that the instrument was being used consistently during the study.Interrater reliability was investigated in cooperation with an IIS consultant, whowas trained in the use of the schedule and independently observed the twovideocassettes. These ratings showed complete agreement on the number of activi-ties observed, with minor discrepancies in timing. There was agreement on 72% ofthe observed categories, with most of the differences being on the minor focus ofan activity. For example, the first observer recorded that one activity was led by theteacher, but the second also recorded that this involved learners working in smallgroups.

Participants and Settings

To provide a cross section of courses, a range of institutions were approached bytelephone to invite participation. These institutions were selected following an

WATCHING FOR WASHBACK 343

Page 12: Watching for Washback_Article

earlier survey of U.K. course providers; they had indicated willingness to partic-ipate in further research and were conveniently located. Three of these (two uni-versities and one private language school) declined the invitation. The remaininginstitutions—six universities, three colleges of further education, and four pri-vate language schools—represented a variety of courses for students intendingto enter higher education in the United Kingdom, both in IELTS preparationand, for purposes of comparison, presessional courses in EAP offered by univer-sities to prepare international students for the language demands of academicstudy.

The intention was to observe a minimum of one writing-focused class for eachmonth of a course, although this did not always prove possible. As summarised inTables 2 and 3, some 36 scheduled classes were observed covering over 51 hours,involving a total of 197 learners and given by 20 different teachers. The classes in-cluded eight different EAP groups. Each of these was observed either once ortwice over periods ranging from 8 to 12 weeks to give a total of 13 classes. Therewere 12 IELTS preparation groups. Each of these was observed on between oneand three occasions over periods of between 4 and 12 weeks to give a total of 22preparation classes.

Following each observation, teachers were briefly interviewed about the class.The interviews focused on five issues: the aims of the class, the extent to which theaims had been met, the place of the focal class in a teaching sequence, the extent towhich the class could be described as typical of writing classes on the course, andthe influence of the IELTS AWC. Because of other commitments, teachers did notalways have the time to take part in these interviews. Nonetheless, 22 of the classeswere accompanied by interview data (8 EAP and 14 IELTS).

Analysis

Comparisons were made between writing-focused IELTS preparation classes andEAP classes across courses (Table 4). The length of time spent on each activityprovided an index of how much time and what proportion of class time (calculatedas a percentage) was given to each form of participant organisation, content focus,content control, and student modality. Mentions of tests or of test-taking strategieswere treated as simple frequency data. As the data were not normally distributed,differences between classes were evaluated for significance (p < .05) through non-parametric Mann–Whitney U tests.

The activities observed were also reviewed qualitatively in the light of thewashback predictions to explore (a) how far the predictions were borne out inIELTS preparation and (b) how far they differentiated between writing-focusedIELTS preparation and EAP classes. Follow-up interviews provided an opportu-nity to probe aspects of teachers’ behaviour.

344 GREEN

Page 13: Watching for Washback_Article

345

TAB

LE2

Sum

mar

yof

IELT

SP

repa

ratio

nC

lass

esO

bser

ved

Cen

tre

Cou

rse

Type

Cla

ssTi

tle

Teac

her

Wee

kN

o.of

Wee

ksH

ours

/W

eek

Cou

rse

Hou

rs/

Wee

kA

ctiv

itie

sO

bsM

inN

o.of

Ss

Col

lege

AIE

LTS

and

EA

PIE

LTS

1(F

)3

68

236

827

Col

lege

BIE

LTS

and

Gen

Eng

lish

IELT

S2

(M)

212

823

713

77

Col

lege

B6

237

929

Col

lege

B10

234

944

Col

lege

CIE

LTS

and

Gen

Eng

lish

IELT

S3

(M)

34

621

495

8C

olle

geD

IELT

San

dG

enE

nglis

hIE

LTS

4(F

)2

88

236

122

10C

olle

geD

723

711

87

Col

lege

EIE

LTS

IELT

Sw

ritin

g5

(M)

16

1025

614

812

Col

lege

E5

253

114

8C

olle

geE

IELT

SIE

LTS

wri

ting

6(M

)1

610

254

6712

Col

lege

E3

256

119

11C

olle

geE

525

411

412

Uni

vers

ityA

IELT

San

dE

AP

IELT

S7

(F)

36

4.5

22.5

993

7U

nive

rsity

A5

22.5

596

7U

nive

rsity

BIE

LTS

and

EA

PIE

LTS

8(M

)2

88

205

559

Uni

vers

ityB

620

546

7U

nive

rsity

BIE

LTS

and

EA

PIE

LTS

9(M

)2

88

202

508

Uni

vers

ityB

620

327

7U

nive

rsity

CIE

LTS

and

EA

PIE

LTS

10(F

)2

87.

525

690

8C

olle

geF

IELT

San

dE

AP

IELT

S11

(M)

38

921

579

14C

olle

geG

IELT

San

dE

AP

IELT

S12

(F)

410

1025

352

5C

olle

geG

625

338

6A

vera

ge7.

508.

0822

.95

5.00

87.6

48.

41

Page 14: Watching for Washback_Article

346

TAB

LE3

Sum

mar

yof

EA

PC

lass

esO

bser

ved

Cen

tre

Cou

rse

Type

Cla

ssTi

tle

Teac

her

Wee

kN

o.of

Wee

ksH

ours

/W

eek

Cou

rse

Hou

rs/

Wee

kA

ctiv

itie

sO

bsM

inN

o.of

Ss

Col

lege

GIE

LTS

and

EA

PSt

udy

skill

s12

(F)

410

1025

661

11C

olle

geF

625

558

11C

olle

geA

IELT

San

dE

AP

Rea

ding

and

wri

ting

13(M

)3

128

239

127

7C

olle

geA

1123

579

5U

nive

rsity

DE

AP

Rea

ding

and

wri

ting

15(F

)4

127.

520

512

315

Uni

vers

ityD

820

512

611

Uni

vers

ityE

EA

PA

cade

mic

wri

ting

16(M

)2

810

284

8913

Uni

vers

ityE

628

352

14U

nive

rsity

EE

AP

Aca

dem

icw

ritin

g17

(F)

28

928

797

14U

nive

rsity

E6

287

9514

Uni

vers

ityF

EA

PR

eadi

ngan

dw

ritin

g18

(M)

78

7.5

254

9312

Uni

vers

ityF

EA

PR

eadi

ngan

dw

ritin

g19

(F)

78

7.5

253

9810

Uni

vers

ityF

EA

PR

eadi

ngan

dw

ritin

g20

(M)

28

7.5

254

7312

Ave

rage

9.25

8.38

24.8

55.

1590

.08

11.4

6

Page 15: Watching for Washback_Article

347

TABLE 4A Comparison of Timing, Episodes, and Activities in IELTS-

and EAP-Focused Classes (Minutes)

Course Type CentreObserved

time Activities EpisodesMin perActivity

Episodesper

Activity

IELTS Preparation College A 82 6 22 13.67 3.67College B 137 7 26 19.57 3.71College B 92 7 27 13.14 3.86College B 94 4 18 23.50 4.50College C 95 4 14 23.75 3.50College D 122 6 14 20.33 2.33College D 118 7 20 16.86 2.86College E 148 6 20 24.67 3.33College E 114 3 8 38.00 2.67College E 67 4 16 16.75 4.00College E 119 6 14 19.83 2.33College E 114 4 16 28.50 4.00University A 93 9 23 10.33 2.56University A 96 5 11 19.20 2.20University B 55 5 14 11.00 2.80University B 46 5 18 9.20 3.60University B 50 2 8 25.00 4.00University B 27 3 9 9.00 3.00University C 90 6 16 15.00 2.67College F 79 5 15 15.80 3.00College G 52 3 10 17.33 3.33College G 38 3 6 12.67 2.00

Average IELTSPreparation

87.64 5.00 15.68 18.32 3.18

EAP College G 61 6 13 10.17 2.17College G 58 5 20 11.60 4.00College A 123 9 32 14.11 3.56College A 126 5 16 15.80 3.20University D 89 5 15 24.60 3.00University D 52 5 22 25.20 4.40University E 97 4 15 22.25 3.75University E 95 3 10 17.33 3.33University E 93 7 25 13.86 3.57University E 98 7 20 13.57 2.86University F 73 4 14 23.25 3.50University F 127 3 12 32.67 4.00University F 79 4 11 18.25 2.75

Average EAP 90.08 5.15 17.31 18.67 3.39Significance

(Mann–WhitneyU test)

0.68 0.89 0.61 0.84 0.39

Page 16: Watching for Washback_Article

RESULTS: FREQUENCY DATA

The observed IELTS and academic writing classes were of similar length. The ra-tio of episodes to activities was also very similar across class types; there were ap-proximately 10 episodes to every 3 activities, with each activity taking up just over18 min on average.

Table 5 compares participant organisation by course type. The results of theMann–Whitney test displayed in the final row of Table 5 indicate that there wereno significant (p < .05) differences between class types in participant organisation.The predominant form across classes was Teacher–Students/Class. Calculatingthis as a percentage of total class time for each course type gives 56% ofpresessional EAP class time and 54% of IELTS preparation class time (see Table5). This did not generally involve lectures, but whole-class interactions centred onthe teacher. The differences found by Read and Hayes (2003) between teachers inhow they organised their classes were not repeated across the broader sample here,nor did students spend as much of their time in class on individual test practice.However, the proportion of class time spent on individual work on the same exer-cise did increase as IELTS classes progressed: an average of 20% of class time inthe first half of these courses compared with 33% in the second half.

Reflecting the focused atmosphere of the classes observed, discipline was notrecorded as the main focus of any activity, although occasionally, a teacher re-minded students to speak English rather than their first language. Procedural issuestook up 9% of time in IELTS and 12% of time in presessional EAP courses. Ob-served variation in the amount of time spent on procedures also appeared to be amatter of teacher style, rather than of test focus.

The results displayed in Table 6 reveal some evidence of differences betweenclass types in content. Forty-six percent of IELTS class time involved a major fo-cus on language form (grammar and vocabulary) as compared with 22% ofpresessional EAP class time. However, this did not prove to be significant in theanalysis (p = .07). Across classes observed, topics were mostly broad: relating toissues of general rather than personal interest. Fifty-eight percent of time in IELTSpreparation classes and 48% of time in presessional EAP classes was spent work-ing with topics in this category (p = .41). Little time (4% of IELTS and 1% ofpresessional EAP classes) was spent on immediate personal topics (and these typi-cally occurred only during brief introductory episodes). These proportions aresimilar to those observed by Read and Hayes (2003). Academic topics—those top-ics that became the focus of the class in their own right—occurred only in twopresessional EAP classes, making up 2% of the total presessional EAP class timeobserved. Again, the differences between courses were not significant (p = .19).

Across the classes observed, topics were generally limited to one or two activi-ties. Switches of topic would occur with each change of task. However, there wasevidence that topics were pursued in more depth in at least some EAP classes. Four

348 GREEN

Page 17: Watching for Washback_Article

349

TAB

LE5

AC

ompa

rison

ofP

artic

ipan

tOrg

aniz

atio

nin

IELT

S-

and

EA

P-F

ocus

edC

lass

es(M

inut

es)

Cou

rse

Type

Cen

tre

Obs

erve

dTi

me

T–C

/SS–

C/S

Indi

vidu

alSa

me

Indi

vidu

alD

iffer

ent

Gro

upSa

me

Gro

upD

iffer

ent

IELT

Spr

epar

atio

nC

olle

geA

8240

039

00

0C

olle

geB

137

480

710

00

Col

lege

B92

515

260

130

Col

lege

B94

640

360

60

Col

lege

C95

530

40

380

Col

lege

D12

226

022

074

0C

olle

geD

118

610

90

3317

Col

lege

E14

857

417

043

0C

olle

geE

114

370

00

750

Col

lege

E67

660

00

00

Col

lege

E11

973

00

046

0C

olle

geE

114

590

00

550

Uni

vers

ityA

9366

010

017

0U

nive

rsity

A96

670

00

290

Uni

vers

ityB

5522

022

011

0U

nive

rsity

B46

150

150

160

Uni

vers

ityB

5038

09

03

0U

nive

rsity

B27

110

00

150

Uni

vers

ityC

9058

025

05

0C

olle

geF

7949

05

025

0C

olle

geG

5238

00

014

0C

olle

geG

3829

09

00

0

(con

tinue

d)

Page 18: Watching for Washback_Article

350

Ave

rage

IELT

Spr

epar

atio

n87

.64

46.7

30.

4114

.50

0.00

23.5

50.

77E

AP

Col

lege

G61

310

30

260

Col

lege

G58

410

80

180

Col

lege

A12

364

1449

00

0C

olle

geA

126

340

120

200

Uni

vers

ityD

8928

013

043

0U

nive

rsity

D52

640

170

450

Uni

vers

ityE

9762

016

07

0U

nive

rsity

E95

502

00

00

Uni

vers

ityE

9378

016

06

0U

nive

rsity

E98

530

350

60

Uni

vers

ityF

7352

246

020

0U

nive

rsity

F12

736

017

540

0U

nive

rsity

F79

2023

022

03

Ave

rage

EA

P90

.08

47.1

54.

8514

.77

2.08

17.7

70.

23Si

gnif

ican

ce(M

ann–

Whi

tney

Ute

st)

0.68

0.90

0.09

0.68

0.06

0.69

0.73

TAB

LE5

(con

tinue

d)

Cou

rse

Type

Cen

tre

Obs

erve

dTi

me

T–C

/SS–

C/S

Indi

vidu

alSa

me

Indi

vidu

alD

iffer

ent

Gro

upSa

me

Gro

upD

iffer

ent

Page 19: Watching for Washback_Article

351

TAB

LE6

AC

ompa

rison

ofC

onte

ntin

IELT

S-

and

EA

P-F

ocus

edC

lass

es(M

inut

es)

Cou

rse

Type

Cen

tre

Obs

erve

dTi

me

Pro

cedu

ral

Dis

cipl

ine

Form

Fun

ctio

nD

isco

urse

Soci

olin

guis

tic

Imm

edia

teB

road

Aca

dem

ic

IELT

Spr

epar

atio

nC

olle

geA

823

00

00

00

720

Col

lege

B13

720

081

1013

00

105

0C

olle

geB

9218

00

074

00

780

Col

lege

B94

00

8210

00

044

0C

olle

geC

957

037

2137

04

500

Col

lege

D12

213

010

147

00

011

30

Col

lege

D11

815

040

034

00

770

Col

lege

E14

87

00

130

029

480

Col

lege

E11

48

015

00

00

600

Col

lege

E67

30

5757

90

08

0C

olle

geE

119

50

100

268

40

530

Col

lege

E11

45

029

03

00

410

Uni

vers

ityA

934

037

010

00

450

Uni

vers

ityA

969

087

4329

00

380

Uni

vers

ityB

553

028

228

00

480

Uni

vers

ityB

464

022

00

040

00

Uni

vers

ityB

509

044

00

00

440

Uni

vers

ityB

271

00

024

00

250

Uni

vers

ityC

9014

040

5119

00

710

Col

lege

F79

150

1452

00

941

0C

olle

geG

525

013

224

00

190

Col

lege

G38

30

350

00

010

0

(con

tinue

d)

Page 20: Watching for Washback_Article

352

Ave

rage

IELT

Spr

epar

atio

n87

.64

7.77

0.00

39.1

816

.09

13.2

70.

183.

7349

.55

0.00

EA

PC

olle

geG

618

00

054

00

00

Col

lege

G58

70

253

373

029

0C

olle

geA

123

180

00

900

058

0C

olle

geA

126

30

2416

450

073

0U

nive

rsity

D89

20

330

629

299

0U

nive

rsity

D52

160

4745

00

035

0U

nive

rsity

E97

70

2274

00

00

0U

nive

rsity

E95

50

00

470

00

23U

nive

rsity

E93

280

480

30

59

0U

nive

rsity

E98

70

00

920

00

0U

nive

rsity

F73

30

00

890

089

0U

nive

rsity

F12

721

00

05

00

710

Uni

vers

ityF

7910

038

00

00

600

Ave

rage

EA

P90

.08

10.3

80.

0018

.23

10.6

236

.00

2.46

0.54

40.2

31.

77Si

gnif

ican

ce(M

an–W

hitn

eyU

test

)

0.68

0.36

1.00

0.07

0.22

0.08

0.28

0.72

0.41

0.19

TAB

LE6

(Con

tinue

d)

Cou

rse

Type

Cen

tre

Obs

erve

dTi

me

Pro

cedu

ral

Dis

cipl

ine

Form

Fun

ctio

nD

isco

urse

Soci

olin

guis

tic

Imm

edia

teB

road

Aca

dem

ic

Page 21: Watching for Washback_Article

presessional EAP classes (29% of those observed) remained focused on a singletopic for the duration of a class, while IELTS classes tended to switch topic morefrequently; only one IELTS class (5%) remained with a single topic throughout.Eleven of the IELTS classes (50% of those observed) included more than five top-ics, while this was true of just three (21%) of the presessional EAP classes.

IELTS was mentioned by participants a total of 129 times during IELTS classes,compared with 10 times during presessional EAP classes (Table 7). Of these 10mentions, 9 were on combination IELTS/EAP courses (courses in which studentswere studying EAP but were also following an IELTS preparation course strand).In these cases, teachers mentioned how the class content could be applied to thetest, or students asked for information about the test. Specific test strategies or“test-taking tips” were provided by teachers on a total of 67 occasions, or just overthree times per class on average. Just 2 of the 22 IELTS classes observed (bothgiven by the same teacher) included no explicit mention of the IELTS test, al-though even here, it remained the implicit focus for class activities and did appearin class materials. References to IELTS and mention of test-taking strategies werethe only quantitative features recorded on the observation schedule to show signifi-cant differences between class types (p = .00012 for references to IELTS and p =.00002 for mention of test-taking strategies).

Tests other than IELTS were mentioned just five times during presessional EAPclasses on courses unrelated to IELTS. One class accounted for four of these men-tions. This included 50 min of explicit preparation for a course exit test to be held 3days after the observation. The teacher introduced the test format, describing thetiming and format of the tasks, and gave 25 min to a practice writing exercise.

Results for content control are also displayed in Table 7. In both IELTS andpresessional EAP classes the teacher or text most often held control of class con-tent (82% of the time in IELTS and 73% of the time in presessional EAP classes).There was no significant difference between course types (p = .21). Students didnot hold sole control of content during any of the classes observed, but control wasshared between teacher, text, and students a little more often in presessional EAP(27% of class time) than in IELTS classes (18% of class time). Again, the differ-ences were not significant (p = .07), but the degree of teacher control was some-what higher and student control lower than that observed by Read and Hayes(2003).

Modality was similar across course types (Table 8). Listening (mostly duringteacher-centred activities) took up just over half of class time in both types of class(p = .93), while writing activities took up around 10% (p = .81).

IELTS teachers were, like those observed by Read and Hayes (2003) and byHawkey (2006), eclectic in their use of materials, using a variety of books andself-produced materials. Four of seven books used in IELTS classes included a ref-erence to IELTS in their titles. Of the remaining three, two were intended as prepa-ration material for other tests. The four IELTS titles were all course books directed

WATCHING FOR WASHBACK 353

Page 22: Watching for Washback_Article

354

TABLE 7A Comparison of Test Focus (Frequency) and Content Control (Minutes) in

IELTS- and EAP-Focused Classes

Course Type Centre IELTS Other Strategies T/Text T/S/Text

IELTS Preparation College A 7 0 6 45 37College B 7 0 4 72 0College B 19 0 3 66 3College B 2 0 3 89 0College C 2 0 1 63 31College D 6 0 3 95 26College D 4 0 3 105 0College E 3 0 1 54 58College E 3 0 1 114 0College E 14 0 6 66 0College E 5 0 4 119 0College E 14 0 6 114 0University A 9 0 7 75 18University A 5 0 3 61 35University B 3 0 2 46 6University B 3 0 3 19 30University B 0 0 0 50 0University B 0 0 0 26 0University C 7 0 7 81 9College F 5 0 1 79 0College G 8 0 9 52 0College G 3 0 0 38 0

Average IELTSPreparation

5.86 0.00 3.32 69.50 11.50

EAP College G 0 0 0 35 26College G 3 0 0 47 11College A 3 0 0 27 7College A 3 0 0 59 20University D 0 0 0 43 62University D 0 1 0 77 31University E 0 0 0 85 0University E 0 0 0 31 21University E 1 0 0 97 0University E 0 0 0 95 0University F 0 0 0 15 75University F 0 4 1 61 37University F 0 0 0 65 23

Average EAP 0.77 0.38 0.08 56.69 24.08Significance

(Mann–WhitneyU test)

0.00 0.06 0.00 0.21 0.07

Page 23: Watching for Washback_Article

355

TAB

LE8

AC

ompa

rison

ofS

tude

ntM

odal

ityin

IELT

S-

and

EA

P-F

ocus

edC

lass

es(M

inut

es)

Cou

rse

Type

Cen

tre

Obs

erve

dti

me

Lis

teni

ngSp

eaki

ngR

eadi

ngW

riti

ngS&

RS&

WL

&R

R&

WO

ther

IELT

SPr

epar

atio

nC

olle

geA

8240

120

012

180

00

Col

lege

B13

746

160

230

00

240

Col

lege

B92

3345

140

00

00

5C

olle

geB

9447

190

280

00

08

Col

lege

C95

3918

290

00

09

0C

olle

geD

122

2216

3026

60

022

15C

olle

geD

118

489

40

90

048

0C

olle

geE

148

9117

040

00

00

0C

olle

geE

114

390

600

00

015

0C

olle

geE

6767

00

00

00

00

Col

lege

E11

972

00

00

00

460

Col

lege

E11

470

00

410

00

00

Uni

vers

ityA

9353

1010

30

03

140

Uni

vers

ityA

9666

029

00

00

10

Uni

vers

ityB

5524

916

00

00

60

Uni

vers

ityB

4611

158

40

00

80

Uni

vers

ityB

5037

94

00

00

00

Uni

vers

ityB

2710

017

00

00

00

Uni

vers

ityC

9061

100

019

00

039

Col

lege

F79

387

290

50

00

0C

olle

geG

5239

014

00

00

00

Col

lege

G38

290

90

00

00

0

(con

tinue

d)

Page 24: Watching for Washback_Article

356

Ave

rage

IELT

SPr

epar

atio

n87

.64

44.6

49.

6412

.41

7.50

2.32

0.82

0.14

8.77

3.05

EA

PC

olle

geG

6131

326

03

00

00

Col

lege

G58

468

20

00

00

3C

olle

geA

123

5012

112

00

1339

0C

olle

geA

126

150

120

270

160

0U

nive

rsity

D89

480

1362

00

00

0U

nive

rsity

D52

478

1725

1710

00

0U

nive

rsity

E97

620

411

50

07

0U

nive

rsity

E95

520

00

00

00

0U

nive

rsity

E93

366

00

63

00

0U

nive

rsity

E98

530

70

120

522

0U

nive

rsity

F73

567

020

30

70

0U

nive

rsity

F12

736

1741

00

00

40

Uni

vers

ityF

7922

298

07

00

70

Ave

rage

EA

P90

.08

42.6

26.

929.

2311

.54

6.15

1.00

3.15

6.08

0.23

Sign

ific

ance

(Man

n–W

hitn

eyU

test

)

0.68

0.93

0.35

0.46

0.81

0.61

0.22

0.16

0.96

0.34

TAB

LE8

(Con

tinue

d)

Cou

rse

Type

Cen

tre

Obs

erve

dti

me

Lis

teni

ngSp

eaki

ngR

eadi

ngW

riti

ngS&

RS&

WL

&R

R&

WO

ther

Page 25: Watching for Washback_Article

toward the test, rather than collections of practice test material. In presessionalEAP courses observed, none of the titles included the acronym IELTS, and nonewas intended primarily as a test preparation text.

ACTIVITIES AND EPISODES: QUALITATIVEOBSERVATIONS RELATING TO TEST DESIGN ISSUES

The frequency data were indicative of broad similarities between courses in teach-ing method. There was little variation associated with the type of course in partici-pant organisation, content control, or student modality. In contrast, the qualitativeobservations captured in the activities and episodes sections of the schedule high-lighted differences in the content that were not well captured in the frequency data.These qualitative observations are presented here in relation to the task descriptioncategories developed by Weigle (2002).

Subject Matter and Stimulus

During work in class, topics were similar in both IELTS and EAP classes. EAPclasswork did not generally relate to specific academic disciplines to any greaterextent than did IELTS preparation, although, as noted previously, the IELTSclasses did involve more frequent changes in topic. Where learners were directedto explore topics further, IELTS teachers suggested reading magazines such as TheEconomist to learn about topics that might occur in IELTS; EAP teachers set pro-jects relating to students’ academic subjects. To this extent, the EAP classes did, aspredicted, take greater account of subject specialisms.

Genre, Rhetorical Task, and Pattern of Exposition

IELTS-directed classes, across teachers and institutions, involved activities with aclear and direct relationship to the IELTS AWC. Frequently observed activities in-cluded question analysis, brainstorming ideas, forming an outline plan with topicsentences and sentence and paragraph building, all apparently directed towardtest-like writing practice exercises. In comparison with IELTS preparation, therewas a greater variety of activities on EAP courses, and these appeared to take ac-count of more features of academic writing.

Limitations on the selection of topics for Task 2 and the choice of data for pre-sentation in Task 1 may have attracted the test preparation teaching strategies ob-served, including

• providing lists of topics• encouraging learners to read about potential IELTS topics in the media

WATCHING FOR WASHBACK 357

Page 26: Watching for Washback_Article

• practice in planning and writing essays (but without incorporating sourcematerial in ways consistent with academic writing)

• encouraging memorisation of formulaic phrases• teaching relevant vocabulary and structures (such as—for AWC Task 1—

phrases for describing trends and reporting on information in graphs and dia-grams or learning past rather than present forms of new verbs).

However, IELTS preparation and EAP courses also had much in common. EAPclasses incorporated concern with the discursive essay genre; the rhetoric of de-scription and evaluation; and cause and effect, comparison, and problem and solu-tion patterns of exposition.

Cognitive Demands

As observed by Brown (1998), Hayes and Read (2004), and Hawkey (2006),IELTS classes were explicitly directed toward test success. In preparation classes,there were frequent mentions of IELTS and of strategies for dealing with the tasks.Most of the materials used were either taken from IELTS textbooks or chosen bythe teacher to reflect the test. Frequent essay writing practice involved test practiceunder timed conditions and completing tasks closely modelled on IELTS (often as-signed as homework). Learners were encouraged to become thoroughly familiarwith the test tasks.

Instruction in EAP courses included stages that did not feature in IELTS prepa-ration. As these courses progressed, learners were often encouraged to go beyondthe classroom to locate material and to integrate this into their writing. Learnerswere never observed to leave an IELTS preparation classroom to carry out otheractivities, but EAP classes observed included one group visit to a library, and in an-other, individuals left the classroom to carry out research work elsewhere. Atten-tion was given in EAP classes to issues of plagiarism, the appropriate use ofsources, and the compilation of bibliographies.

Specification of Audience, Role, Tone, and Style

In IELTS classes, primary attention was given to the expectations of examiners,rather than to university lecturers or educated nonspecialist readers: the audiencesspecified at the time in the task instructions (since January 2006, audiences are nolonger specified in the IELTS task rubrics). Audience and the role of the writerwere not a major focus of instruction in the observed EAP classes, although onedid include a discussion of university teachers’ expectations of written work. BothIELTS preparation and EAP classes gave time to formal language, but explicitteaching of features of academic style (hedging) was observed only in EAPclasses.

358 GREEN

Page 27: Watching for Washback_Article

Performance Conditions: Length and Time Allowed

As predicted, reflecting the test format, timed writing activities were more fre-quent in IELTS preparation classes, but these were also observed in two EAPclasses. Some IELTS classes did, contrary to expectations, provide opportunitiesfor redrafting and error correction, though there would be limited opportunities forthese under test conditions. When questioned about this in a follow-up interview,one teacher reported that he saw the relevance of these exercises to IELTS in theawareness of essay structure they developed. This may be an instance of teacherbeliefs about language learning and skill building outweighing the direct influenceof the test format in guiding behaviour.

The length of the test tasks directly influenced the requirements set by teachersfor written work: Essays assigned in IELTS preparation classes were to be 150 or250 words in length. Two IELTS teachers observed provided formulaic openingsto Task 1, and most included sentence expansion activities as methods of increas-ing the number of words in a response. Teachers sometimes related such activitiesdirectly to the IELTS length requirements.

Prompts and Transcription Mode

Writing task prompts assigned in IELTS classes employed similar wording to stan-dard test tasks. In addition to producing full-length essays, IELTS preparation stu-dents were given practice in writing essay plans based on a range of IELTS prompts.Analyzing prompts for key words was another frequently observed activity.

Rating Scales and Criteria

The IELTS rating scales were updated in 2005. At the time of this study, these in-cluded task fulfilment; coherence and cohesion (Task 1); arguments, ideas, and ev-idence; communicative quality (Task 2); and vocabulary and sentence structure(both tasks). The new scoring criteria are task achievement (Task 1); task response(Task 2); coherence and cohesion; lexical resource; and grammatical range and ac-curacy. Bridges and Shaw (2004) provided an overview of their revision.

Although the differences were not statistically significant, the frequency datasuggested the possibility of a greater focus on form in IELTS classes. Qualitativedata suggested that more attention was given to grammar exercises in IELTSclasses, compared with a stronger focus on discourse in EAP classes. This mightreflect the importance IELTS afforded to vocabulary and sentence structure, theonly criterion used to score both Task 1 and Task 2. However, attention to formcould also reflect assumptions about learning and the needs of students; teachersmight assume that learners could quickly improve their language skills through in-struction focused on this area. It is also possible that IELTS learners made moregrammatical errors and that teachers were reacting to this with remedial activities.

WATCHING FOR WASHBACK 359

Page 28: Watching for Washback_Article

360

TABLE 9Summary of Findings Related to Weigle’s (2002) Writing Task Characteristics

Dimension Observations

Subject matter Lists of typical IELTS topics were prepared for students on IELTSpreparation courses. IELTS topics were broad and were not exploredbeyond task demands. Topics were sometimes sustained for longer inEAP courses, and these included some attention to students’ academicdisciplines.

Stimulus IELTS-like task stimuli heavily used in IELTS courses. Focus on topics ofgeneral interest. Students were encouraged to read newspapers or seriousmagazines outside class to learn about relevant topics. This was not thecase in EAP courses.

Genre IELTS classes limited to writing only Task 1–and Task 2–type essays. EAPcourses involved wider range.

Rhetorical task Rhetorical tasks limited to those required for the test: predominantlydescription for Task 1 and evaluation or hortation for Task 2. Nointegration of academic sources (unlike EAP, where this was a focus).

Pattern of exposition IELTS classes limited to Task 1 and Task 2. EAP involved greater variety,for example, classes on writing definitions

Cognitive demands IELTS classes involved reproducing information: from graph or frompersonal knowledge. EAP classes included more writing from sourcematerial.

SpecificationAudience Some mention in IELTS classes of university lecturers (the specified

audience), but more attention given to examiner expectations, includingteaching of “ways to impress the examiner.”

Role IELTS classes did not focus on the role of the university student. This wasan occasional focus in EAP classes.

Tone, style IELTS preparation taught formal tone, but not including features of anacademic style such as hedging, which were included in EAP classes.

Length Essays for the IELTS classes were short (100–300 words). Writingassignments were generally longer in EAP classes. IELTS students wereencouraged to count words. Teaching focus in some IELTS preparationclasses on how to use more words—sentence expansion.

Time allowed Timed essay practice activities were more frequent in IELTS classes.However, redrafting of work and error correction were common practicein both EAP and IELTS classes.

Prompt wording IELTS preparation involved question analysis based on the generic IELTStask prompts.

Choice of prompts Students were sometimes given a choice of topic for IELTS practice essays,but this was always limited to IELTS-like tasks.

Transcription mode Little word processing observed in IELTS writing (and none in class). Useof IT was observed in EAP classes.

Scoring criteria Feedback on essays was mostly in the form of band scores in IELTSpreparation classes.

IELTS preparation involved(a) Teaching of organisational templates for coherence, argument

structure(b) Teaching grammar points relevant to test—error analysis/useful

structures(c) Encouraging use of more formal and varied vocabulary

Page 29: Watching for Washback_Article

Postobservation interviews with teachers did not cast light on this as teachers didnot generally seem to be aware of an emphasis on grammar in IELTS classes.

RESULTS: POSTOBSERVATION TEACHER INTERVIEWS

Although the analysis of the frequency data revealed few differences betweenclasses in organisation, modality, or content, interviews with teachers indicatedthat they approached the two class types very differently. Thirteen of the 14IELTS classes for which interview data were available included mention of theIELTS writing test as an aim of the class. The one IELTS preparation class saidnot to be influenced by the test came in the first week of a 6-week course. Itemerged from the postobservation interview that the central activity (findingsupporting examples for popular proverbs from students’ countries) was in-tended as a means of preparing students for the demands of Task 2 without di-rectly introducing test-like questions. Thus, although the teacher maintained thatthe test did not directly influence the class, the demands of Task 2 were the ulti-mate goal.

Class aims in IELTS preparation courses included both practice in performing di-rectly test-derived tasks (Task 1 or Task 2 writing practice) and a wide variety ofother activities intended to build test-relevant skills. These aims were categorised asfollows:

• gaining an overview of test demands• building grammar and vocabulary related to test demands• analyzing Task 2 questions• learning about thesis statements, topic sentences, and paragraph structure• supporting propositions with evidence• selecting data in response to Task 1• focussing on specific areas of difficulty through self- or peer correction• understanding the IELTS assessment criteria.

Some of the aims for presessional EAP classes were similar to those reportedfor IELTS:

• learning how to describe processes• learning how to construct paragraphs• learning about a problem and solution essay structure• reviewing the tense system• debating an issue.

WATCHING FOR WASHBACK 361

Page 30: Watching for Washback_Article

Others seemed to have no parallel among the IELTS classes:

• learning how to write definitions• learning about hedging in academic writing• distinguishing one’s own ideas from others’• learning to integrate source material• learning how to construct a bibliography.

Both IELTS and presessional EAP classes were said to follow a similar cycle,with input from the teacher, practice writing tasks, and diagnostic feedback. Fromthe interview data it appeared to be the content of this cycle, not the process, thatdifferentiated presessional EAP from IELTS preparation classes. This reportedemphasis was borne out in the content of the activities and episodes observed. AsBrown (1998) found in the courses he observed, the IELTS preparation cycle wasclosely tied to test content and practice of test tasks. Teachers said that the fre-quency of test practice intensified as the courses progressed, and this was againconsistent with the observational data reported previously. In the presessional EAPclasses, teachers built toward longer writing tasks, with learners being givengreater independence (for research activities and library work) as the coursesneared completion. Where courses involved a final test (as with University F),there might be some attention given to this, with students having opportunities fortest practice. However, in contrast to the IELTS preparation classes, which focusedon the test throughout, the teacher of the test familiarisation class observed at Uni-versity F reported that this would be the one session (in week seven of theeight-week course) to concentrate on the test.

In one combination course the teacher reported that as the test date approached,IELTS preparation leaked across the curriculum, exerting an influence on the con-tent of classes beyond the identified IELTS component, with students requestingpractice in test tasks during non- IELTS lessons. This was consistent with the ob-served mention of IELTS in non-IELTS classes at College A and University E,where learners requested more information about the AWC in nontest classes.

Teachers of IELTS courses claimed that writing class content was entirely dic-tated by the AWC. Conversely, those of EAP courses either dismissed the idea thatIELTS had any influence on their classes or suggested that it served as a usefulbaseline for their teaching; they could assume that learners arriving in their courseswith an IELTS score would have some knowledge of how to write a basicfive-paragraph essay.

Student Work

It was plain that the work collected from IELTS and EAP classes differed in the va-riety of tasks completed by learners. Reflecting the focus reported by teachers andseen in the classes observed, all written work collected from IELTS classes con-

362 GREEN

Page 31: Watching for Washback_Article

sisted of responses to practice IELTS tasks (with varying degrees of guidance fromsupporting materials). EAP tasks ranged from timed writing exercises on broadtopics of general interest based on personal knowledge or experience (similar toIELTS Task 2) to extended projects on topics relating to students’ academic disci-plines and including tables of contents, references, and bibliographies.

The task responses also differed in their presentation. All but two of the practiceIELTS tasks were handwritten, while the work collected from presessional EAPcourses, with the exception of work done in class under time constraints, was allword processed. As anticipated, there was a much greater range in the length ofEAP task responses than of IELTS responses. IELTS tasks collected ranged inlength from 98 to 445 words (compared with the IELTS requirements of 150 and250 words), while the presessional EAP essays ranged from 128 to 3,495 words. Itwill be interesting to see whether the recent introduction of a computer-basedIELTS, which offers test takers the option of word processing their responses, willlead to greater use of computers in IELTS preparation courses.

Of the six IELTS teachers, five marked student work using IELTS band scores,with one providing a breakdown of the score by the criteria used on the test. Thefive teachers giving scores often added a comment to the awarded score, such as“good 6” or “5.0+.” Two of the three EAP teachers provided scores (one as marksout of 20, the other as percentages), while the third made written comments but didnot give a score. One of the teachers giving scores used an analytical style of re-porting that might have been influenced by IELTS. This used the criteria “contentand task achievement,” “organisation and coherence,” “range and accuracy of lan-guage,” and (clearly beyond the scope of IELTS) “improvement between drafts.”

CONCLUSIONS

Although the primary concern of this article has been differences between IELTSand EAP classes, it should be emphasised that, as the washback model would pre-dict, there were found to be considerable areas of overlap. Organisation, contentcontrol, and student modality were all very similar across classes. Differences inlanguage content were nonsignificant. Excluding the references to the test, manyof the activities in the IELTS classes observed might not have been out of place inthe EAP classes. Both class types involved brainstorming and planning, with fre-quent practice in extensive writing. Although there was some variation, both en-couraged a formal, objective style of writing; offered instruction in discourse-levelorganisation; were concerned with the requirements implicit in task instructions;and involved work (often in the form of remediation) on grammar and vocabulary.

There was also evidence that teacher variables (such as common beliefs aboutthe value of editing and redrafting) may encourage practices that cannot be pre-dicted from test design. Given the lack of differences between class types in thefrequency data, it appears that many of the differences observed between classes

WATCHING FOR WASHBACK 363

Page 32: Watching for Washback_Article

might be linked rather to teacher or institutional variables, such as levels of profes-sional training and beliefs about effective learning, than to the influence of the test.In the evidence from qualitative observations and from examples of student work,self-correction of essays emerged as another area of variation between teachers,rather than between IELTS and EAP classes. Further research is needed to accountfor the relative influence of tests, training, resources, and prior beliefs on teachingmethods. It seems clear, in this study as elsewhere, that tests exert a less direct in-fluence on this aspect of teaching than on content.

Although there was evidence of common practice across course types, and al-though courses were taught by teachers with varying levels of experience and atti-tudes toward the test, IELTS preparation classes differed in consistent and salientways from EAP classes. As predicted by the washback model, differences betweenclasses could be traced to test design features, and test preparation focused nar-rowly on these features could not be said to offer the same range of skills andknowledge as EAP programs. The focus on the test in IELTS preparation classesdirected learners away from their academic subjects and toward the topics and texttypes featured in the test. Writing in IELTS preparation classes was time con-strained and brief, as it is in the test, while EAP learners also worked toward moreextensive and less speeded assignments.

This study may also point to some of the limitations of “watching for washback.”Superficially at least, as reflected in the frequency data, the IELTS classes lookedvery similar to the EAP classes. However, watching classes may not tell us enoughabout how they are experienced: about which aspects of a class are attended to by thelearners, what they learn from them, and whether the attention given to the test is re-warded with improvements in test scores. The teacher interviews showed that even ifthe content often appeared similar, they approached the two class types in very dif-ferent ways. This may have been equally true of the learners. There are also limita-tions in the explanatory power of the observations. Interviews provided some oppor-tunity to probe why teachers believed certain activities might be useful in preparingfor IELTS, but why do learners choose to study in IELTS courses? And why docourses and course materials take the shapes seen here?

Although its effectiveness has not been demonstrated, among the teachers ob-served, there was a consistent approach to preparing for the IELTS AWC thatcentred on building relevant writing skills. The shared nature of the approachand the relationship of this to the test design are indicative of washback. As theskills required by the IELTS AWC are, for the most part, relevant to writing inhigher education, there was much in common between IELTS and other forms ofEAP instruction. However, the restrictions on the IELTS tasks outlined in the re-view of the literature were also reflected in the narrower focus ofIELTS-directed classes. This suggests that learners will need to pass beyondIELTS preparation if they are to be fully equipped with the language skills theywill need for academic study.

364 GREEN

Page 33: Watching for Washback_Article

REFERENCES

Alderson, J. C., & Hamp-Lyons, L. (1996). TOEFL preparation courses: A study of washback. Lan-guage Testing, 13, 280–297.

Alderson, J. C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14, 115–129.Bachman, L. F. (2005). Building and supporting a case for test use. Language Assessment Quarterly, 2,

1–34.Bailey, K. M. (1996). Working for washback: A review of the washback concept in language testing.

Language Testing, 13, 257–279.Bailey, K. M. (1999). Washback in language testing. Princeton, NJ: ETS.Banerjee, J. (1996). The design of the classroom observation instruments (Internal Report). Cambridge,

UK: University of Cambridge Local Examinations Syndicate.Banerjee, J. (2000). Using English language screening tests in your institution. Paper presented at the

BALEAP Professional Issues Meeting, Trent University, Nottingham, UK.Bridges, G., & Shaw, S. (2004). IELTS writing: Revising assessment criteria and scales. Research

Notes, 18, 8–12.Brown, J. D. H. (1998). An investigation into approaches to IELTS preparation, with particular focus on

the Academic Writing component of the test. In S. Wood (Ed.), IELTS research reports (Vol. 1, pp.20–37). Sydney: ELICOS/IELTS.

Brown,J.D. (2000).Universityentranceexaminations:Strategies forcreatingpositivewashbackonEng-lish language teaching in Japan. Shiken JALT Testing and Evaluation SIG Newsletter, 3(2), 4–8.

Burrows, C. (1998). Searching for washback: An investigation of the impact on teachers of the imple-mentation into the adult migrant English program of the assessment of the certificates in spoken andwritten English. Unpublished doctoral dissertation, Macquarie University, Sydney.

Burrows, C. (2004). Washback in classroom-based assessment: A study of the washback effect in theAustralian Adult Migrant English Program. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washbackin language testing: Research contexts and methods (pp. 113–128). Mahwah, NJ: LawrenceErlbaum Associates, Inc.

Chalhoub-Deville, M., & Turner, C. E. (2000). What to look for in ESL admission tests: Cambridge cer-tificate exams, IELTS, and TOEFL. System, 28, 523–539.

Chapman, D. W., & Snyder, C. W. (2000). Can high stakes testing improve instruction: Re-examiningconventional wisdom. International Journal of Educational Development, 20, 457–474.

Cheng, L. (2005). Studies in language testing: Vol. 21. Changing language teaching through languagetesting: A washback study. Cambridge, UK: Cambridge University Press.

Coffin, C. (2004). Arguing about how the world is or how the world should be: The role of argument inIELTS tests. Journal of English for Academic Purposes, 3, 229–246.

Deakin, G. (1997). IELTS in context: Issues in EAP for overseas students. EA Journal, 15, 7–28.Douglas, D. (2000). Assessing languages for specific purposes. Cambridge, UK: Cambridge University

Press.Ferman, I. (2004). The washback of an EFL national oral matriculation test to teaching and learning. In

L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback in language testing: Research contexts andmethods (pp. 191–210). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Green, A. (2003). Test impact and English for academic purposes: A comparative study in backwashbetween IELTS preparation and university presessional courses. Unpublished doctoral dissertation,University of Surrey, Roehampton, UK.

Hale, G., Taylor, C., Bridgeman, B., Carson, J., Kroll, B., & Kantor, R. (1996). A study of writing tasksassigned in academic degree programs. Princeton, NJ: ETS.

Hamp-Lyons, L. (2005). The effect of the newly developed TOEFL format on EFL/ESL teaching andlearning in four countries. ILTA Online Newsletter, 1. Retrieved November 4, 2005, from http://www.iltaonline.com

WATCHING FOR WASHBACK 365

Page 34: Watching for Washback_Article

Hawkey, R. (2006). Studies in language testing: Vol. 24. Impact theory and practice: Studies of theIELTS test and Progetto Lingue 2000. Cambridge, UK: Cambridge University Press.

Hayes, B., & Read, J. (2004). IELTS test preparation in New Zealand: Preparing students for the IELTSAcademic Module. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback in language testing: Re-search contexts and methods (pp. 97–112). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Heyneman, S. P., & Ransom, A. W. (1990). Using examinations and testing to improve educationalquality. Educational Policy, 4, 177–192.

Hughes, A. (1989). Testing for language teachers. Cambridge, UK: Cambridge University Press.Hughes, A. (1993). Backwash and TOEFL 2000. Unpublished manuscript, University of Reading,

Reading, UK.International English Language Testing System. (2005). The IELTS handbook. Cambridge, UK: Uni-

versity of Cambridge Local Examinations Syndicate.Kellaghan, T., & Greaney, V. (1992). Using examinations to improve education: A study of fourteen Af-

rican countries. Washington, DC: The World Bank.Messick, S. (1996). Validity and washback in language testing. Language Testing, 13, 241–256.Mickan, P., & Slater, S. (2003). Text analysis and the assessment of academic writing. In R. Tulloh

(Ed.), IELTS research reports (Vol. 5, pp. 59–88). Canberra: IELTS Australia.Moore, T., & Morton, J. (1999). Authenticity in the IELTS Academic Module Writing Test: A compara-

tive study of Task 2 items and university assignments. In R. Tulloh (Ed.), IELTS research reports(Vol. 2, pp. 64–106). Canberra: IELTS Australia.

Qi, L. (2004). Has a high-stakes test produced the intended changes? In L. Cheng, Y. Watanabe, & A.Curtis (Eds.), Washback in language testing: Research contexts and methods (pp. 171–190).Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Read, J., & Hayes, B. (2003). The impact of the IELTS test on preparation for academic study in NewZealand. In R. Tulloh (Ed.), IELTS research reports (Vol. 5, pp. 153–206). Canberra: IELTS Austra-lia.

Read, J., & Hirsh, D. (2005). English language levels in tertiary institutions (Export Levy ProgrammeProject E4). Wellington: Education New Zealand.

Resnick, L. B., & Resnick, D. P. (1992). Assessing the thinking curriculum: New tools for educationalreform. In B. G. Gifford & M. C. O’Conner (Eds.), Changing assessments: Alternative views of apti-tude, achievement and instruction (pp. 37–75). Boston: Kluwer Academic.

Saville, N. (2000). Investigating the impact of international language examinations. Research Notes, 2,4–7.

Spada, N., & Fröhlich, M. (1995). COLT observation scheme: Coding conventions and applications.Sydney: National Centre for English Language Teaching and Research, Macquarie University.

Thorp, D., & Kennedy, C. (2003). What makes a “good” IELTS answer in academic writing? (IELTSResearch Report). London: British Council.

Turner, C. (2001). The need for impact studies of L2 performance testing and rating: Identifying areasof potential consequences at all levels of the testing cycle. In C. Elder, A. Brown, A. K. Hill,N. Iwashita, T. Lumley, T. McNamara, & K. O’Loughlin (Eds.), Experimenting with uncertainty: Es-says in honour of Alan Davies (pp. 138–149). Cambridge, UK: Cambridge University Press.

Turner, J. (2004). Language as academic purpose. Journal of English for Academic Purposes, 3,95–109.

Wall, D. (1996). Introducing new tests into traditional systems: Insights from general education andfrom innovation theory. Language Testing, 13, 334–354.

Wall, D. (2005). Studies in language testing: Vol. 22. The impact of high-stakes testing on classroomteaching: A case study using insights from testing and innovation theory. Cambridge, UK: Cam-bridge University Press.

Wall, D., & Horák, T. (2004). TOEFL impact study: The influence of next generation TOEFL on prepa-ration classes in Central and Eastern Europe. Language Testing Update, 36, 40–41.

366 GREEN

Page 35: Watching for Washback_Article

Watanabe, Y. (1996). Does grammar translation come from the entrance examination? Preliminaryfindings from classroom-based research. Language Testing, 13, 318–333.

Watanabe, Y. (2001). Does the university entrance examination motivate learners? A case study oflearner interviews. In A. Murakami (Ed.), Trans-equator exchanges: A collection of academic pa-pers in honour of Professor David Ingram (pp. 100–110). Akita, Japan: Akita University.

Watanabe, Y. (2004). Methodology in washback studies. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.),Washback in language testing: Research contexts and methods (pp. 19–36). Mahwah, NJ: LawrenceErlbaum Associates, Inc.

Weigle, S. C. (2002). Assessing writing. Cambridge, UK: Cambridge University Press.

WATCHING FOR WASHBACK 367

Page 36: Watching for Washback_Article

368

Page 37: Watching for Washback_Article