L_zHamp Lyons the Challenge of Classroom-based Assessment

8/13/2019 L_zHamp Lyons the Challenge of Classroom-based Assessment

1/33

Liz HampLiz Hamp--LyonsLyons

The challenge ofhe challenge ofclassroomlassroom basedasedassessmentssessment


2/33

Copyright Liz HampCopyright Liz Hamp--Lyons 2013Lyons 2013 22

How formal highHow formal high--stakes tests relate tostakes tests relate to

actual language learningactual language learning**Do you agree that is a(n) effective way tolearn English

useful TOEFL

preparation

activity

%* rank %* rank

writing essays

100 1

914

giving oral presentations 100 1 24 17listening to real-life English 100 1 82 7reading real-life English texts 100 1 71 8writing a summary of something you have read 100 1 50 11writing a summary of something you have listened to 100 1 50 11giving an oral summary of something you have read

100

145

13

giving an oral summary of something you have listened t 100 1 41 15completing listening comprehension exercises 96 9 100 1having conversations in English 96 9 41 15completing reading comprehension exercises 91 11 100 1doing group projects 87 12 24 17studying an English language learning textbook

87

12 45 13learning grammar rules and doing grammar exercises 86 14 86 6

doing timed TOEFL tests 50 15 91 4studying test preparation materials 48 16 96 3memorizing examples of good essays 41 17 64 9memorizing vocabulary lists 30 18 64 9

Percentage of positive responses (A g r e e and St r o n g l y a g r e e )

Table 12: Comparison of students beliefs about useful TOEFL preparation activities andeffective language learning activities


3/33


What is the fundamental issue?What is the fundamental issue?

How teachers teach matters.How teachers teach matters. How teachers integrate assessment into theirHow teachers integrate assessment into theirteaching matters.teaching matters. Black & Wiliam defined formativeBlack & Wiliam defined formativeassessment asassessment as all those activitiesall those activities

undertaken by teachers, and/or theirundertaken by teachers, and/or their

students, which provide information to bestudents, which provide information to be

used as feedback to modify the teaching andused as feedback to modify the teaching and

learning activities in which they are engagedlearning activities in which they are engaged(1998, pp.7(1998, pp.7--8)8)


4/33


What is the fundamental issue?What is the fundamental issue?

How teachers teach matters.How teachers teach matters. How teachers integrate assessment into theirHow teachers integrate assessment into theirteaching matters.teaching matters. Black & Wiliam defined formativeBlack & Wiliam defined formativeassessment asassessment as all those activitiesall those activities

undertaken by teachers, and/or theirundertaken by teachers, and/or their

students, whichstudents, which provide informationprovide information to beto be

used as feedbackused as feedback toto modify the teaching andmodify the teaching and

learning activitieslearning activities in which they are engagedin which they are engaged(1998, pp.7(1998, pp.7--8)8)


5/33


More than one challengeMore than one challenge

Terminology/definitionTerminology/definition Identifying principlesIdentifying principles Relating principles to second/foreignRelating principles to second/foreignlanguage learning and teachinglanguage learning and teaching

Classroom assessment literacyClassroom assessment literacy


6/33Copyright Liz HampCopyright Liz Hamp--Lyons 2013Lyons 2013 66

TerminologyTerminology

In the last decadeIn the last decade--plus, a good deal of attention inplus, a good deal of attention in

the field of educational assessment has beenthe field of educational assessment has beenturned to various forms of assessment that areturned to various forms of assessment that are notnottestingtesting::

formative assessmentformative assessment assessment for learningassessment for learning

schoolschool--based assessmentbased assessment

classroomclassroom--based assessmentbased assessment learninglearning--oriented assessmentoriented assessment (and more)

(and more)


7/33


Principles:Principles:

Questions to be consideredQuestions to be considered

Is classroomIs classroom--based assessment necessarilybased assessment necessarily

formative? Are there conditions under which itformative? Are there conditions under which itmay not be formative?may not be formative?

Is overt, explicit feedback a necessary element ofIs overt, explicit feedback a necessary element ofclassroomclassroom--based assessment?based assessment?

Can assessment beCan assessment be for learningfor learning (as in Black &(as in Black &

Wiliam 1998; Black et al 2003) if it is taking placeWiliam 1998; Black et al 2003) if it is taking placein the classroom but being reported upwards toin the classroom but being reported upwards to

accountability systems?accountability systems?


8/33


Classroom PrinciplesClassroom Principles

Setting clear goalsSetting clear goals Designing appropriate learning andDesigning appropriate learning andassessment tasksassessment tasks Communication of assessment criteria toCommunication of assessment criteria toteachers and studentsteachers and students

Provision of highProvision of high--quality feedbackquality feedback

Conscious provision of opportunities forConscious provision of opportunities forselfself-- and peerand peer--assessmentassessment


9/33


Technical principlesTechnical principles

Teacher assessment should be the productTeacher assessment should be the productof professional judgement involvingof professional judgement involving

critical interpretation of evidence from studentcritical interpretation of evidence from student

performancesperformances

multiple sources of evidencemultiple sources of evidence

application of knowledge of appropriateapplication of knowledge of appropriate

measurement principlesmeasurement principles

value judgements about the meanings ofvalue judgements about the meanings of

resultsresults

Decision consistencyDecision consistency


10/33


Ethical principlesEthical principles

Classroom assessment must be fair.Classroom assessment must be fair. It must be aware of its unintended as wellIt must be aware of its unintended as wellintended consequences.intended consequences. It should positively influence studentsIt should positively influence studentsmotivation and learning.motivation and learning. It should be efficient and feasible forIt should be efficient and feasible forteachers.teachers.

It should be empowering for teachers.It should be empowering for teachers.


11/33


A new kind of fairnessA new kind of fairness

In the US, Linda DarlingIn the US, Linda Darling--Hammond (2010)Hammond (2010)wrote an important paper in which shewrote an important paper in which sheargued that while standardized testing is notargued that while standardized testing is notequitable, alternative assessment may not beequitable, alternative assessment may not be

equitable either. She argues that all kinds ofequitable either. She argues that all kinds ofassessmentassessment depend not only on the designdepend not only on the designof the assessments themselves, but also onof the assessments themselves, but also on

how well the assessment practices arehow well the assessment practices areinterwoven with the goals of authentic schoolinterwoven with the goals of authentic school

reform and effective teachingreform and effective teaching

for theirfor their

fairness. (p.5)fairness. (p.5)


12/33


Underlying processes or dataUnderlying processes or data

collection methods that can informcollection methods that can informassessment in the classroomassessment in the classroom

ObservationObservation

Recorded lessonsRecorded lessons

Portfolio collectionPortfolio collection Journals, questionnairesJournals, questionnaires InterviewsInterviews Recall protocolsRecall protocols PeerPeer--peer talk recordingspeer talk recordings


13/33


A different way of seeingA different way of seeing

We can see that all of these are tools toWe can see that all of these are tools tocreate acreate a windowwindow through which we canthrough which we cansee into the studentsee into the student beyond the productbeyond the product

of their learningof their learning

For classroom assessment, we want toFor classroom assessment, we want toknow not just what happened, but how andknow not just what happened, but how andwhy it happened.why it happened.

We also want to know that soon enough toWe also want to know that soon enough toput the knowledge to useput the knowledge to use..


14/33


Principles forPrinciples for languagelanguage

classrooms?classrooms?The third challenge in the context of a groupThe third challenge in the context of a group

such as EALTA is to consider whether (andsuch as EALTA is to consider whether (andperhaps which of) the principles hold trueperhaps which of) the principles hold true

when working with second, third or fourthwhen working with second, third or fourth

language learners, and also whether therelanguage learners, and also whether there

may be some additional principles that applymay be some additional principles that apply

specifically to the teaching and learning ofspecifically to the teaching and learning oflanguages.languages.


15/33


Is the medium the message?Is the medium the message?

The temptation will always be to operationalize aThe temptation will always be to operationalize aclassroomclassroom

--based languagebased language

assessmentassessment

as aas a

series of miniseries of mini--tests of specific linguistictests of specific linguisticknowledgeknowledge instead we must always look to theinstead we must always look to thepurposepurpose of the assessment.of the assessment.

But we must also look to the underlyingBut we must also look to the underlyingmessage about language that is projected by themessage about language that is projected by thetypes of assessments we use. Some minitypes of assessments we use. Some mini--teststestsof discrete language areas serve a usefulof discrete language areas serve a usefulpurpose; but need to be balanced bypurpose; but need to be balanced byassessments that focus on the realassessments that focus on the real--life value oflife value oflanguage.language.


16/33


ReaRea--Dickins (2007):Dickins (2007): the means by which tothe means by which toachieve consistency in making judgments aboutachieve consistency in making judgments aboutlanguage samples are well rehearsed, but muchlanguage samples are well rehearsed, but much

uncharted territory remains in the developmentuncharted territory remains in the development

of quality formative assessment.of quality formative assessment. The types of criteria that become important inThe types of criteria that become important inclassroom language assessment must includeclassroom language assessment must include

the provision of a rich array of opportunities forthe provision of a rich array of opportunities forlearners to use and stretch their linguisticlearners to use and stretch their linguistic

resourcesresources (p.516)(p.516)


17/33


In the context of languageIn the context of language

assessmentassessmentHampHamp--Lyons 2007, Figure 1: two extremes ofLyons 2007, Figure 1: two extremes of

assessment culturesassessment cultures

Classroom-based assessment Classical testing

Fluency-focused Accuracy-focused

Individual-focused Group- or norm-focused

Achievement/progress-focused Proficiency-focused

Learner-focused Language-focusedProcess-focused Product-focused

Teachers/students voices Rule-makers voicesLeads to assessment of learning Leads to teaching to the test


18/33


Message about language or aboutMessage about language or about

learning?learning? Black & Wiliam (1998) say:Black & Wiliam (1998) say: the core of thethe core of the

activity of formative assessment lies in theactivity of formative assessment lies in thesequence of two actions. The first is thesequence of two actions. The first is theperception by the student of a gapperception by the student of a gap

between a desired goal and his/her ownbetween a desired goal and his/her ownstate (of knowledge, and/or understanding,state (of knowledge, and/or understanding,and/or skills). The second is the actionand/or skills). The second is the action

taken by the learner to close the gap intaken by the learner to close the gap inorder to attain the desired goal.order to attain the desired goal. (p.21)(p.21)

This puts the student at the centre of theirThis puts the student at the centre of theirown assessment.own assessment.


19/33


Students as assessorsStudents as assessors

It is becoming clear that progress on effectiveIt is becoming clear that progress on effectiveclassroomclassroom--based assessment will not move farbased assessment will not move farforward until a body of practical support isforward until a body of practical support isdeveloped to help teachers learn how to teachdeveloped to help teachers learn how to teachtheir student to assess their own languagetheir student to assess their own language

development.development. Among the very many studies of the value ofAmong the very many studies of the value ofstudent feedback, most fail to convince becausestudent feedback, most fail to convince because

the support given to students was inadequate.the support given to students was inadequate. We take it for granted these days that teachersWe take it for granted these days that teachersneed to learn about assessmentneed to learn about assessment the same is truethe same is true

for students.for students.


20/33


The fourth challengeThe fourth challenge

is perhaps the greatest.is perhaps the greatest. Since classroomSince classroom--based assessment isbased assessment isboth new and different, and yet at theboth new and different, and yet at the

heart of what good teachers already do,heart of what good teachers already do,

how can teachers learn and practice thehow can teachers learn and practice theskills of assessment in their classrooms?skills of assessment in their classrooms?

FurthermoreFurthermorehow can they learn andhow can they learn andpractice the skills ofpractice the skills of languagelanguage assessmentassessmentusing formative principles?using formative principles?


21/33


Classroom assessmentClassroom assessmentsimple (?)simple (?)

tools for teacherstools for teachers QuestioningQuestioning

Probing questions (Probing questions (stretchstretch questions)questions)

Switchback questioning (drop/raise level)Switchback questioning (drop/raise level)

Collaborative questioningCollaborative questioning

FeedbackFeedback

CommentComment--onlyonly markingmarking

Peer feedbackPeer feedback

Feedback as feedFeedback as feed--forwardforward

Wait timeWait time

T h t t i th t iTeacher strategies that require


22/33


Teacher strategies that requireTeacher strategies that require

advanced understanding of howadvanced understanding of how

language workslanguage works

Teachers sharing criteria with learnersTeachers sharing criteria with learners

Setting targetsSetting targets

Deconstructing criteriaDeconstructing criteria

Group use of criteriaGroup use of criteria

Identifying progressIdentifying progress

SelfSelf--assessmentassessment

PeerPeer--toto--peer commentary and revisionpeer commentary and revision


23/33


BUTBUT----

an activity or elicitation procedure in itselfan activity or elicitation procedure in itselfis neutral. It is only in its implementationis neutral. It is only in its implementation

and the use to which the data that emergesand the use to which the data that emergesfrom a given activity is put that therefrom a given activity is put that theredevelops its formative or summativedevelops its formative or summative

potentialpotentialteachers may be trapped in their thinkingteachers may be trapped in their thinkingabout assessment between two potentiallyabout assessment between two potentially

competing paradigms: to measurecompeting paradigms: to measureachievement and progress on the oneachievement and progress on the onehand, or to facilitate through differenthand, or to facilitate through different

means their studentsmeans their students language learning onlanguage learning onthe other.the other. (Rea(Rea--Dickins 2011, p. 514)Dickins 2011, p. 514)


24/33


The dual roles of teachersThe dual roles of teachers

This the main site of tension in assessmentThis the main site of tension in assessment In the Hong Kong SBA we not onlyIn the Hong Kong SBA we not onlydeveloped as humanistic and interactive andeveloped as humanistic and interactive an

assessment of speaking as we couldassessment of speaking as we could

conceive within our parameters, weconceive within our parameters, weprovided a large amount of teacher PD,provided a large amount of teacher PD,and onand on--site support, and we did this over asite support, and we did this over a

55--year period.year period.


25/33


BUT still longitudinal research and PhDsBUT still longitudinal research and PhDsby our students show that some teachersby our students show that some teachershave interpreted SBA in ways differenthave interpreted SBA in ways different

from each other and in some cases farfrom each other and in some cases far

from the original intention and philosophy,from the original intention and philosophy,

but also sometimes far from thebut also sometimes far from the

specifications and guidelines provided inspecifications and guidelines provided indetail.detail.


26/33


Understanding the tensionUnderstanding the tension

Black & Wiliam (1998) argue that it is notBlack & Wiliam (1998) argue that it is notpossible to introduce or strengthenpossible to introduce or strengthenformative assessment in classroomsformative assessment in classroomswithout some radical change in overallwithout some radical change in overall

pedagogy because formative assessmentpedagogy because formative assessmentdemands ademands a constructivist classroomconstructivist classroom..

A classroom in which formativeA classroom in which formativeassessment is the norm and is effectiveassessment is the norm and is effectiverequires changes in the teacher and in therequires changes in the teacher and in the

students as well as in the assessmentstudents as well as in the assessmentmethods.methods.


27/33


Rising to the challengeRising to the challenge

Muchun Yin (2010) strikes a positive noteMuchun Yin (2010) strikes a positive notewhen he talks about the spread of teacherwhen he talks about the spread of teacherthinking research into research onthinking research into research onassessment.assessment.

We are increasingly coming to theWe are increasingly coming to theunderstanding that, if we are to changeunderstanding that, if we are to changeassessment practice, we must impactassessment practice, we must impactteachersteachers core beliefs and understandingcore beliefs and understandingabout what testing/assessment is for, andabout what testing/assessment is for, and

how they might affect practice.how they might affect practice.


28/33


Teacher assessment literacyTeacher assessment literacy

Professional development, training materials forProfessional development, training materials forspecific classroom assessment methods,specific classroom assessment methods,guidelines for giving feedbackguidelines for giving feedback these and manythese and manyother resources are needed BUT first teachersother resources are needed BUT first teachers

need to beneed to be assessment literateassessment literate..

Good language teachers are already and alwaysGood language teachers are already and alwayslooking, listening, monitoringlooking, listening, monitoring they arethey arecollecting data about their studentscollecting data about their students -- all theirall theirstudents.students.

The problem for many teachers is that they haveThe problem for many teachers is that they have

no schema with which to process the data andno schema with which to process the data andmake systematic judgements.make systematic judgements.


29/33


Where are we now?Where are we now?

Stiggins (2001) toldStiggins (2001) told a story of disregard by policya story of disregard by policymakers, school leaders, and the measurementmakers, school leaders, and the measurementcommunitycommunity for classroom assessment that hasfor classroom assessment that haskept classroom assessment from evenkept classroom assessment from even

approximating its potentialapproximating its potential (p.5)(p.5)

Have things changed since then?Have things changed since then? In the measurement community and in languageIn the measurement community and in languagetesting/assessment, there are many signs of smalltesting/assessment, there are many signs of small

changes.changes.


30/33


Classroom language assessmentClassroom language assessment

and politicsand politics It is policy makers, school leaders, and toIt is policy makers, school leaders, and to

some extent parents who are slower tosome extent parents who are slower tochange.change.

ThereThere areare positives: the Hong Kong SBApositives: the Hong Kong SBAnow spreading to Singapore and Brunei;now spreading to Singapore and Brunei;European initiatives such as DIALANG andEuropean initiatives such as DIALANG and

the European Language Portfolio.the European Language Portfolio.

Portfolio assessment of writing in manyPortfolio assessment of writing in manycountries.countries.

ButBut


31/33


Who makesWho makes

changechange

happen?happen?

ReRe defining classroomdefining classroom basedbased


32/33


ReRe--defining classroomdefining classroom--basedbased

assessmentassessment

any reflection by teachers (and/or learners) on the qualitiesany reflection by teachers (and/or learners) on the qualities

of a learnerof a learners (or group of learnerss (or group of learners) work which has the potential) work which has the potentialfor the use of that information by teachers (and/or learners) fofor the use of that information by teachers (and/or learners) forrteaching, learning (feedback), reporting, management orteaching, learning (feedback), reporting, management orsocialization purposes.socialization purposes.

the assessment opportunity (any actions, interactions orartefacts (planned or unplanned, deliberate or unconscious,

explicit or embedded) which have the potential to provideinformation on the qualities of a learners (or group of learners)performance), enables consideration of the more intuitive forms

of teacher decision-making in CBA.(Hill & McNamara 2012: 396)(Hill & McNamara 2012: 396)


33/33

Copyright LizCopyright Liz HampHamp Lyons 2013Lyons 2013 3333

Some ReferencesSome References

Black, P. & D.Black, P. & D. WiliamWiliam. (1998). Assessment and. (1998). Assessment andclassroom learning.classroom learning.Assessment in Education, 5Assessment in Education, 5,1: 7,1: 7--7373..

DarlingDarling--Hammond. L. (2010). PerformanceHammond. L. (2010). Performance--basedbasedassessment and educational equity.assessment and educational equity. Harvard EducationalHarvard EducationalReview,Review, 6464,1: 5,1: 5--31.31.

Hill, K. & T. McNamara. (2012). Developing aHill, K. & T. McNamara. (2012). Developing a

comprehensive, empirically based research frameworkcomprehensive, empirically based research frameworkfor classroomfor classroom--basedbased assessment. Language Testing.assessment. Language Testing.

ReaRea--DickinsDickins, P. (2007).Classroom, P. (2007).Classroom--based assessment:based assessment:Perils and pitfalls. In Cummins & Davison Eds.Perils and pitfalls. In Cummins & Davison Eds.International Handbook of ELT.International Handbook of ELT.

Yin, M. (2010). Understanding classroom languageYin, M. (2010). Understanding classroom languageassessment through teacher thinking research.assessment through teacher thinking research. LAQ 7LAQ 7,,2: 1762: 176--195.195.

Documents

L_zHamp Lyons the Challenge of Classroom-based Assessment