View
4.903
Download
0
Category
Preview:
DESCRIPTION
This slideshow was used for teacher training workshops I conducted in the fall of 2011 at the Center for English as a Second Language, University of Arizona (Tucson, USA).
Citation preview
‘Tes%ngatest’–Evalua%ngourAssessmentTools
EddyWhite,Ph.D.
AssessmentCoordinator
CenterforEnglishasaSecondLanguage
UniversityofArizona
1. Mybackground
2. Classroombasedassessment
3. Tests‐purposes/func%ons
4. The‘cardinalcriteria’forevalua%ngatest
5. Conclusions
Targets
2
1. Mybackground
2. Classroombasedassessment
3. Tests‐purposes/func%ons
4. The‘cardinalcriteria’forevalua%ngatest
5. Conclusions
Targets
3
(1994‐2009)
Classroom-based Assessment
• AssessmentofLearning
• AssessmentforLearning
1. Mybackground
2. Classroombasedassessment
3. Tests‐purposes/func%ons
4. The‘cardinalcriteria’forevalua%ngatest
5. Conclusions
Targets
8
Thegoalofassessmentisto...
9
The goal of assessment has to be, above all, to support the improvement of
learning and teaching.
(Fredrickson&Collins,1989)
10
definiGon:ClassroomAssessment
Assessment
Planning
CollecGng
Analyzing
ReporGng
ESLAssessment‐Purposes• idenGfystrengthsandweaknessesofindividualstudents,
• adjustinstrucGontobuildonstudents’strengthsandalleviateweaknesses,
• monitortheeffecGvenessofinstrucGon,• providefeedbacktostudents(sponsors,parents,etc.),and
• makedecisionsabouttheadvancementofstudentstothenextleveloftheprogram.
12(Source:ESLSeniorHighGuidetoImplementaGon,2002)
Consider • Researchsuggeststhatteachersspendfromone‐quartertoone‐thirdoftheirprofessionalGmeonassessment‐relatedacGviGes.
• Almostalldosowithoutthebenefitofhavinglearnedtheprinciplesofsoundassessment.
(S%ggins,2007)
Teacherslearnhowtoteachwithoutlearningmuchabouthowtoassess.(Heritage,2007)
14
Assessmentliteracy
• thekindsofassessmentknow‐howandunderstandingthatteachersneedtoassesstheirstudentseffecGvely
• AssessmentliterateeducatorsshouldhaveknowledgeandskillsrelatedtothebasicprinciplesofqualityassessmentpracGces
(SERVECenter,UniversityofNorthCarolina,2004)
AssessmentLiteracy
Know‐howandunderstandingteachersneedtoassessstudentseffec%velyand
maximizelearning
• Wemaynotlikeit,butstudentscananddoignoreourteaching;
• howeveriftheywanttogetaqualificaGon,theyhavetoparGcipateintheassessmentprocesseswedesignandimplement.
(Brown,S.2004.Assessmentforlearning.LearningandTeachinginHigherEduca0on,1,81‐89)
Importanceofclassroomassessment
43/W
Whoaretheassessment‘deciders’atyour
insGtuGon?
Classroom-Based Assessment: Challenges, Choices, and Consequences
AssessmentFrameworks
Assessmentframework
• ‐theseriesofassessmenttools(exams,tasks,projects,etc.)thatarescoredandusedtoarriveatasumma%vegradeforacourse
• ‐itshouldbeskills‐basedandknowledge‐based(i.e.SsdemonstratewhattheyknowaboutandcandowithEnglish)
• basedonlearningoutcomes
(Rowntree,1987)
• Thespiritandstyleofstudentassessmentdefinesthedefactocurriculum.
defacto=exisGnginfact,actual,whetherintendedornot
1. Mybackground
2. Classroombasedassessment
3. Tests‐purposes/func%ons
4. The‘cardinalcriteria’forevalua%ngatest
5. Conclusions
Targets
25
QuizGme!
26
AssessinganEnglisharGclesquiz
Context
• ConversaGonclass(listening&speaking)
• high‐beginnerlevel27
Whatisafundamentalproblemwiththisquiz?
28
Answer
29
1. Mybackground
2. Classroombasedassessment
3. Tests‐purposes/func%ons
4. The‘cardinalcriteria’forevalua%ngatest
5. Conclusions
Targets
30
Whatisatest?
31
Atest...• isamethodofmeasuringaperson’sability,knowledge,orperformanceinagivendomain.
• isaninstrument–asetoftechniques,procedures,oritems–thatrequiresperformanceonthepartofthetest‐taker.
32
Tests–measuringfunc%on
33
Atestmustmeasure• Sometestsmeasuregeneralability,whileothersfocusonveryspecificcompetenciesorobjecGves.
• Examples
• AmulG‐skillproficiencytestmeasuresgeneralability;
• aquizonrecognizingcorrectuseofdefinitearGclesmeasuresveryspecificknowledge.
34
• Atestmeasuresperformance,...
• but,theresultsimplythetest‐takersability,orcompetence.
35
• Performance‐basedtestssamplethetest‐takersactualuseoflanguage,
• butfromthosesamplesthetestadministratorinfersgeneralcompetence.
36
• Awell‐constructedtestisaninstrumentthatprovidesanaccuratemeasureofatest‐taker’sabilitywithinaparGculardomain.
• Construc%ngagoodtestisacomplextask.
37
Yourassessmentprac%ces?
38
Thinkaboutwhatis
happeninginyourcontextandyour
assessmentpracGces
• Inventories• Checklists• PeerRaGng• SelfRaGng• Journals• Porkolios• ObservaGons• Discussions• Interviews
• True–FalseItem• MulGpleChoice• CompleGon• ShortAnswer• Essay• PracGcalExam• Papers/Reports• Projects• QuesGonnaires• PresentaGons
YourassessmentpracGces?
Howdoyouassessyourstudents?
Foryou,whichofthefourskillsaremore/lesschallengingtotest?
41
1. Mybackground
2. Classroombasedassessment
3. Tests‐purposes/func%ons
4. The‘cardinalcriteria’forevalua%ngatest
5. Conclusions
Targets
42
QuizGme!
43
2010
• Exploringhowprinciplesoflanguageassessmentcanandshouldbeappliedtoformaltests.
• Theseprinciplesapplytoassessmentofallkinds.
• Howtousetheseprinciplestodesignagoodtest.
45
• Whatarethe‘fivecardinalcriteria’thatcanbeusedtodesignandevaluatealltypesofassessment?
46
Q.HowdoyouknowifatestiseffecGve,appropriate,useful,or,indown‐to‐earth
terms,a“good”test?
47
Fivekeyassessmentprinciples?
• Discuss• 3minutes
• Hint(fivenouns)48
Fivekeyassessmentprinciples
• PracGcality• Reliability• Validity• AuthenGcity• Washback
49
50
KeyAssessmentPrinciples
• ThesequesGonsprovideanexcellentcriteriontoevaluatethetestswedesignanduse.
52
53
1.PracGcality
54
• IstheprocedurerelaGvelyeasytoadminister?
Prac%calityconsidera%ons
• thelogisGcalandadministraGveissuesinvolvedinmaking,givingandscoringanassessmentinstrument
• theamountofGmeittakestoconstructandadminister
• theeaseofscoring• easeofinterpreGng/reporGngtheresults
55
AneffecGvetestisprac%cal.
Thismeansthatit:• isnotexcessivelyexpensive• stayswithinappropriateGmeconstraints
• isrelaGvelyeasytoadminister,and
• hasascoring/evaluaGonprocedurethatisspecificandGmeefficient
56
ThevalueandqualityofatestsomeGmeshingeonsuchni`y‐gri`yprac%cal
considera%ons.
57
• InclassroombasedtesGng,_________isalmostalwaysacrucialpracGcalfactorforbusyteachers.
58
59
2.Reliability
60
• Is all work being consistently marked to the
same standard?
• Areliabletestisconsistentanddependable.
• Ifyougivethesametesttothesamestudentormatchedstudentsontwodifferentoccasions,thetestshouldyieldsimilarresults.
61
Whatfactors
contributetothe
unreliabilityofatest?
62
TestUnreliability‐contribuGngfactors
• Studentrelatedreliability• Raterreliability(inter,intra)• Testadministra%onreliability
• Testreliability63
Q.Whatisonekeywaytoincreasereliability?
A.Userubrics
64
• Rubricsarescoringguidelines.• Theyprovideawaytomakejudgmentsfairandsoundwhenassessingperformance.
• Auniformsetofpreciselydefinedcriteriaorguidelinesaresetforthtojudgestudentwork.
65
66
3.Validity
‐ mostcomplexcriteria
‐ mostimportantprinciple
• Does the assessment
measure what we
really want to
measure?
Validity‐definiGon
• ‘Theextendtowhichinferencesmadefromassessmentresults
areappropriate,meaningful,andusefulintermsofthepurposeof
theassessment.’
(Gronlund,1998,p.226)68
• Avalidtestofreadingability...
• actuallymeasuresreadingability–
• notmathskills
• orpreviousknowledgeinasubject
• norwriGngskills• norsomeothervariableofquesGonablerelevance
69
Howisthevalidityofatestestablished?
1. Contentvalidity
2. Facevalidity
70
Contentvalidity• Ifatestrequiresthetest‐takertoperformthebehaviorthatisbeingmeasured...
• itcanclaimcontent‐relatedevidenceofvalidity(contentvalidity)
• e.g.Atestofaperson’sabilitytospeakanL2requiresthestudenttoactuallyspeakwithinsomesortofauthenGccontext.
• AtestwithpaperandpencilmulGplechoicequesGonsrequiringgrammaGcaljudgmentsdoesnotachievecontentvalidity.
71
AnotherwayofunderstandingcontentvalidityistoconsiderthedifferencebetweendirectandindirecttesGng.
• directtes%ng–involvesthetest‐takerinactuallyperformingthetargettask
• indirecttes%ng‐studentsnotperformingthetaskitself,butarelatedtask.
• e.g.tes%ngoralproduc%onofsyllablestress
72
Toachievecontentvalidityin
classroomassessment,trytotestperformance
directly.
73
74
Howisthevalidityofatestestablished?
1. Contentvalidity
2. Facevalidity
75
Facevalidity
• Theextenttowhichstudentsviewtheassessmentas:
1. fair2. relevant3. usefulforimprovinglearning
• Facevalidityreferstothedegreetowhichatestlooksright,andappearstomeasuretheknowledgeorabiliGesitclaimstomeasure.
76
Highfacevalidity:thetest...• iswell‐constructed,expectedformatwithfamiliartasks
• isclearlydoablewithinallouedGme• hasitemsthatareclearanduncomplicated• direcGonsthatarecrystalclear• hastasksrelatedtocoursework(contentvalidity)
• hasadifficultylevelthatpresentsareasonablechallenge
77
• Mostsignificantcardinalprincipleofassessmentevalua%on.
• Ifvalidityisnotestablished,allotherconsideraGonsmayberendereduseless.
78
79
4.AuthenGcity
80
• Arestudentsaskedtoperformreal‐worldtasks?
Testtaskauthen%city
• tasksrepresent,orcloselyapproximate,real‐worldtasks
• thetaskislikelytobeenactedinthe“realworld”
• notcontrivedorarGficial81
AuthenGcitychecklist• Isthelanguageinthetestasnaturalaspossible?
• Aretopicsascontextualizedaspossibleratherthanisolated?
• AretopicsandsituaGonsinteresGngenjoyable,and/orhumorous?
• IssomethemaGcorganizaGonprovided,suchasthroughastorylineorepisode?
• Dotasksrepresent,orcloselyapproximate,real‐worldtasks?
82
83
5.Washback
84
• Does the assessment have positive effects on learning
and teaching?
Washback=theeffectoftesGngonteaching
andlearning
85
‐ posi%vewashback‐ nega%vewashback
Washback• Classroomassessment:theaffectsofanassessmentonteachingandlearningpriortotheassessmentitself(preparaGon)
• Anotherformofwashback=theinformaGonthat‘washesback’tostudentsintheformofusefuldiagnosesofstrengthsandweaknesses.
• Formaltestsprovidenowashbackifstudentsreceiveasimpleleuergradeorsingleoverallnumericalscore.
86
Atestthatprovidesbeneficialwashback...
• posiGvelyinfluenceswhatandhowteachersteach
• posiGvelyinfluenceswhatandhowstudentslearn
• offerslearnersachancetoadequatelyprepare• giveslearnersfeedbackthatenhancestheirlanguagedevelopment
• providescondiGonsforpeakperformancebythelearner
87
Teachers’challenge
• tocreateclassroomteststhatserveaslearningtoolsthroughwhichwashbackis
achieved
88
89
1. Mybackground
2. Classroombasedassessment
3. Tests‐purposes/func%ons
4. The‘cardinalcriteria’forevalua%ngatest
5. Conclusions
Targets
Q.HowdoyouknowifatestiseffecGve,appropriate,useful,or,indown‐to‐earthterms,a“good”
test?
91
Answer.A‘good’test:• canbegivenwithinappropriateadministraGveconstraints,
• isdependable,• accuratelymeasureswhatyouwantittomeasure,
• thelanguageinthetestisrepresentaGveofreal‐worldlanguageuse,and
• thetestprovidesinformaGonthatisusefulforthelearner.
92
• TheseprincipleswillhelpyoumakeaccuratejudgmentsabouttheEnglishcompetenceofyourstudents.
• TheyprovideusefulguidelinesforevaluaGngexisGngtests,anddesigningourown.
93
AssessmentLiteracy
Know‐howandunderstandingteachersneedtoassessstudentseffec%velyand
maximizelearning
• Thereisnogewngawayfromthefactthatmostofthethingsthatgowrongwithassessmentareourfault,
• theresultofpoorassessmentdesign‐andnotthefaultofourstudents.
(Raceetal.,2005)
• Improvingstudentlearningimpliesimprovingtheassessmentsystem.
• Teachersoxenassumethatitistheirteachingthatdirectsstudentlearning.
• InpracGce,assessmentdirectsstudentlearning,becauseitistheassessmentsystemthatdefineswhatisworthlearning.
(Havnes,2004,p.1)
• ThereissubstanGalevidencethatassessment,ratherthanteaching,hasthemajorinfluenceonstudents’learning.
• ItdirectsauenGontowhatisimportant,actsasanincenGveforstudy,andhasapowerfuleffectonstudent’sapproachestotheirwork.
(Boud&Falchikov,2007)
RethinkingAssessmentinHigherEduca0on
“We owe it to ourselves and our students to devote at least as much energy to ensuring that our assessment practices are worthwhile as we do to ensuring that we teach well”.
Dr.DavidBoud,UniversityofTechnology,Sydney,Australia
98
ThankyouforyourGmeandparGcipaGon
Recommended