Embed Size (px)
Citation preview
AnAssessmentofSpeakingforteachingandtesting–apracticalguide
NeilBullockIATEFLTEASIG/CRELLAAutumnConference-workshopLuton28/29thOctober2017
Background
• MAAppliedLinguistics–dissertationintestdevelopment
• RaterTrainer&Rater
• Speaking&Listeningskills
• TestDevelopmentTeamLeader
• Examcentreadministrator
• CambridgeOralExaminer
• ICAEABoardmember–Research
• PhD-constructredefinitioninTLU????
Overview
• Whatisspeaking&assessment?
• Assessmentcontext&rationale
• Validity• Test/assessmentdevelopment
• Review
Whatisspeaking?
• Mentaloperation>assembledutterance(Field,2011)
• Audiochannel(Bachman&Palmer,1996)
• Socialevent(Fairclough,2003)• Useofcommunicationstrategies(Hedge,2000)
• Transactional&Interpersonalfunction(Harmer,2007)
• Cognitiveprocessing(Weir,2005)
WhatisSpeaking?
• Words• Grammar• Register• Pronunciation• Context• Interaction• Flow• Pragmatic• Strategic• Channel• Listening
Considerationsinspeakingassessment
• Typeofassessmenttask• Itemwriting• Humanelement• Training• Interlocutors&Raters• Administration• Instructions• Material• Room/waitingarea• Environment
Considerationsinspeakingassessment
• TypeoftaskèOPI,F2F/Voiceonly,TaskBased,semi-direct• Itemwritingèopenquestions,elicitlanguage,context• Humanelementèbodylang,givinganswers,translating• TrainingèInterlocutors,raters,administrators,standards• Interlocutors&RatersèHowmany?• Candidatesèsingle,pair,group• Administrationèpaperwork,forms,applications,ID• Instructionsèclear,writtenorspoken• Materialèprotection,recordings,availability,samples• Room/waitingareaèatease,quiet,lighting,heating• Environmentèdrinks,toilets,
Contextandrationale–Task1
TestSpecification
Validity?
• Accuraterepresentationofacandidate’sleveloflanguageknowledgeor(and)skills.(Weir,2005)
• Inferencesfromatestscorearejustifed.(Fulcher,2010)
• Towhatextent...(Bachman&Palmer,1996)
• Theessentialqualityofgoodassessment(Green,2014)
Validity–Task2
Validity
• Qualitativedata–oralquestionnaire
• Quantitativedata–writtenquestionnaire
• Developers,standardsetters,triallees,TTs
• Featuresofexpectedresponse
• Featuresofstimulustasks
Quantitativedata–writtenquestionnaire
Qualitativedata–oralquestionnaire
TestDevelopment–Task3
test purpose
criterion
construct
test specification
item writing
moderation a priori validity
pre-test trials
moderation
a posteriori validity
in-service operation
operational validity
(Fulcher,2010/Bullock2015)
Summary–thelearningliteracycontinuum
Needsanalysis
CurriculumMaterials
Methodology
Formativeassessment
LearningSummativeAssessment
FormalTesting
RealWorld
Objectives
Summary
• Defineyourcontextandrationale
• Learnersattheheartofstakeholders
• Knowhowtoqualifyandquantifyvalidity
• Haveasystematicprocess
• Realworldcommunicativeneeds
• https://www.coe.int/t/dg4/linguistic/ManualLanguageTest-
Alte2011_EN.pdf
References
• Bachman,L.F.(1990).FundamentalConsiderationsinLanguageTesting.Oxford:OUP.• Bullock,N.(2015).Content,Standards,AndResponsibility–TowardsAValidAndReliableTestOfEnglishLanguage
ProficiencyInAeronauticalCommunication,MADissertation.• Bullock,N.(2017).ARe-evaluationOfWashbackForLearningAndTestingLanguageInAeronautical
Communication,ICAEADubrovnikConferencePaper,ERAUScholarlyCommons,http://commons.erau.edu/icaea-workshop/2017/monday/19
• Fairclough,N.(2003),AnalysingDiscourse,Abingdon:Routledge.• Field,J.(2013)CognitiveValidity,inTaylor,L.(ed.)ExaminingSpeaking–StudiesinLanguageTesting.pp65-111,
Cambridge:CambridgeUniversityPress.• Fulcher,G.(2010).PracticalLanguageTesting.London:• Green,A.(2014).ExploringLanguageAssessmentandTesting.Abingdon:Routledge.• Harmer,J.(2007).ThepracticeofEnglishlanguageteaching.Harlow:PearsonLongman.
• Hedge,T.(2000).Teachingandlearninginthelanguageclassroom.Oxford:OxfordUniversityPress.• Weir,C.J.(2005).LanguageTestingandValidation–AnEvidenceBasedApproach.London:PalgraveMacmillan.
