Bridging the Gap between Theory and Practice? J Charles AldersonLancaster University
What Gap?Between What and What?Whose View? From what Perspective?Should the Gap be Filled?By Whom?When and How?
Conference claimIn the last 25 - 30 years, research in the field of Language Testing has grown exponentially.A perennial concern, however, is that the day-to-day practice of language testing has not changed as quickly as our understanding of the theory of language testing. Language testing practitioners complain that research reports are hard to understand and often address matters that are irrelevant or at least far removed from their daily concerns. Researchers and practitioners seldom co-operate. We sense, however, that this climate is changing and we think that EALTA can contribute to and perhaps accelerate the pace of change.
The dichotomiesLong-standing as old as the hillsBut true?Beware dichotomiesIs action research a solution? Exploratory practice?Can theory ever be relevant?
Is the first term always better than the second? Or the second better than the first?Is reality better than what we aspire to? Are the two in conflict?Why opposition? Why not synergy?
Is practice necessarily behind research?Can theory not learn from practice?Is research necessarily hard to understand?Must everything be applicable to all realities tomorrow?Perhaps theory can help us understand practice, rather than be directly applicable
Is rejection of theory and research mere Philistinism? A refusal to try to understand, a rejection of jargon without going beneath the surface?A result of practitioners unwillingness to come to grips with unfamiliar concepts?Are theorists and researchers remote from everyday practice?
ChallengeIs there such a gap in language testing?Between theory and practice?Research and the concerns of practitioners?Test researchers and test developers?Testers and Teachers?Official claims and actual practice?The Ideal and the Real?
Does the Gap exist? And if so, is it a problem?
Research articles: irrelevant?The effect of test-taker gender, audience and topic on task performance in tape-mediated assessment of speakingExamining the predictive validity of a screening test for court interpretersDo visual chunks and planning impact performance on the graph description task in the SPEAK exam?
Research articles: irrelevant? To change or not to change: investigating the value of MCQ answer changing for Gulf Arab studentsRelating examinations to the Common European Framework: a ManualLimitations of the Common European Framework for developing comparable examinations and tests
Research articles: irrelevant? The development of a suite of computer-based diagnostic tests based on the Common European FrameworkThe Common European Framework and the European Language Portfolio: involving learners and their judgements in the assessment processAssessing the language of young learners
Research articles: irrelevant? Progress and problems in reforming public language examinations in Europe: cameos from the Baltic States, Greece, Hungary, Poland, Slovenia, France and GermanyLanguage testing in the military: problems, politics and progressDark alleys and blind bends: testing the language of learning
Research articles: irrelevant? Stakeholders' conflicting aims undermine the washback function of a high-stakes testReading to learn and reading to integrate: new tasks for reading comprehension tests?Linguistic sources of item bias for second generation immigrants in Dutch testsGeneralizability of writing scores: an application of structural equation modeling
Research articles: irrelevant? Did they take the same test? Examinee language proficiency and the structure of language testsSelf-assessment, preparation and response time on a computerized oral proficiency testPeer assessment of language proficiency Building and supporting a case for test use
Research articles: irrelevant? The impact of wearing a face mask in a high-stakes oral examination: An exploratory post-SARS study in Hong KongExamining rater effects in TestDaF writing and speaking performance assessments: A Many-Facet Rasch analysisIndividual feedback to enhance rater training: Does it work?
Research articles: irrelevant?Re-thinking second language admission requirements: Problems with language residency criteria and the need for language assessment and supportResolving score differences in the rating of writing samples: Does discussion improve the accuracy of scores?
Testers are remote from practitioners?The example of ENLTA1 year experience in secondary (12 year olds)17 years experience teaching adults10 years experience teacher-training (primary,secondary, adult)
Testers are remote from practitioners?The example of ENLTATeaching languages for 25 years:9 years training school 5 years secondary school11 years primary school
Testers are remote from practitioners?The example of ENLTA3 years teaching in evening classes/study circles6 months teaching in secondary school2 months teaching in upper secondary schoolTwo summer courses for adult students15 years teaching in municipal adult education20 years in-service training for teachers 15 years teaching language didactics and assessment at university level
Testers are remote from practitioners?The example of ENLTA2 years teaching language courses to adults and children2 years teaching part-time continuing courses for people who did not graduate from a primary, secondary or a vocational school; 1 year teaching secondary school pupils; One-to-one language teaching (mostly at home), usually helping school children to improve their language skills6 years teaching university students language classes, and language testing,
Testers are remote from practitioners?The example of ENLTATeacher in international language school In-company business English teacherKindergarten/primary school teacher in private schoolTemporary EFL summer school teacherTemporary pre-sessionals Teacher Trainer British Council Teaching OperationDirector of Studies, University Language Teaching centre, Language courses for teachers Undergraduates Teacher Training CollegePrivate teaching: young learners 7-14; secondary school learners 16-19; adults
Testers are remote from practitioners?The example of ENLTA3 years teacher of English at primary level,10 years EFL teacher, technical secondary school,3 years EFL teacher, grammar secondary school,9 years EFL teacher, private language school More than 15 years private one-to-one English teaching
ENLTA was remote from practitioners?
Pre conference workshopsKrakow, 2006Assessing speaking at B1 levelDesigning and reporting research
Voss, 2005 Using the European Language PortfolioClassical test analysis
ENLTA was remote from practitioners?
Activity Two: Survey of training in testing
Activity Four: Survey of attitudes to testing
Are these books irrelevant?
Only high-stakes proficiency testing? Irrelevant to practitioners?DIALANG no stakes, freely available, diagnosticNew school-leaving examinationsTraining teachers in modern approaches to language testing
Only high-stakes proficiency testing? Irrelevant to practitioners?Increasing interest in diagnosing students strengths and weaknessesResearching what language develops as learners progress through the CEFR levelsWashback of tests on teachers and learnersTeachers practices in assessing learners in class
Importance of classroom testingIncreased interest in researching classroom testing and understanding the issues and the needs.Important to understand better both what actually happens in classsroom assessment and what should or could happen, how it might be improved.EALTAs mission is to contribute to this.
Gaps? Codes of practice as the interface with realityMany Codes of Practice already (ILTA, AERA/APA, JALT, ALTE, etc etc)Who needs another one?Who monitors compliance, or even activity?What use a code of practice that is not implemented?
Gaps? Codes of practice and RealityAlderson, J.C. and G. Buck (1993). Standards in testing: a study of the practice of UK examination boards in EFL/ ESL testing. Language Testing 10(1), 1-26
LINKAGE TO THE COMMON EUROPEAN FRAMEWORK
What evidence is there of the quality of the process followed to link tests and examinations to the Common European Framework? Have the procedures recommended in the Manual and the Reference Supplement been applied appropriately?Is there a publicly available report on the linking process?
Gaps? Claims about links with the CEFR and realityImportance of CEFR in testing, training, publishing and curriculaMany claims of links to CEFRWho monitors quality? Council of Europe?ALTE?Self-monitoring?EALTA
Gaps? Claims about links with the CEFR and realityRole of EALTA as Europes prime, independent body of language testing practitioners?How? Where? When? Who? How much? What?Why? Why Not?
Bridging the Gap between Theory and Practice?Necessary?Classroom formative and summative testing and assessment Codes of practiceCEFR