Changing Tests for Teachers: English Language School-Based Assessment in Hong Kong Chris Davison Faculty of Education, University of Hong Kong University.

  • Published on

  • View

  • Download

Embed Size (px)


<ul><li><p>Changing Tests for Teachers: English Language School-Based Assessment in Hong KongChris DavisonFaculty of Education, University of Hong Kong</p><p>University of Hong Kong Faculty of Education Inaugural Seminar Series English Language School-Based Assessment: Integrating Theory and Practice, Dec 19th 2005</p></li><li><p>Curriculum and assessment reform in Hong KongThe Hong Kong Examinations and Assessment Authority (HKEAA) has recently moved from norm-referenced to standards-referenced assessment, including the incorporation of a substantial school-based summative oral assessment component into the compulsory English language subject in the Hong Kong Certificate of Education Examination (HKCEE).</p></li><li><p>Curriculum and assessment reform in Hong KongThe initiative aims to align assessment more closely with current English Language teaching syllabus (Curriculum Development Council, 1999) and the new outcomes-based Senior Secondary curriculum. to assess learners achievement in areas not easily assessed by public examinations, in particular speaking and extensive reading. enhance student self-evaluation and life-long learning.</p></li><li><p>The SBA initiativeStarting in S.4, teachers assess their own students oral English language competencies through a range of authentic classroom-embedded activities over 2 years.Initiative developed by a team of researchers at the Faculty of Education, University of Hong Kong, in collaboration with HKEAA.Part of an evolving but coherent and systematic programme of research and evaluation, professional development and system-wide support</p></li><li><p>The SBA initiativeSBA is integrated into the teaching and learning process, with teachers involved at all stages of the assessment cycle: planning the assessment programme identifying and developing appropriate assessment tasks overseeing the assessment process making final judgments</p></li><li><p>The SBA initiative But previous studies have found Changes to summative assessment did not automatically lead to improvement in learning (Andrews 1994; Cheng 1998; Andrews, Fullilove &amp; Wong 2002).Assessment innovation was severely constrained by traditional school culture and by teacher, parental and student expectations (Cheung &amp; Ng 2000; Carless 2001; Adamson &amp; Davison 2003).Wide variation in teachers interpretations of student performance and of their role in the assessment process (Yung 2001).</p></li><li><p>The SBA content and structureAimsContentStructure Assessment criteria Making judgmentsStandardization Safeguards</p></li><li><p> SBA as research The SBA initiative has stimulated multi-level and multi-purpose research efforts from a growing research team with more than 12 interrelated research projects now underway, or under review. In addition, the research efforts have attracted the interest of a growing number of MPhil and PhD students, as well as the attention of the international assessment and English language education research community. </p></li><li><p> SBA as research</p><p>The particular characteristics of this research that makes it so interesting: its significance to the educational communityits high-impact and visibilityits concern to address both fundamental theoretical problems in language assessment (basic research) and the needs of school communities (applied research) its action-oriented collaborative approach</p></li><li><p> SBA as researchMany concerns and issues about SBA have been systematically gathered from key stakeholder groups during the process of SBA development, implementation and evaluation. Data was gathered over a year via questionnaires, individual and focus interviews, classroom observation and stimulated recall, and public briefing sessions as well as from the trialling of the tasks and assessment processes, and the training workshops, in total involving more than 1800 teachers and over 600 schools. </p></li><li><p>SBA as research Adapting Brindleys (1995) taxonomy, these concerns and issues can be divided into threetypes: Sociocultural (political)Theoretical (technical)Practical </p></li><li><p>Socio-cultural issuesStill a traditional testing culture in Hong Kong: Assessment practices are still primarily oriented towards providing data to select students for education or employment (Biggs 1995). External exam results are still the dominant way schools, students and teachers are evaluated and held accountable.Exam preparation is the traditional role of assessment in the senior secondary classroom (cf. classroom-based assessment).</p></li><li><p>Socio-cultural issuesTraditional cultural practices exemplified in a study of writing assessment in senior secondary English (Davison &amp; Tang, 2003):Many teachers reported they cannot assess, only mark. They feel unable to make a difference in teaching and learning, to respond to individual needs, because of community expectations of convergence and commonality. Teachers feel their assessment process are expected to change, without fundamental purposes being explicitly challenged. Such role conflict results in increasing stress and a decline in perceived teacher expertise.</p></li><li><p>Socio-cultural issuesHowever, the official adoption of the UK Assessment Reform Groups (1998) distinction between assessment for learning, and assessment of learning has stimulated the beginnings of a major paradigm shift Hong Kong schools are moving from a culture of testing to a learning and assessment culture (Hamp- Lyons 1999; in press).</p></li><li><p>Socio-cultural issuesSome teachers in the SBA study perceived cultural differences as a major stumbling block to assessment reform:I feel it takes time because the culture. The education culture in Hong Kong is different from other western countries and the students may not used to that kind of assessment. They like to do exam paper, they think they have something to learn.</p></li><li><p>Socio-cultural issuesAnother teacher newly returned from overseas, commented:Actually I was so surprised that ... How slow we Hong Kong people are in terms of education because I remembered when I was in Canada, we never ... You would never be graded on just one exam. Its quite like what we are trying to do actually, I believe that (assessment for learning) has been practised in those places for years and I was actually surprised nobody did anything (here) I am totally for assessment for learning.</p></li><li><p>Socio-cultural issuesResearch shows need for much more understanding, even at the system level, of the dual role of the teacher as facilitator and assessorthe concept of a student being assessed against criteria (rather than other students)the need for teacher consultation and interaction as part of the standardization process.</p></li><li><p>Socio-cultural issuesThe schools which were already doing extensive reading and whose students engaged in oral group work and individual presentations on a regular basis found it the easiest to integrate the assessment tasks into their existing practice:I just briefly tell the students about the task because it is in mid May, so they were quite busy that moment. So I asked them just make use of what they have been doing, say they just, they can just took from ERS report and work on it, say prepare a better review so that they can just have their presentation based on the review and I told them </p></li><li><p> Socio-cultural issuesThats what I did at the very beginning. Later on, I, I met them some days later, and I asked them to show me the book review they had written and I took a look at it and I found that there werent any major problems in it. So I just returned them the review and they started to prepare those tasks and later on, just right before they did the presentation, I helped them with the vocabulary and the names because they didnt know how to pronounce them. So I just helped them pronounce them correctly (but) I gave them more guidance according to the SBA documents because the five questions listed there (see Appendix II) suggested some sort of high order thinking skills So I try to scaffold them to think in that way.</p></li><li><p>Socio-cultural issuesSome teachers took longer to come to grips with the principles involved in SBA, and their implications for teaching and learning as well as for assessment practice:For students of higher forms, the time (8 min) is quite limited. They cant have enough time to introduce their books and ask each other questions.</p><p> but no time limits actually prescribed in SBA. </p></li><li><p>Socio-cultural issuesFairness was and is - a major concern. Fairness is fundamentally a sociocultural, rather than a technical, issue, a justice that goes beyond acting in agreed upon ways and seeks to look at justice of the arrangements leading up to and resulting from those actions. (Stobart, 2005, p. 1). Fair assessment cannot be considered in isolation from both the curriculum and the educational opportunities of the students some similarities to the more traditional notion of fairness embodied in the classical examinations for the Chinese civil servicethe concept that conditions should be consciously created to make opportunities open to all (Hamp-Lyons, 2005).</p></li><li><p>Socio-cultural issues</p></li><li><p>Socio-cultural issuesPreliminary findings suggest that these different approaches to fairness can be reconciled to some extent by providing teachers with opportunity to tailor classroom-based assessments to the needs of their students, according to commonly-agreed processes, outcomes and standards, with teacher assumptions about students and their oral language levels being made explicit through collaborative sharing and discussion with other teachers.</p></li><li><p>Theoretical issuesAt the theoretical level concerns with SBA revolve around the understanding and interpretation of traditional concepts such as reliability, validity and authenticity, eg. The students will memorize everything.Some teachers will tell students the task ahead of timeTeachers will favour their best students.Panel chairs will make all teacher agree with them. Schools will make up results.Tutorial schools will coach students. The HKEAA should take up all the scripts to check the accuracy of the marks. </p></li><li><p>Theoretical issuesThe traditional positivist position on language testing, with the tendency to map the standard psychometric criteria of reliability and validity onto the classroom assessment procedures, has been called into question, and the scope of validity has been significantly broadened (Chapelle 1999; Lynch 2001, 2003; McNamara 2001) and taken further by a number of researchers. (Rea-Dickins, in press)</p></li><li><p>Theoretical issuesDefining characteristics of school-based assessment (Stiggins &amp; Conklin 1992; Black &amp; Wiliam 1998; Brookhart 2003):Teacher-mediated Co-constructed and dialogic Context-dependent Multiple and varied Dynamic and evolving </p></li><li><p>Theoretical issuesSuch an approach can be seen as constructive and enabling because of its focus on assessing the process of learning, its attempt to elicit elaborated performance, and its emphasis on collaborative activity </p><p>Rea-Dickins (in press)</p></li><li><p>Theoretical issues However, if there is no reinterpretation of traditional conceptualizations of reliability and validity, SBA may be reduced to a series of summative mini-achievement tests external to the teaching and learning programme.the assessment of rehearsed monologues or dialogues with little or no opportunity for authentic language use.competition rather than collaboration</p></li><li><p>Theoretical issues</p><p>Cf. Clapham (2000, p. 21)Traditional test criteria do apply to alternative assessment:A problem with methods of alternative assessment, however, lies with their validity and reliability: Tasks are often not tried out to see whether they produce the desired linguistic information; marking criteria are not investigated to see whether they work; and raters are often not trained to give consistent marks.</p></li><li><p>Theoretical issuesPreliminary findings suggest Potentially much greater validity with SBA than external oral paper - More natural and authentic tasks - Students have a genuine reason to communicate as have read different texts- Students in comfortable, familiar environment-- Class teacher already familiar with the range of student performance, can ask questions to ensure the text is students own work little possibility of cheating</p></li><li><p>Theoretical issuesPotentially much greater reliability with SBA than external oral paper - A series of assessments (instead of one) by a teacher who is familiar with the students - Exemplars of student performances can provide a strong basis for discussing and internalizing the set standards.- Multiple opportunities for assessor reflection and standardization.- Disagreement between teachers the key to establishing trust (and trustworthiness).</p></li><li><p>Theoretical issuesBut initially lots of misconceptions Superficial tasksA lot of over-rehearsal and memorizationUse of buzzers and rigid timelinesOver-reliance on scripts and formulaAn emphasis on performance and recording Overbearing or stressed teachersUnhappy students </p></li><li><p>Practical issuesNot surprisingly, many practical concerns: - Access to appropriate assessment and extensive reading) resources- Access to activities and techniques, models/resources- Access to technical resources/expertise - Lack of recognition/support at the school-level- Concerns about student and parental expectations- (In)adequacy of training- Lack of time- Competing priorities </p></li><li><p>Practical issuesTeachers also expressed a need to understand much better the underlying assumptions of SBA, how to modify their teaching and learning, and how to set up effective assessment tasks:Many teachers have an urgent need to view others practices and share experiences ... We can film the good lessons for teachers and analyze the lessons. We (need to) focus on teaching instead of assessment only.</p></li><li><p>Practical issues More training will be provided to all teachers teaching S4 English between January and April so that they can be supported during the whole of the assessment period in second semester. SBA handbook, introductory DVD and booklet are available, on-line and face to face support.</p></li><li><p>Practical issuesPreliminary findings suggest Much extra time and effort required during the first year(s) of implementation as SBA is still new to both teachers and students, but once SBA a routine part of classroom activities, should be no significant increase in student/teacher workload?SBA changes nature of what is done rather than how much is done; the key is to integrate assessment, teaching and learning?</p></li><li><p>Positive impact on teachers, learners and schoolsDespite many concerns, the attitudes towards SBA by teachers, students and school communities are generally very positive:Personally, I enjoyed this trialing experience. I learnt how to judge the students through this activity. Moreover, my students tried to do the presentation based on the guiding questions given to them. Students found this presentation quite interesting and motivating. They learnt how to speak confidently and bravely during this assessment activity. They found this presentation rewarding since they can learn not only from the book but also through their actual participating experience.</p></li><li><p>Positive impact on teachers, learners and schoolsThose...</p></li></ul>


View more >