1

Potentials and boundaries of formative assessment:

Insights from New Zealand

Yongqi GuVictoria University of Wellington

New Zealand

peter.gu@vuw.ac.nz

2

The big question

•What is assessment for?

3

Assessment purpose: norm- vs. criterion-referenced

•To rank or select among the students: Norm-referenced

•To see who can achieve set goals or standards: Criterion-referenced

4

Assessment purposes: Summative vs formative

Formative SummativePrimary Purpose

Adjusting instruction

Assigning grades, accountability

Proximity to Instruction

During After

Feedback Frequent, detailed

Infrequent, general

Stakes Low High

5

Purposes of classroom assessment• To obtain info on how effective teaching

has been• To provide feedback to learners:

– Progress? – Weaknesses?

• To give learners an incentive to study• To clarify to both teacher and learners

what learners need to achieve in terms of: – what to learn– amount of learning– level of performance

• …

6

Language assessment in China• Language assessment research

–Review of 9 key journals–All assessment articles (total=91) published between 2001 and 2005

• Language assessment practice

7

Journals reviewed1. Foreign Language Teaching and Research2. Modern Foreign Languages3. Journal of Foreign Languages4. Foreign Language World5. Foreign Languages and Their Teaching6. Journal of PLA University of Foreign Languages7. Foreign Language Research8. Foreign Language Education9. Foreign Languages Research

8

Journals reviewed (Chinese)1. 外语教学与研究2. 现代外语3. 外国语4. 外语界5. 外语与外语教学6. 解放军外国语学院学报7. 外语学刊8. 外语教学9. 外语研究

9

Language assessment articles published in China: CategoriesLable Topics A Proficiency testB Testing oral EnglishC Testing listeningD Testing writingE Testing readingF Integrative testingG Computer assisted testingH Reform in language testingI Band 4 and Band 6

10

Language assessment articles published in China: CategoriesJ Test analysisK Test taking strategiesL Test format M SLA and testingN AuthenticityO Curriculum and testingP Social aspects of language testingQ Pragmatic aspects of testingR Testing trendsS WashbackT Formative assessmentU Using tests for diagnostic purposes

11

Language testing articles published in China (2001-2005)

0

2

4

6

8

10

12

14

16

A B C D E F G H I J K L M N O P Q R S T U

Categories

Num

ber o

f arti

cles

12

Research concentration (2001-2005)G Computer assisted

testing7

S Washback 7H Reform in language

testing 9

I Band 4 and Band 6 10B Testing oral English 14

13

Four articles on formative assessment

Article Author Journal Issue在线同伴评价在写作能力发展中的作用 蒋宇红 外语教学与研究 2005/03

形成性评估在中国大学非英语专业英语写作教学中的运用曹荣平张文霞 周燕

外语教学 2004/05

自我评估同辈评估与培养自主学习能力之间的关系楼荷英 外语教学 2005/04

学生英文写作能力的自我评估 刘建达 现代外语 2002/03

14

Using summative tests for diagnostic purposes Article Author Journal Issue

大学英语校内测试模式的调查与分析 郭丽 外语界 2003/02

用重测法分析大学新生英语水平 王丽琴 解放军外国语学院学报 2001/04

15

Summary• Chinese researchers have concentrated

on summative assessment (assessing OF learning), not on formative assessment (assessing FOR learning).

• No article touched on criterion-referenced assessment from 2001 to 2005.

• Chinese teachers have not extensively explored formative assessment as a teaching or learning tool for the classroom

16

Formative assessment• Formative assessment refers to

assessment that is specially intended to generate feedback on performance to improve and accelerate learning (Sadler, 1998)

17

Assessment that can be formative is assessment that• Is embedded in a pedagogy of which it is an

essential part (eg constructivist).• Shares learning goals with students.• Involves students in self-assessment.• Provides feedback which leads to students

recognising the gap and closing it.• Is underpinned by confidence that every student

can improve. • Involves reviewing and reflecting on assessment

data.(Harlen, 1998)

18

Three conditions for formative feedback• Knowledge of standards (knowing what

good performance is)• Comparing these standards (desired

level of performance) to the student’s own current level of performance

• Taking action to close the gap between current and desired performance

Sadler (1989)

19

How does feebback work?Feedback in a self-regulated learning framework

Paths of internal feedback

Self-regulatory processes(Cognition, motivation & behavior)

Teacher sets taskGoals/criteria/standards

Domain knowledge

Strategy knowledge

Motivational beliefs

Student goals

Tactics & strategies

Internal learning outcomes

Externally Observableoutcomes

External feedback(Teacher/peers)

Processes internal to student

Butler and Winne (1995)

20

Does formative feedback work?Empirical evidence

Black and Wiliam (1998) summarized over 250 studies, and concluded that:

1. Improving formative assessment raises standards, dramatically. And FA helps low achievers more than other students.

2. Something should be done about it inside the classroom

3. Teachers can improve formative assessment

21

Factors that influence the effectiveness of FA1. Providing effective feedback to students.2. Student’s active involvement in their own

learning.3. Adjusting teaching to take account of the

results of assessment.4. Recognising the profound influence of

assessment on students’ motivation and self-esteem.

5. Ensuring pupils reflect on their learning and understand how to improve.

22

Inhibiting factors• A tendency for teachers to assess quantity

and presentation of work rather than quality of learning.

• Greater attention given to marking and grading, much of it tending to lower self esteem of students, rather than providing advice for improvement.

• A strong emphasis on comparing students with each other, which demoralizes the less successful learners.

23

The New Zealand Experience

• Standards-based assessment (SBA) at the national level

• Qualifications awarded by completing sets of unit standards (desired learning outcomes), not in terms of years of learning, and not by comparison with other students.

• Teachers design assessment tasks which relate to the specific learning needs of their learners and the context in which they are studying

24

The New Zealand Qualifications Framework • Aim: a single, coherent framework for

New Zealand• Unit standard: a set of specific learning

outcomes and the criteria for their assessment

• Assessment/performance level: achieved/not achieved

• Particular set of unit standards are required for certificates and diplomas at each level.

25

The NZ qualifications system: 10 levels

26

Level 1: DescriptionPROCESS LEARNING

DEMAND RESPONSIBILITY

Carry out processes that: •are limited in range •are repetitive and familiar •are employed within closely defined contexts

Employing: •recall •a narrow range of knowledge and cognitive skills •no generation of new ideas

Applied: •in directed activity •under close supervision •with no responsibility for the work or learning of others

27

Level 3: DescriptionPROCESS LEARNING

DEMAND RESPONSIBILITY

Carry out processes that: •require a range of well developed skills •offer a significant choice of procedures •are employed within a range of familiar contexts

Employing: •some relevant theoretical knowledge •interpretation of available information •discretion and judgement •a range of known responses to familiar problems

Applied: •in directed activity with some autonomy •under general supervision and quality checking •with significant responsibility for the quantity and quality of output •with possible responsibility for the output of others

28

NCEA Assessment• National Certificate of Educational

Achievement (NCEA): the National Qualifications Framework in schools

• NCEA=Levels 1-3 in NQF: senior secondary school

•  8 achievement standards for each school subject

• Each stardard has a credit value and sits at a specified level on the NQF.

• Credits are accumulated towards a qulification

29

Stadands-based Assessment in NCEA • Four levels of performance:

– Excellence – Merit – Credit – Not achieved

• Half of the standards for each subject are assessed by classroom teachers, the other half externally assessed nationally

30

An example: ESOL unit standard 17363

• Read independently information texts, using ESOL

• Level: 3• Credit: 5

31

Summary of SBA in NZSBA in NZ is criterion-referenced assessment.

It achieves a much better formative purpose than a summative one. This is because– SBA makes desired standards of learning explicit

to both teachers and learners– It enables and encourages rich and frequent

feedback from both the teacher and the students themselves

– It does not emphasise the grade, and gives students multiple chances to achieve a standard.

32

What’s good about Standards-based assessment? 1• Clear and specific goals and

standards for both learners and teachers

• Students assessed in terms of what they can do, and not how they compare with peers

• Slow learners are given multiple chances to achieve.

33

What’s good about Standards-based assessment? 2• Rich formative tasks during or at

the end of particular units of work• peer, and self assessment based

on observation, questioning, and feedback

• give detailed information about students' learning needs

• suggest appropriate teaching strategies to meet student needs

34

Problems with standards-based assessment 1• Assessment by unit standards greatly

increased teacher workloads as they designed tasks and implemented them, kept records of student achievement, and participated in the necessary moderation activities.

• Unit standards are assessed on an achieved/not achieved basis. There is no provision for rewarding superior performance and academically able students may not be challenged to achieve to the best of their ability.

35

Problems with standards-based assessment 2• Despite moderaton procedures,

ensuring consistent and fair assessment on a national basis is a tall order.

• SBA is good for low-stakes, formative purposes, but is used for high stakes purposes such as certification.

36

Insights for language assessment in China1. At the system level, the main purpose of

foreign language assessment in China is probably always going to remain norm-referenced, summative, and selective.

2. At the pedagogical level, setting clear and realistic goals and standards for Chinese teachers and learners of English is a natural step we have to start. However, a NZ-style standards-based assessment is both impractical and unnecessary.

37

Insights for language assessment in China 3. At the classroom level, I see an urgent

need for teachers to explore formative assessment tools such as self- and peer- assessment, portfolios that provide rich feedback information.

4. At the learner level, formative assessment will not only improve learning results, but also encourage self-reflection, self monitoring, and self-regulation.

38

Insights for language assessment in China 5. At the research agenda level,

it is not a question of whether, but a question of how formative assessment should be made use of systematically by Chinese teachers and learners to improve learning.

39

Purpose of assessment?• If a major purpose of assessment is

to inform teaching and learning, why aren’t we exploring formative assessment for its potentials in the foreign language classroom?

40

Thank you!