Research on formative assessment practices

2013 AERA Classroom Assessment SIG Symposium:Advancing Research in Classroom AssessmentDylan Wiliam

2

Section 3: Formative assessment

Four chapters Formative and summative aspects of assessment:

Theoretical and research foundations in the context of pedagogy (Black)

Gathering evidence of student understanding (Heritage)

Feedback and instructional correctives (Wiliam) Examining formative feedback in the classroom

context: New research perspectives (Ruiz-Primo & Li)

Formative and summative aspects of assessment: Theoretical and research foundations in the context of pedagogy

Paul Black

Formative and summative assessment

A simple model of instruction Clear aims Planning activities Interaction Review of the learning Summing up

Regulation of learning processes

Proactive Interactive Retroactive

4

Tensions arise between summative and formative functions, especially in terms of instruments, methods, and agents

The role of teachers5

Teachers’ exclusion from summative functions of assessment weakens the kinds of inferences that can be made

But, inclusion of teachers brings its own issues Poor construct definition makes assessment design

more of a discretionary process than it should be Teachers model their assessments on tests “Summative drives out formative” Alignment of curriculum, assessment and instruction

Case studies (Australia, England, Scotland, Sweden)

Gathering evidence of student understanding

Margaret Heritage

Eliciting evidence7

Two fundamental purposes of assessment provide information on current achievement inform future instruction

Sources and quality of evidence8

Sources of evidence What learners say, write, make, or do (Griffin 2007) Interactions (questions, discussions) Tasks

Directed (model-eliciting activities) Partially directed (activities) Undirected (observations)

Quality of evidence Construct underrepresentation Construct-irrelevant variance

systematic random (i.e., unreliability)

Learning progressions9

“What gets better when someone gets better” Two key aspects

1. progressions lay out in successive steps, increasingly more sophisticated understandings of core concepts and principles in a domain, and

2. progressions describe typical development over an extended period of time.

Two approaches to their development Top-down (theoretical) Bottom-up (empirical)

Feedback and instructional correctives

Dylan Wiliam

Reviews of research on feedback

Fuchs & Fuchs (1986) Natriello (1987) Crooks (1988) Bangert-Drowns et al. (1991) Dempster (1991, 1992) Elshout-Mohr (1994) Kluger & DeNisi (1996) Black & Wiliam (1998)

Nyquist (2003) Brookhart (2004) Allal & Lopez (2005) Köller (2005) Brookhart (2007) Wiliam (2007) Hattie & Timperley (2007) Shute (2008)

11

Feedback: an evolving concept (Brookhart, 2007)

Conceptualization

Information about the learning process…

… that teachers can use for instructional decisions…

…and students can use to improve performance…

…which motivates students

Source(s)

Scriven (1967)

Bloom, Hastings and Madaus (1971)

Sadler (1983; 1989)

Natriello (1987); Crooks (1988); Black and Wiliam (1998); Brookhart (1997)

12

Back to the future13

“These considerations of utility and alternative interventions suggest that even [a feedback intervention (FI)] with demonstrated positive effects on performance should not be administered whenever possible. Rather, additional development of [feedback intervention theory] is needed to establish the circumstance under which positive FI effects on performance are also lasting and efficient and when these effects are transient and have questionable utility. This research must focus on the processes induced by FIs and not on the general question of whether FIs improve performance—look at how little progress 90 years of attempts to answer the latter question have yielded.” (Kluger & DeNisi, 1996 p. 278)

Dual pathway theory (Boekaerts)14

“It is assumed that students who are invited to participate in a learning activity use three sources of information to form a mental representation of the task-in-context and to appraise it:

(1) current perceptions of the task and the physical, social, and instructional context within which it is embedded;

(2) activated domain-specific knowledge and (meta)cognitive strategies related to the task; and

(3) motivational beliefs, including domain-specific capacity, interest and effort beliefs.” (Boekaerts, 2006 p. 349)

15

As a result of the appraisal, the student activates energy and attention along one of two pathways the growth pathway (increasing competence) the well- being pathway (prevent harm, threat or loss)

Integration of other theories Mindset (Dweck, 2000) Mastery and performance goals (Dweck, 2000) Interest (Hidi & Harackiewicz, 2000) Self-regulated learning (Deci & Ryan, 1994)

Examining formative feedback in the classroom context:New research perspectives

Maria Araceli Ruiz-Primo and Min Li

Formative assessment in the classroom17

Updated review of research on feedback in classroom learning Vast majority of studies are poorly designed

No control groups Feedback effects evaluated in the instructional session No evidence of pedagogical orientation (e.g., orientations

to the purpose of feedback, or how it could be used)

A new, expanded, notion of feedback18

Feedback should: Be seen as a process guided by the learning goals towards

which the teacher and students work Actively involve students in the process Be considered as an instructional scaffold that goes beyond

written or oral comments. Be specifically intended to improve learning outcomes Ensure its usefulness by making feedback accessible and

practical. Consider different sources of information Demonstrate, over time, alignment with a learning trajectory

Future priorities19

Defining feedback Understanding variability in feedback practices Understanding feedback in use Reviewing feedback research from new theoretical

understandings

Major unresolved questions20

Can teachers be involved in the summative assessment of their students without compromising the potential of formative assessment to improve achievement?

How can learning progressions be developed? How can we better theorize the impact of

feedback interventions?