Learning from History: Assessment & Improvement of Student Learning Lorrie A. Shepard University of Colorado at Boulder National Center for Research on Evaluation, Standards, and Student Testing (CRESST) CRESST Conference UCLA September 7-8, 2005 Previous Studies of Historical Themes The role of testing in reform: 5 decades 50s tracking, 60s accountability, 70s minimum competency, 80s school accountability, 90s standards-based reform (Linn, 2000, ER) Behaviorist learning theory underlying criterion- referenced testing and teaching to the test (Shepard, 1991) Historical conceptions of differentiated curriculum and scientific measurement out of sync with current conceptions of teaching and learning (Shepard, 2000) Three cycles of test-based reform. Each corrected narrowing effects of previous reforms but continued the use of external tests to incentivize change. (Shepard, 2002) Past Efforts to Connect Assessment and Instruction E. F. Lindquist. (1951). Educational Measurement. the functions of educational measurement are concernedwith the facilitation of learning (Cook, 1951). educational measurement is conceived, not as a process quite apart from instruction, but rather as an integral part of it (Tyler, 1951). Our Measurement Forefathers Made 3 Major Mistakes They trusted that objective formats were sufficient to represent important learning goals. They did not foresee the important differences between day-to-day formative assessment in classrooms and once-per- year formative program evaluation. They assumed (erroneously) that knowing what students didnt know would be enough to know what to do about it. Our assessment reform efforts over the past 2 decades have been aimed chiefly at correcting these errors. In the 1980s when research began to show the negative effects of test- driven reformsi.e. curriculum distortion and test score inflation Curriculum experts and measurement specialists in the U.S. responded with authentic assessments and performance assessments. In Australia, the U.K., and New Zealand, they responded with the classroom processes of formative assessment. Our Measurement Forefathers Made 3 Major Mistakes (Mistake # 1) They trusted that objective formats were sufficient to represent important learning goals. (Mistake # 2) They did not foresee the important differences between day-to-day formative assessment in classrooms and once-per-year formative program evaluation. They assumed (erroneously) that knowing what students didnt know would be enough to know what to do about it. Wyoming Body Evidence Activities Consortium Researchers in England, Australia, New Zealand, Scotland, & Wales opposed standardized tests by focusing on formative assessment. (Crooks, 1988). Reviewed motivational and cognitive effects of classroom assessment. (Sadler, 1989). Proposed a model of formative assessment including feedback and self-monitoring. (Assessment Reform Group, 1999). Promoted the idea of assessment for learning. Paul Black & Dylan Wiliam (1998). Assessment and Classroom Learning Assessment becomes formative assessment when evidence is actually used to adapt the teaching work to meet student needs. Formative assessment experiments produce effect sizes of , larger than found for most educational interventions. Many studies show that improved formative assessments help low achievers most. Knowing What Students Know Pellegrino, Chudowsky, & Glaser. (2001). NRC. Cognitive science findings on key aspects of learning processes can be translated into targeted features of formative assessment: Accessing prior knowledge Strategic use of feedback Teaching and assessing for transfer Meta-cognitive benefits of self-assessment Shepard (2000). The Role of Assessment in a Learning Culture. We have to change the social meaning of evaluation. Dynamic assessment and instructional scaffolding in Vygotskys ZPD are essentially the same processes. Developing an identity of mastery occurs as learners participate in a community of practice (Lave & Wenger, 1991). Self-assessment also increases students responsibility for their own learning and makes the student-teacher relationship more collaborative (Gipps, 1999). Subject-matter experts blended content and process reforms. Marie Clay (1985). Invented assessment strategies embedded in the acts of reading. Yetta Goodman (1985). Reintroduced the concept of kidwatching. Teal, Hiebert, & Chittenden (1987). Collected samples of student work to gain insight into childrens thinking and to document progress over time. NCTM Standards (1989) and Everybody Counts (NRC, 1989). Learning mathematics as a process of inquiry and sense making, not mindless mimicry. More extended, non-routine problems needed to engage students and to assess mathematical power. Classroom discourse became a focus to provide students with the opportunity to conjecture and explain their reasoning. Embedded, informal assessments observations, teacher questioning, and journal writing to gain insights into students thinking. Curriculum-embedded assessments are much more likely to produce coherent and instructionally relevant assessment insights. Most importantly they help teachers gain qualitative insights about student understandings to build on student strengths. In contrast, formal benchmarking systems mean administering earlier and earlier versions of external tests at quarterly or monthly intervals. The result is a long list of discrete skill deficiencies requiring inexperienced teachers to give a 1000 mini lessons. These systems threaten to hijack formative assessment reform efforts by repeating mistakes 1 and 2. Our Measurement Forefathers Made 3 Major Mistakes They trusted that objective formats were sufficient to represent important learning goals. They did not foresee the important differences between day-to-day formative assessment in classrooms and once-per- year formative program evaluation. They assumed (erroneously) that knowing what students didnt know would be enough to know what to do about it. What about mistake # 3 as yet unaddressed? Curriculum-embedded assessments help if teachers have the training to make qualitative judgments and to build on existing knowledge. Curriculum-embedded or benchmark assessments help if they are built to uncover specific misconceptions. Learning progressions go further than isolated assessment tasks because, by definition they are tied to a model of developing competence. Beyond Curriculum-Embedded Assessments, Learning Progressions answer the question about what to do. An understanding of learning progressions or learning trajectories is important for monitoring and supporting learning and development over time. How learning typically unfolds helps teachers know what next and how to back up ( though we must also be aware of natural variations and departures from the typical pattern ). Learning progressions or Progress Maps also provide an underlying model of learning to coherently link classroom and large-scale assessments (KWSK). Geoff Masters & Margaret Forster (1996). Developmental Assessment Progress maps describe skills, understandings, and knowledge in the sequence in which they typically develop: a picture of what it means to improve in an area of learning. North Carolina Childrens Writing Continuum Strategies in Children's Spelling at Different Stages Prominent Strategy DescriptionExample PrephonemicLetters are used to write words but the sound-symbol relationships are unrelated to target word. "'C" for "hat" Early phonemicSome phonemes are represented by letters, typically most salient phoneme(s) in a word. "DR" for "Dear" PhoneticAttempts are made to represent most sounds in words, often letter name that most closely resembles sound. "wns" for "once" Simple AssociationsSimple vowels and consonants are represented correctly but complex patterns are not. "bid" for "bird" Strategic ExtensionsWith complex vowels and consonants, attempts reflect complex English patterns, although not the conventions of English. "bote" for "boat" Conventional Hiebert and Raphael (1998) Marja van den Heuvel- Panhuizen (2001). In the Netherlands, learning-teaching trajectories are being developed to provide pedagogical insights needed to support the development of students thinking over time. 3 interwoven components: A learning trajectory describes the learning process. A teaching trajectory links teaching strategies with the learning process. A subject matter outline reflects the core elements of the mathematics curriculum that should be taught.