Assessment of Reading

Reading Assessment

Solving Problems in the Teaching of LiteracyCathy Collins Block, Series Editor

Recent Volumes

Success with Struggling Readers: The Benchmark School ApproachIrene West Gaskins

Making Sense of Phonics: The Hows and WhysIsabel L. Beck

Reading Instruction That Works, Third Edition:The Case for Balanced Teaching

Michael Pressley

Narrowing the Literacy Gap: What Works in High-Poverty SchoolsDiane M. Barone

Reading Assessment and Instruction for All LearnersEdited by Jeanne Shay Schumm

Word Sorts and More: Sound, Pattern, and Meaning Explorations K3Kathy Ganske

Reading the Web: Strategies for Internet InquiryMaya B. Eagleton and Elizabeth Dobler

Designing Professional Development in Literacy:A Framework for Effective Instruction

Catherine A. Rosemary, Kathleen A. Roskos, and Leslie K. Landreth

Best Practices in Writing InstructionEdited by Steve Graham, Charles A. MacArthur, and Jill Fitzgerald

Classroom Literacy Assessment:Making Sense of What Students Know and Do

Edited by Jeanne R. Paratore and Rachel L. McCormack

Fluency in the ClassroomEdited by Melanie R. Kuhn and Paula J. Schwanenflugel

Reading Assessment, Second Edition: A Primer for Teachers and CoachesJoAnne Schudt Caldwell

ReadingAssessment

A Primer for Teachers and Coaches

SECOND EDITION

JoAnne Schudt Caldwell

THE GUILFORD PRESSNew York / London

2008 The Guilford PressA Division of Guilford Publications, Inc.72 Spring Street, New York, NY 10012www.guilford.com

All rights reserved

Except as noted, no part of this book may be reproduced, translated, stored in aretrieval system, or transmitted, in any form or by any means, electronic,mechanical, photocopying, microfilming, recording, or otherwise, without writtenpermission from the Publisher.

Printed in the United States of America

This book is printed on acid-free paper.

Last digit is print number: 9 8 7 6 5 4 3 2 1

LIMITED PHOTOCOPY LICENSE

These materials are intended for use only by qualified professionals.

The Publisher grants to individual purchasers of this book nonassignable permissionto reproduce all materials for which photocopying permission is specifically grantedin a footnote. This license is limited to you, the individual purchaser only, for usewith your own clients or students. Purchase by an institution does not constitute asite license. This license does not grant the right to reproduce these materials forresale, redistribution, or any other purposes (including but not limited to booksand handouts or slides for lectures or workshops). Permission to reproduce thesematerials for these and any other purposes must be obtained in writing from TheGuilford Press.

Library of Congress Cataloging-in-Publication Data

Caldwell, JoAnne (JoAnne Schudt)Reading assessment : a primer for teachers and coaches / by JoAnne Schudt

Caldwell. 2nd ed.p. cm.

Includes bibliographical references and index.ISBN-13: 978-1-59385-579-6 (pbk. : alk. paper)ISBN-10: 1-59385-579-6 (pbk. : alk. paper)ISBN-13: 978-1-59385-580-2 (hardcover : alk. paper)ISBN-10: 1-59385-580-X (hardcover : alk. paper)1. ReadingAbility testing. I. Title.LB1050.46.C23 2008372.48dc22

2007021755

About the Author

JoAnne Schudt Caldwell, PhD, is a Professor at Cardinal Stritch University inMilwaukee, Wisconsin. She is presently the Associate Dean of the College ofEducation and Leadership. Dr. Caldwell is coauthor, with Lauren Leslie, of TheQualitative Reading Inventory 4, an informal reading inventory, and InterventionStrategies to Follow Informal Reading Inventory Assessment: So What Do I Do Now?She is also an author, with Joyce Holt Jennings and Janet W. Lerner, of ReadingProblems: Assessment and Teaching Strategies.

v

Preface

When the first edition of this book was published in 2002, assessment was a bit of astepchild. Although it was acknowledged as an important component in educa-tion, the primary focus was on instruction. As a result, assessment waited in thewings while instruction basked in the spotlight. But in January 2002, the No ChildLeft Behind (NCLB) Act was signed into law, and a lot of things changed. Schoolsmust now demonstrate adequate yearly progress (AYP) for all students or risk seri-ous consequences. How else can AYP be demonstrated unless through the processof assessment? And so assessment moved out onto center stage.

Much of the content of the 2002 edition of this book remains timely to teach-ers and coaches, but the changing face of assessment has required the addition ofcertain components. Information about the construction, scoring, and function ofstandardized tests has been updated and expanded. Because accountability to theNCLB law rests primarily with scores on standardized tests, these high-stakes mea-sures are having a monumental effect on schools, school districts, teachers, coaches,and parents. Such tests certainly have their critics; they also have their supporters.Accordingly, the chapter on standardized tests has been expanded to address theconsequences of the NCLB Act, AYP, and arguments for and against high-stakestesting. What is perhaps more meaningful to teachers and coaches, it contains prac-tical suggestions for dealing with the reality of such measures.

Assessment involves more than standardized measures. Most assessmenttakes place in the classroom and is conducted by teachers on a daily and ongoingbasis. However, such assessment is highly individualistic and idiosyncratic acrossteachers, coaches, and schools. As such, classroom assessment presently offers aweak alternative to standardized measures. That is, classroom assessment doesnot offer either a validation of or an alternative to standardized scores. In otherwords, if standardized scores are low, classroom assessments cannot be used as anargument that the standardized measures might be in error. Accordingly, a newchapter has been added to consider the possibility that under certain circum-stances, classroom assessment can be used and reported to school boards and thegeneral public in much the same way as standardized measures are. This chaptercontains suggestions for raising the validity and reliability of classroom assess-ments and for organizing and reporting classroom literacy data.

vii

Early literacy has become a big business since the first edition of this book.The availability of federal Reading First funds has sparked the construction of avariety of early literacy measures, such as the Dynamic Indicators of Basic EarlyLiteracy Skills (DIBELS), the Early Reading Diagnostic Assessment (ERDA),and the Phonological Awareness Literacy Screening (PALS). A fair amount ofcontroversy presently surrounds Reading First assessment measures, and it is notthe role of this book to take a stand one way or another. The purpose of thisbook is to offer information to teachers and coaches about informal and formalliteracy assessments. Appendices have been added to Chapters 47, listing andbriefly describing the many published assessments currently available. In addi-tion, guidelines for selecting and using such published assessment batteries havebeen provided.

All chapters have been updated as needed, often by the inclusion of sugges-tions from prior users. However, the overall purpose of the book remainsthe same: to provide an informative and practical resource for teachers andcoachesone that addresses the busy and complex nature of todays classroom.Like the first edition, this book is an effort to provide simplicity and practicalityfor teachers, coaches, and staff developers, but within a framework of basic the-ory and research. Assessment is conceptualized as involving a limited number ofgood reader behaviors that cross the key areas of early literacy, word identifica-tion, fluency, vocabulary, and comprehension. These good reader behaviors pro-vide a focus for assessment and can be easily tied to state and curriculum stan-dards. Because they represent a manageable number, teachers and coaches canbecome more focused and specific in their assessment efforts.

I consider the audience for this book to include all teachers or coaches, butparticularly those who are just beginning their careers as educators. The teachershortage and low teacher retention rates, especially in urban areas, determinethat more and more new teachers will enter the profession after an abysmallyshort induction period. Once they secure a teaching position, there is no cer-tainty that they will be provided with effective mentoring or professional devel-opment support. In most cases, they will do as too many teachers have donebefore them: enter their classrooms, close their doors, and struggle in solitudewith myriad problems and concerns. I hope that this book can help them to dealrealistically with assessment of literacy performance.

This book also represents a primer approach; that is, it offers first steps forteachers and coaches to take as they begin to pay more attention to the possibili-ties and pitfalls of classroom assessment. In this respect, the second edition hasnot changed: It provides a starting point for new skill development. Effectivestarting points are simple in nature, but provide the basis for later complexity. Asa primer, this book is designed to help a teacher or coach take manageable firststeps in improving classroom literacy assessments. Later, with evidence of grow-ing ability and success in assessment, the teacher or coach will naturally move tomore complex instruments and processes.

viii Preface

Contents

ONE Overview of the Reading Process: What Do Good Readers Do,and How Do Teachers and Coaches Assess This?

1

TWO Assessment as Part of Instruction: How Can We Assessas We Teach?

15

THREE The Informal Reading Inventory Process: How Does It Addressthe Four Purposes of Reading Assessment?

49

FOUR Early Literacy: What Do We Need to Knowabout Beginning Readers?

93

FIVE Word Identification: How Can We Assess a Students WordIdentification Ability?

131

SIX Reading Fluency: How Can We Assess Reading Fluency? 162

SEVEN Comprehension of Words and Text: How Can We AssessUnderstanding of Words and Text?

175

EIGHT Motivation: What Makes Students Want to Read, and HowCan We Assess This?

220

NINE Standardized Tests: What Do They Tell Usabout Reading Performance?

232

TEN Schoolwide Reading Assessment: How Can We Collect,Organize, and Present Classroom Data?

249

ix

General Summary: Assessing the Good Reader Behaviors 267

Overview of Published Literacy Assessments 271

References 275

Index 287

x Contents

ONE

Overviewof the Reading ProcessWhat Do Good Readers Do, and How Do

Teachers and Coaches Assess This?

CHAPTER TOPICS

Purpose and Scope of the Book 1Description of the Four-Step Assessment Process 2Purposes of Reading Assessment 4Identifying Good Reader Behaviors and Areas

of Weakness5

Determining Reading Level 9Noting Student Progress 11Choosing Evidence for Assessing Reading Performance 12Summary 13Activities for Developing Understanding 14

Purpose and Scope of the Book

The first months and years of teaching can be incredibly overwhelming. Teachersand coaches face a multitude of different tasks each day. They must cope with acomplex array of issuescurriculum, classroom organization, classroom man-agement, and the needs of individual students, to name only a few. They have tomake decisions about what to teach and how to teach it. They must evaluatewhat their students have learned, and must assess their own effectiveness asteachers. When a carefully constructed lesson plan fails in the first moments ofits presentation, a teacher or coach has to make immediate adjustments andmove on. When a teachable moment occurs, the teacher or coach must recog-nize it and modify planned activities to take advantage of an unexpected oppor-

1

tunity for student learning. Every day in the classroom is different. Every daypresents new challenges. It is not surprising that many teachers and coaches feelbesieged by the incredible complexity of teaching.

This book represents a first step in reducing some of the complexity that sur-rounds teaching and assessing reading. It is a primer, and as such, it offers anintroduction to the subject of reading assessment. This book is only a beginning.It offers the teacher or coach some simple guidelines for effectively assessing thereading process. Reading is an extremely complex and multifaceted process, andassessing this process is also complex and multifaceted. However, a teacher orcoach must start somewhere, and this primer represents a starting point.

Think about other complex processes, such as knitting, woodworking, ordeveloping computer expertise. The first steps in learning any process are small.However, confidence grows with each baby step that is successfully mastered.With confidence comes the willingness to move to more complexity and greaterdepth. The learner eventually takes giant steps. A beginning knitter does notattempt a complex pattern involving a variety of different stitches, but insteadchooses a simple model. Later, after success with simple patterns, the knitter willcompetently handle complex designs. First computer efforts involving simpleword-processing strategies may gradually extend to advanced formatting tech-niques, the use of clip art, and perhaps the construction of spreadsheets.Learning to assess a students growth as a reader is no different. The teacher orcoach begins with small stepswith relatively simple strategies. These becomethe basis for developing increasing skill and competence in assessing reading.

This book describes a variety of beginning steps that a teacher or coach cantake to develop expertise in the complex process of reading assessment. Someassessment strategies are described in detail, with instructions for implementingthem in the reading classroom. Others are mentioned for the purpose of expand-ing the teacher or coachs knowledge base, but with no expectation that they willbe put into practice at this stage of his or her career. They represent giant steps,and the premise of this book is that teachers and coaches must begin with thesimple and move gradually to the complex. The primary purpose of this book isto guide the teacher or coach through first steps in developing an effective systemfor assessing reading.

Description of the Four-Step Assessment Process

This is a book about reading assessment, so it makes sense to describe first whatwe mean by assessment. Assessment is so much a part of our daily lives that weseldom think about it. We tend to take it for granted, and we just do it! Whenwe assess, we collect evidence. Then we analyze this evidence so that we canmake a judgment about something. This judgment generally leads to a decision

2 READING ASSESSMENT

or to some form of action. For example, we assess the weather in the morning.Our evidence may be what we see as we look out the window, the local weatherreport, or a combination of the two. Based upon this, we decide to wear certainclothes or take an umbrella. We assess our appearance by looking in the mirror,and as a result of the evidence, we may change a tie or add a scarf. We use the gasgauge in the car to assess the state of the gas tank, and decide either to stop forgas now or wait until after work. We assess the amount and flow of traffic on thefreeway, and perhaps turn off at the next exit. We assess the mood of our cowork-ers by listening to what they say, considering their tone of voice as they say it,and noting their body language. As a result, we either attempt to meet theirneeds, or we decide to avoid them for the rest of the day.

Assessment involves four steps. First, we identify what we want to assess. Weusually do this in the form of a question. For example, as we move out of thedriveway, we might ask, Do I need to stop for gas before driving to work? Sec-ond, we collect information or evidence. We check the gas gauge. If it is very low,we may estimate how many miles we can drive before it hits empty. Will stop-ping for gas make us late for work? If we are late, will this cause any problems?Third, we analyze the evidence. The gas gauge is low, but not low enough to pre-vent us from getting to work. We would be late if we stopped for gas, and wehave a conference with a very concerned parent. Fourth, as a consequence of ouranalysis, we make a decision. We decide to stop for gas after work, not before.

Most of the time, the assessment process runs so smoothly that we tend totake it for granted. However, when the process fails us, we become very awarethat something has gone wrong. We can run into problems with any of the foursteps. Sometimes we are not specific enough about what we are assessing. Whatis the weather going to be like today? may be less effective than a more specificquestion: Will it rain today? Of course, the effectiveness of any assessment isheavily dependent upon the quality of the evidence we select. A glance out thewindow may provide less sensitive evidence about the weather than a weatherreport may. Sometimes our analysis of the evidence is faulty. Does the presenceof gray clouds always indicate rain? Should we place more weight on a glance outof the window or the meteorologists predictions? Finally, we can make a wrong

Overview of the Reading Process 3

THE FOUR STEPSOF THE ASSESSMENT PROCESS

Identify what to assess.

Collect evidence.

Analyze the evidence.

Make a decision.

decision. Anyone who has ever decided not to take an umbrella on a day thatbecame very rainy understands what that means!

Assessing a students reading performance is no different. We ask a questionabout a students reading. We select evidence that is appropriate for answeringour question. We analyze the evidence and use it to make judgments about thestudents strengths and needs. Then we take instructional action. At any point inthe process, we can encounter the same problems we meet while assessing theweather or the state of the gas tank. However, reading assessment poses severaladditional problems.

First, we may not know as much about the processes of reading as we doabout weather and gas gauges. Reading is an incredibly complex act! Good read-ers engage in a variety of different activities as they comprehend a selection.These activities are very different if the reader is a first grader who is just begin-ning to make sense of print, a fourth grader dealing with a social studies text-book or project, or an adult struggling with directions for filling out income taxforms. It is not enough to ask, How well is the student reading? That questionis much too general. Instead, we need to know what specific student behaviorsindicate good reading and at what level of development.

Another important issue is the type of evidence we choose to documentgood reader behaviors. There are only so many options for assessing the weatheror the state of the gas tank. However, there are many published instrumentsavailable for reading assessment. In addition, we can construct our own mea-sures. It is difficult to sort through all the possible instruments and judge theirquality. Some instruments may not be suitable for answering specific questionsabout a students reading performance. And often, when we collect evidence, wedo not know what to do with it. If we note that the gas gauge is almost on empty,we know exactly what to do. It is quite another thing to deal with a variety ofscores or observations, some of which may conflict with each other, and none ofwhich suggest a clearly defined direction for instruction. We need guidelines forselecting evidence of good reader behaviors and for using this evidence to designinstruction. This book provides such guidelines.

Still another issue is that the stakes for reading assessment are very high. Ifwe make a wrong assessment about the weather or gas, we may just get wet orspend an annoying hour walking to and from a gas station. But a wrong orincomplete reading assessment can have serious consequences for a student.

Purposes of Reading Assessment

There are four basic purposes of reading assessment. First, a teacher or coach usesthe assessment process to identify the good reader behaviors a student displays.Readers are not passive. They engage in a variety of activities as they construct


meaning. Teachers and coaches must have an understanding of these processes,so they are equipped to select valid evidence that documents good reading.

Second, a teacher or coach must identify areas of weakness with regard to thegood reader behaviors, in order to align instruction with student needs. If theteacher or coach recognizes which good reader behaviors are absent or weak, heor she can design and focus instruction to introduce or strengthen them.

Third, teachers and coaches need to know how to determine whether a spe-cific book is too difficult for a student. They also need to know how to deter-mine a students reading levelthat is, the grade level at which a student can readin an acceptable fashion. Knowing a students reading level allows us to chooseappropriate reading material for the student to read on his or her own, as well asappropriate material for instruction. In addition, comparing a students readinglevel with his or her chronological grade level can suggest the existence of a read-ing problem and how serious it may be. For example, a fifth grader who can onlyread second-grade material is probably experiencing quite a bit of difficulty inschool.

Last, teachers and coaches need to document evidence of progress on thepart of the student. Unfortunately, much assessment compares the student to hisor her peers. Often, for a student who is reading below the level of peers, thisobscures any progress that the student may have made. A teacher or tutor needsto compare a student to him- or herself. Let us examine each of these compo-nents in turn.

Identifying Good Reader Behaviors and Areasof Weakness

What is this process called reading? What are readers doing as they read, andwhat do they have to learn? As teachers and coaches, what should we look for aswe assess the developing literacy of our students? We have all been successfulreaders for many years. As a result, we seldom think about the act of reading; infact, we tend to take it for granted. Lets engage in a short exercise to help us real-ize what we do so effortlessly every day of our lives as we read newspapers, maga-


PURPOSES OF READING ASSESSMENT

Identify good reader behaviors.

Identify areas of weakness.

Determine student reading level.

Document student progress.

zines, grocery lists, road signs, novels, memos from the principal, and notes froma parent, to name only a few things.

I am going to ask you to read a short selection sentence by sentence andexamine what went on in your mind as you read. Ready?

Peter led Bridget into the waiting room.

What did you do as you read this sentence? I am certain that you are primar-ily conscious of a mental picture of Peter and Bridget. But before you could formthis picture, you performed a variety of other actions. You recognized the indi-vidual words, and you assigned some form of meaning to each one. You did thisaccurately and very quickly. In addition to constructing an image of Peter andBridget, you probably made some inferences, predicting perhaps that the waitingroom was in a doctors office or maybe a bus station. How could you do this? Youhave some prior knowledge of waiting rooms, and you used this to make thisprediction. You may have also predicted that you are reading a story. If the firstsentence had been There are a variety of ways in which viruses differ fromcells, you would have predicted a nonfiction passage. You can do this becauseyou have a sense of text structure and of how narratives differ from expositoryselections. You no doubt have questions. How old were Peter and Bridget? Whywas he leading her? What could happen next? These questions lead you to readon to find answers.

He realized that she was extremely nervous, so he gentlysuggested that she sit down.

You did the same things with this sentence, but in addition, you connectedit to the first sentence. Good readers synthesize information as they read. Perhapsthe fact that Bridget was nervous confirmed your prediction that they were in adoctors office. If you had originally thought that they were in the waiting roomof a bus station, perhaps the word nervous led you to think of another possibility,such as a dentists office. You have added to your visual image, and I suspect youare feeling quite positive toward Peter and his gentle ways. Developing emotionstoward a character or situation is an important component of the reading pro-cess.

Bridget ignored him and began to pace frantically.

You have more questions than you did before, and you are using your priorknowledge and the clues in the text to predict possible answers. You are monitor-ing your comprehension. You are aware of your questions. You know what youare unsure of. Why did Bridget ignore Peter? Could they have had an argument?


She was nervous, and nervous people might pace, but why was her pacing sofrantic? You are probably considering several possible reasonsand, I might add,enjoying the suspense. Why do we stay up way past bedtime reading a book? It isbecause we have many questions about what will happen, and we want to findthe answers. Otherwise, we know that sleep will elude us!

The other patients watched her warily, and several alsobegan pacing.

Perhaps one of your inferences was confirmed. The setting was a doctors ordentists office. As a good reader, you matched your knowledge of waiting roomsto the clue patients and arrived at an answer. Perhaps you are unsure of the exactmeaning of warily. What did you do? I doubt that you reached for a dictionary.Caught up with the story action, you probably used the context of the story toassign a temporary meaning, such as carefully or suspiciously. You then read on,eager to find out what will happen next.

As a scream rang out from the inner office, Peter angrily forcedBridget to sit down.

The plot has thickened! Do you like the angry Peter quite as much as thegentle Peter? Who was screaming and why? What did this scream have to dowith Bridget? You are not conscious of identifying words and assigning mean-ings. You are caught up with what is happening! You are very aware of what youunderstand and of the questions you have. You want answers! You are probablyeven impatient with my comments, which are interrupting the account of Peterand Bridget. So lets move on.

Bridget moved closer to Peter, who leaned down and tenderlyscratched her ears.

Were you surprised? And if you were, wasnt it fun? Had you considered thepossibility that Bridget was a dog? Had a vets office even entered your list ofwaiting room possibilities? If you are a dog owner, Ill bet the facts that Peter ledBridget into the room and that the other patients copied her pacing were power-ful clues.

Lets summarize. What did you do as you read these few short sentences?First, you identified individual words. Most (if not all) were familiar to you, andyou did not need to match letters and sounds. The words were what we call sightwordswords that are recognized immediately without analysis. Warily mayhave been unfamiliar, but as a good reader, you were able to sound it out andpronounce it even though you had never seen it in print before. You assigned


appropriate meanings to the words you identified. You connected the things youknew about waiting rooms, patients, and ear scratching with the information inthe text. You made inferences and predictions based on your knowledge and theclues in the text. You asked questions and located their answers. You constructedvisual images. You monitored your own comprehension or lack of it. In short,you were extremely active, and your sole focus was upon constructing meaningand finding out what was going on.

These key reading behaviors are the basis for reading assessment. We need tofind out whether readers are able to do what you did so effortlessly. If they can-not, we need to teach them strategies for doing so. And it is not enough to iden-tify which good reader behaviors to assess. We need to assess these in differentforms of text. You were extremely successful with the Peter and Bridget scenario,but would you have been as capable reading an insurance form, a mortgageagreement, or a statistics textbook? Students may employ good reader behaviorswhen reading stories, but may become hopelessly confused in a social studies orscience textbook. We must help them to employ effective strategies in a wide


GOOD READER BEHAVIORS

Before, during, and after reading, good readers . . .

Use letter and sound patterns to pronounce unfamiliar words.

Pronounce words accurately.

Pronounce words automatically.

Read fluentlythat is, accurately, quickly, and expressively.

Attach meaning to words.

Learn new words and refine the meanings of known ones.

Connect what they know with the information in the text.

Determine what is important in the text.

Recognize the structure of the text.

Summarize and reorganize ideas in the text.

Make inferences and predictions.

Construct visual images.

Ask questions of themselves and the author, and read to find answers.

Synthesize information from different sources.

Form and support opinions on ideas in the text.

Recognize the authors purpose/point of view/style.

Monitor comprehension and repair comprehension breakdowns.

Choose to read for enjoyment and enrichment.

variety of reading situations. We will return to the good reader behaviors in eachchapter of this book. The Good Reader Behaviors box provides a list thatshould be a focus for both instruction and assessment. We will return to this listagain and again.

Determining Reading Level

Determining reading level involves two tasks. One is to determine whether a stu-dent can successfully read a specific selection. Texts that are used in a classroom,or those that are available to students for independent reading, can vary widelyin difficulty level. The topic of the text, the presence or absence of pictures, thelength of the text, and the vocabulary that is used are just some of the factors thatcan make one selection more difficult than another. The teacher or coach needsto know which texts the student can handle independently. Which ones can thestudent read and understand if given support? Which ones represent a frustratingexperience for the student?

The teacher or coach also needs to have an estimate of the students generalreading level. Can the student handle most of the selections that are used at hisor her grade level? What is the first indication of a reading problem? It is gener-ally that a student cannot read as well as his or her classmates. If a third gradercan read and comprehend selections that are appropriate for third grade, we saythat this student is reading at grade level. A third grader who can read and com-prehend selections appropriate for fifth grade is reading above grade level. Onewho is only comfortable with a first-grade selection is reading below grade level.An important category of reading assessment is determining what general level oftext the student can read successfully.

The seriousness of a reading problem often depends on the gap between astudents reading level and his or her chronological grade level. A third graderreading at a second-grade level may have a less severe reading problem than afifth grader reading at that same level. How big a discrepancy signals the possibil-ity of a problem? Spache (1981) has offered the following guidelines. For firstthrough third graders, a difference of one year or more between grade placementand reading level is cause for concern. For fourth through sixth graders, a differ-ence of two years or more warrants concern. For students in seventh grade andabove, a difference of three years or more is a signal that a reading problem exists.The younger the student, the less difference is needed to indicate a severe prob-lem. Why is it important to consider severity? Students with severe reading prob-lems often need more concentrated intervention in the form of daily and/or indi-vidual classes.

What makes a selection appropriate for one grade and not for another? Inother words, how do we decide that a selection is at a specific grade level? One


way to identify the grade level of a selection is to use a readability formula. Read-ability formulas are based upon the premise that longer sentences and longerwords make text more difficult. These formulas count such things as the numberof words in a sentence, the number of syllables in the words, and the number ofwords that are not considered common or frequent. There is software that willhelp you do this and you can also do it by hand. However, it is very time-consuming, and busy teachers and coaches generally leave it to others to fix gradelevels through readability formulas.

Readability formulas have serious limitations (Zakaluk & Samuels, 1988).Various factors beyond sentence and word length interact to make a selectioneasy or difficult, or appropriate for one grade level and not for another. Readabil-ity formulas do not take account of these factors. The presence or absence of pic-tures can make a text easy or difficult. Predictable text with often-repeatedrefrains or rhyme is generally easier and more appropriate for the lower grades.Text structure is another factor. Narratives are easier to understand than exposi-tory text. Page layout and the presence or absence of headings and other graphicaids are other considerations. Text coherence also influences difficulty level.Coherent texts are well organized, and authors clarify how each new piece ofinformation is related to what has already been presented. They signal the intro-duction of new topics and organize ideas according to importance. In addition, areaders prior knowledge is a powerful determinant of text difficulty. A studentwho knows quite a bit about the topic of the text will find it easier to read andunderstand than an unfamiliar text at the same readability level.

Given all of these determinants of text difficulty, how do teachers or coacheschoose appropriate texts for instruction? Often they have little choice withregard to these texts. Many schools and districts employ reading anthologies,often referred to as basal readers. Publishers of such anthologies generally do anacceptable job of matching selections to appropriate grade levels. Even if ateacher or coach can choose instructional texts, he or she is probably too busy touse readability formulas or extended analyses of text features in order to deter-mine whether a text is appropriate for a group of students. Sometimes the diffi-culty level of a book is indicated. If it is not stated, the teacher or coach may beable to locate other sources for estimating difficulty level, such as publisher cata-logues. However, this takes time, and time is a precious commodity!

A simple but effective way to choose an appropriate selection is to examine itin relation to other selections that your students have read and enjoyed. Wouldyour students be interested in the topic? Is it a relatively familiar topic? Does thenew text look like past selections (as far as length, number of pictures, size ofprint, etc., are concerned)? Ask one or two students to read a few pages out loud,to determine whether they can handle most of the words. You can use the sameprocess to choose texts for individual students. If the text seems suitable to you,


go for it. Trust your instincts! With more and more experience, you will becomevery adept at choosing selections that are appropriate for all of your students.

Noting Student Progress

All too often, we assess students by comparing them to their peers. For example,we compare one basketball player to the teams star and say that he or she is anaverage or poor player. We compare a third grader to other readers in his orher class and say that this student is the poorest reader. In doing this, we miss avery important pointindividual progress. The poor basketball player may havestarted the year with a free-throw percentage of 20%, but now maintains a 56%average. Wouldnt you call that progress? The third-grade child may be reading ata second-grade level, but may have started the year at a first-grade level. Again,wouldnt you call that progress? The most important comparison is that of a stu-dent to him- or herself. The effectiveness of our instruction must always beexamined in relation to the individual students progress. What we should beinterested in is whether each child is growing, not whether he is better than thechild across the district or the country. We should compare a childs work to hisown work earlier in the year: our reference point should be the childs work(Johnston, 1992b, p. 250).

Gregg was a fourth grader who was having much difficulty keeping up withhis peers. When this became apparent to his classroom teacher, she notified theschool reading specialist, who agreed to work with Gregg. She gathered evidencein order to assess Greggs level of reading and the good reading behaviors heemployed. Gregg recognized most words accurately, but read very slowly andobviously needed to develop a greater measure of fluency. He was most comfort-able reading selections appropriate for second grade. In his fourth-grade class,Gregg had great difficulty reading the social studies and science textbooks, andhe became easily discouraged. The reading specialist decided to focus on activi-ties to improve Greggs reading fluency and on strategies for understanding andremembering nonfiction selections. After four months of instruction, she againcollected evidence. Gregg was reading much more fluently, which led to morecomprehension on his part. He was comfortable in third-grade material andmore confident in his reading ability. In comparison to his fourth-grade class-mates, Gregg was still the poorest reader in his classbut wouldnt you agreethat, in comparison to himself, Gregg had made a lot of progress?

It is not enough to determine the progress of an individual student withregard to reading level and the good reader behaviors. Today, in our era ofaccountability, parents, administrators, and the community want to know theextent to which all students are meeting the literacy standards set by their state.


This requires moving beyond individual scores and looking at group data. Forexample, Gregg made substantial progress as an individual student, although hedid not meet grade-level standards. But what about all fourth graders in Greggsschool? Are they meeting the required standards, or are the majority, like Gregg,progressing but still below grade-level expectations? Grouping literacy data acrossgrade levels and schools moves assessment beyond the individual level and pro-vides information about the performance of a school as a whole. Presently, theonly measure of school effectiveness that we have is annual reporting of standard-ized test scores. These have their limitations, as noted in Chapter 9, and it makesgood sense to provide an alternative to such measures. Group data at the grade orschool level could be such an alternative. We will return to this concept in Chap-ter 10.

Choosing Evidencefor Assessing Reading Performance

Many different measures can provide evidence of a students reading perfor-mance. A teacher or coach can use published instruments, or can construct his orher own instruments. Whatever the form, assessment measures should possesstwo characteristics: authenticity and trustworthiness (Valencia, 1990a). Authenticassessment assesses what we have defined and valued as real reading (Valencia,1990a, p. 60). Trustworthy assessment uses clearly established procedures forgathering evidence. Generally, we divide assessment measures into two catego-ries: formal and informal.

Formal measures are usually commercial instruments prepared by publish-ers. Many are norm-referenced tests, which are designed and often scored by indi-viduals outside the classroom (Wolf, 1993). When we use a norm-referencedtest, we compare the students score with the score of the norm samplethat is,the sample of students who were used to standardize the test. What does thatmean? To put it simply, suppose a student correctly answered 43 items. If theaverage third-grade score of the norm sample was also 43, the student would beassigned a third-grade reading level. Other formal tests are criterion-referenced:We compare the students score to a cutoff score (or criterion) set by the testauthors. Formal tests require us as teachers or coaches to follow all directions foradministering and scoring, without adapting or changing any of the procedures.Although standardized tests describe a students performance in terms of a num-ber, they often do not tell us what we really need to know in order to designeffective instruction. Chapter 9 discusses the advantages and disadvantages offormal measures in more detail.

Informal measures are assessments developed, administered, and/or scoredby classroom teachers. They can take the form of conferences, observations, and


reviews of student work (Wolf, 1993). Informal measures allow a teacher orcoach to design or modify an instrument in order to collect the type of evidencethat is needed. Informal tests are extremely flexible, and a teacher or coach canpersonalize assessment to the needs of the student. Although some informal mea-sures are published, teachers or coach es construct the majority of them.

Informal instruments are more authentic than formal measures. They aresimilar to the actual task of reading. Reading a passage and retelling its contentsare more authentic than reading a short paragraph and answering multiple-choice questions by filling in little bubbles on a scan sheet. Unfortunately, tosome individuals, the term informal suggests assessments that are subjective orcasual (Wolf, 1993). Nothing could be farther from the truth! Assessmentsdesigned and carried out by knowledgeable teachers provide the most accurateand useful information about student achievement (Wolf, 1993, p. 519). Thisbook focuses primarily on the use of informal measures to gain maximum infor-mation about how a student reads different selections for the purposes of gainingboth information and enjoyment.

Summary

Assessment is a four-step process: identifying what to assess, collecting evi-dence, analyzing the evidence, and making a decision based on the analysis.

There are four purposes for reading assessment: identifying good readerbehaviors, identifying areas of weakness, determining reading level, and notingstudent progress.

A teacher or coach identifies good reader behaviors and weaknesses inthose behaviors as a specific focus for assessment.

A teacher or coach needs to find out whether a student can successfullyread a specific selection. A teacher or coach also needs to estimate the studentsgeneral reading level.

A teacher or coach determines reading level in order to choose appropriatereading material for a student and to estimate the seriousness of a reading prob-lem.

A teacher or coach notes a students progress and compares the student tohim- or herself as opposed to classmates.

A teacher or coach recognizes the importance of group data to measureschool effectiveness and the extent to which the students are meeting state stan-dards.

Choosing evidence for assessing reading performance involves formal andinformal measures.

Informal tests provide the most sensitive and useful information to theteacher or coach.


Activities for Developing Understanding

Select something in your life that you assess frequently (grocery supplies,the organization of your desk, the state of your checkbook, etc.). Show how youractions fit into the four steps of assessment.

Select a short piece of text. Read it sentence by sentence. Stop after eachsentence. What were you thinking of as you read the sentence? Which goodreader behaviors did you employ? Share this exercise with a friend. Compareyour thoughts and your good reader behaviors.

Choose two selections written at the same grade level. How are they alike?How are they different? Which one seems easier and why?

Identify something that you have just learned to do (knitting, playing golf,target shooting, etc.). What progress have you made? Do you feel that compari-son to a master or pro is a fair description of your progress? Why or why not?Relate this to determining student progress in reading.


TWO

Assessment as Partof Instruction

How Can We Assess as We Teach?

CHAPTER TOPICS

The Four-Step Assessment Process in the Classroomor Coaching Session

15

Sharing Good Reader Behaviors with Students 32Matching Assessment and Instruction to Standards 34Making Reading Assessment Manageable 36Using Reading Portfolios 42Dealing with Report Cards and Grade Books 44Making Time for Individual Reading Assessment 45Summary 47Activities for Developing Understanding 48

The Four-Step Assessment Process in the Classroomor Coaching Session

Which comes first, assessment or instruction? Does instruction drive assessment,or does assessment drive instruction? There is a tendency to think of assessmentas something that happens after instruction, something apart from instructionsuch as asking students to answer questions after they have read a textbook chap-ter or a story. There is also a tendency to think of all assessment measures aspaper-and-pencil tests. These represent very limited views of the assessment pro-cess. Good assessment is actually embedded in the process of instruction (Sweet,2005; Winograd, 1994). Assessment and instruction can happen at the sametime. The most powerful assessment for students learning occurs in the class-room, moment-to-moment among teachers and students (Johnston, 1992a, p.

15

60). Instruction and assessment cannot be separated; they are two sides of thesame coin! The goal of any literacy lesson is the development of good readerbehaviors. A teacher or coach who has identified a good reader behavior as thefocus of a lesson knows what to assess. However, if the teacher or coach is unsureabout the lesson goal, assessment will probably be unfocused and ineffective.Teachers and coaches need to develop their own assessment systems based upontheir instructional goals. In this way, assessment emerges from the classroom; it isnot imposed upon it (Tierney, 1998).

Assessment can happen at any point in a lesson. It can be a planned andcarefully thought-out procedure. It can also be unplanned. Sometimes the unex-pected occurs. A teacher or coach who knows the goal of the lesson can use theunexpected as an added assessment tool.

Many sensitive and effective assessments do not look anything like a typicaltest. They do not necessarily involve a question-and-answer procedure or apaper-and-pencil format. For example, teacher observation of what students doand say in the reading classroom is a powerful assessment measure.

In Chapter 1, we have identified four steps in the assessment process. Let usreexamine these in terms of reading instruction.

Step 1: Identify the Good Reader Behaviors That You AreTeaching and Assessing

The first step is to identify what you are teaching and assessing (Au & Raphael,2007). In other words, you must decide which good reader behaviors you wantto develop in your students. The focus of all literacy lessons should be on thegood reader behaviors. These behaviors drive both the instruction and the assess-ment process. If you do not know what good reader behaviors you are develop-ing in your students, it will be difficult to select appropriate evidence for deter-mining whether your students have learned them.

I remember teaching a novel to eighth graders. I really did not have any spe-cific goals for my lessons. I didnt know anything about good reader behaviorsand the importance of developing these in my students. I just wanted my stu-dents to understand the novel and (I hoped) enjoy it. Class discussion focused on


THE FOUR STEPS OF THE ASSESSMENT PROCESSIN READING INSTRUCTION

Identify the good reader behaviors that you are teaching and assessing.

Collect evidence that is related to the good reader behaviors.

Analyze the evidence on the good reader behaviors.

Make an instructional decision about the good reader behaviors.

literal understanding of what happened to the characters. I simply did not thinkabout teaching my students to summarize, to self-question as they read (I did allthe questioning!), or to draw inferences about the characters and the time inwhich they lived.

After we finished reading and discussing the book, I constructed a long andinvolved multiple-choice test. I corrected the test, turned my students scoresinto letter grades, and entered them into my grade book. If you had asked mewhat I was assessing, I would have said, Reading. But what did that mean?Some questions revolved around unimportant details. Others required that thestudents make inferences in order to arrive at correct answers. Others focused onvocabulary. I never attempted to separate these. I never even considered whethera multiple-choice test was the best way to assess my students comprehension.And because I did not allow them to look back in the novel to find the answers (Iwould have called that cheating), I actually didnt know whether I was assessingreading comprehension or student memory. Perhaps I was not even assessing

Assessment as Part of Instruction 17

GOOD READER BEHAVIORS: REVIEW

Before, during, and after reading, good readers . . .

Use letter and sound patterns to pronounce unfamiliar words.

Pronounce words accurately.

Pronounce words automatically.

Read fluently.

Attach meaning to words.

Learn new words and refine the meanings of known ones.

Connect prior knowledge with text information.

Determine what is important in the text.

Recognize text structure.

Summarize and reorganize ideas.

Make inferences and predictions.

Construct visual images.

Ask questions and read to find answers.

Synthesize information.

Form and support opinions.

Recognize the authors purpose/point of view/style.

Monitor comprehension and repair comprehension breakdowns.

Read for enjoyment and enrichment.

anything at all related to reading! Perhaps I was simply assessing my studentsmotivation or persistence in completing such a lengthy task.

If I had focused my instruction on specific good reader behaviors, I wouldhave known what and how to assess. For example, if I had attempted to developmy students ability to synthesize information from different sources, a multiple-choice test would have been a sorry assessment instrument. Wouldnt it havemade more sense to ask them to compare a character or situation in the novel totheir own lives, to a movie or television series, or to the contents of anothernovel? What if I had focused upon teaching my students to summarize? I couldhave asked them to write a short summary of a specific episode in the novel fromthe perspective of one of the characters. Knowing what good reader behaviorsyou are teaching and assessing is the first step in the instructionassessment pro-cess.

Tying the Good Reader Behaviors to Standards and Expectations

We have tended to break the reading process into many little pieces, assumingthat if students know the pieces, they will know the whole (DeLain, 1995).Nothing could be farther from the truth! Have you ever attempted to puttogether a jigsaw puzzle without referring to the picture? It is extremely difficultto know where to begin or how to group the pieces. Do pieces of a similar colorbelong together, or do they form two separate areas in the puzzle? And the morepieces you have, the more difficult it is to see the whole. Reading is a very com-plex act, and we have attempted to understand it by dividing it into parts. Butperhaps we have constructed too many parts andas in the case of thepictureless jigsaw puzzlehave lost a sense of the whole.

When I taught from a basal reading anthology, I had a long list of readingskills to develop. It was called a scope and sequence chart. At the end of a gradingperiod, I was required to check the skills that each individual student had mas-tered. Quite frankly, I had no idea. There were so many skills that it was impossi-ble to assess each one adequately. All were regarded as being of equal importance.And many of them seemed to be the same thing! For example, how was drawingan inference different from making a judgment or reaching a conclusion? Andbecause dealing with the scope and sequence chart was such an overwhelmingtask for me, I resorted to assessing reading, whatever that meant.

The scope and sequence chart is still with us. However, we now talk aboutstudent performance expectations or target behaviors as opposed to studentskills. In addition, we have overarching standards or outcomes that stipulatewhat students should be able to know and do at a particular grade level. Statesand districts have constructed these standards, and teachers and coaches areexpected to use them. They are expected to connect classroom activities toclearly stated instructional outcomes and evaluate student success on these


activities against clearly stated criteria (DeLain, 1995, p. 441). The No ChildLeft Behind (NCLB) Act now requires that yearly standardized testing be tied tosuch outcomes.

Although the standards are well intentioned, they can pose problems. First,they tend to be very general, all-inclusive, or a combination of both. In somecases, standards are nothing more than the old scope and sequence chart servedup in a different format. Consider the following fourth-grade standard from thestate of Wisconsin (Benson, Fortier, Grady, Karbon, & Last, 1998, p. 2):

Use effective reading strategies to achieve their purposes in reading

Use a variety of strategies and word recognition skills including rereading,finding context clues, applying knowledge of lettersound relationships, andanalyzing word structures

Infer the meaning of unfamiliar words in the context of a passage by examin-ing known words, phrases, and structures

Demonstrate phonemic awareness by using lettersound relationships as aidsto pronouncing and understanding unfamiliar words and text

Comprehend reading by using strategies such as activating prior knowledge,establishing purpose, self-correcting, self-monitoring, rereading, making pre-dictions, finding context clues, developing visual images, applying knowledgeof text structures, and adjusting reading rate according to purpose and diffi-culty

Read aloud with age-appropriate fluency, accuracy, and expression Discern how written texts and accompanying illustrations connect to convey

meaning Identify and use organizational features of texts, such as headings, paragraphs,

and format to improve understanding Identify a purpose for reading such as gaining information, learning about a

viewpoint, or appreciating literature

This is only the first standard! There are three more, and each one is also accom-panied by a lengthy list of skills or expectations.

One cannot quarrel with the purpose behind standards and lists of studentperformance expectations. They provide goals for teaching and learning. How-ever, just as the scope and sequence chart overwhelmed me years ago, so cantodays standards and expectations overwhelm a teacher or coach. I once workedwith a district that had carefully matched its pupil performance expectations foreach grade to the state standards. At one grade alone, there were 50 differentpupil expectations to be assessed! Lets do a bit of arithmetic and determinewhether this is even possible. With one student and 50 expectations, the teacheror coach is responsible for assessing 50 items. With two students, the teacher orcoach is responsible for 100 items. But with 25 students (the average size ofmany classrooms), the teacher or coach is responsible for instructing, assessing,


and reporting on 1,250 reading behaviors! Is this even reasonable? How can wemake it reasonable?

The good reader behaviors are the keys. You can take these good readerbehaviors and fit them under any standard, no matter how broad it may be. Youcan also match any minute expectation to a good reader behavior. For example,lets examine the following good reader behavior: Good readers determine whatis important in the text. Of the 50 items on that district list, the following couldfit nicely under the good reader behavior of determining importance: Identifiesmain idea and details, Sequences key events, Retells story events or keyfacts, Summarizes text, and Recognizes elements of plot. Another goodreader behavior is this: Good readers pronounce unfamiliar words accurately.Under this behavior, we could place the following items from the district list of50 expectations: Reads grade-level sight vocabulary, Knows and applies voweldiphthongs, Reads multisyllable words, Uses chunking to decode, Iden-tifies root/base words, Recognizes the number of syllables in a word, Knowsand applies plurals and possessives, Knows and applies prefixes, and Knowsand applies suffixes.

Consider the fourth component of the Wisconsin standard that I havequoted above: Comprehend reading by using strategies such as activatingprior knowledge, establishing purpose, self-correcting, self-monitoring, reread-ing, making predictions, finding context clues, developing visual images, apply-ing knowledge of text structures, and adjusting reading rate according to purposeand difficulty (Benson et al., 1998, p. 2). This part alone includes 11 differentitems. Match these to the good reader behaviors and you will see that the listbecomes considerably smaller. In fact, depending upon what you count, there areapproximately 27 different performance items on the entire first standard alone.I can match these to 9 good reader behaviorsa considerable saving and a muchmore workable solution!

Perhaps, at this point, you would like to offer an argument. You may feelthat the good reader behaviors, like the standards, are too general. Shouldntteachers or coaches specifically address all those different pupil expectations thatI have so casually grouped under one behavior? Of course. Teachers or coachesmust teach a variety of word identification strategies and a variety of organiza-tional text patterns. But can they realistically keep track of every students prog-ress on every one without doing a bit of regrouping?

We have to realize that all those skills or expectations are not necessarily sep-arate entities. For example, suppose a reader comes to an unfamiliar word andpauses, trying to figure it out. If the reader is successful, does he or she use lettersand sounds, prefixes and suffixes, syllable patterns, or a combination of all these?The district list that I have described even included specific prefixes (bi-, ex-, co-,tri-, sub-, and auto-) and suffixes (-ly, -ful, -ation, -ion, and -less) under the expec-


tations. Is it really possible for one teacher or coach to identify which of these isknown or unknown by each individual student? Is it possible even to know forcertain which of these the student uses to identify a specific word? And mightnot this change, depending upon the word itself? That is why it makes moresense to group specific expectations or skills under a manageable number of goodreader behaviors, and to use these good reader behaviors to drive instruction,assessment, and reporting to parents on student progress.

There are other difficulties with lists of pupil expectations or skills. On a list,they all tend to take on equal importance. However, they are not equally crucialto successful reading performance. For example, is Identifies mood as impor-tant as Summarizes information? It is difficult to believe that it is. Some pupilexpectations lend themselves to rich instructional and assessment options; othersdo not. For example, not one teacher in the district could tell me how he or shewould assess Identifies mood beyond asking this simple question: What is themood of the selection? (In addition, there was no clear agreement among theteachers on a definition of mood.) Yet all of the teachers could offer a variety ofoptions for assessing Retells story events or key facts. Although all of the expec-tations can be assessed in some fashion, how can a teacher or coach, within theconfines of a busy classroom, keep track of the progress of all the students acrossall 50 expectations?

If assessment is to be effective, it must be manageable. For this reason, it iscrucial to have a realistic starting point. The good reader behaviors provide thisstarting point. By focusing on them, a teacher or coach can better handle assess-ment of all students and be accountable for either the standards or the moredetailed performance expectations that fit under them. The role of the teacher orcoach is to match performance expectations to a good reader behavior, and to tiethat behavior to the district or state standards. We know that the reading processis much more than the sum of the good reader behaviors, but these behaviorsallow us to realistically handle the instruction and assessment of an extremelycomplex process.

Teaching the Reading Process, Not the Content of the Text

I often ask my students at the university what they are planning to teach. Theygenerally respond by naming a specific story or a chapter in a textbook. As ateacher or coach, however, you do not teach a story or chapter. Instead, you usethat story or chapter to develop good reader behaviors. The good reader behav-iors are your focus; the story or chapter is the tool that you will use to develop it.Of course you want your students to understand the selection. Asking questionshelps you to determine whether they have. However, what do you want them toget out of reading it? Do you want them to remember that a character lost his


brothers football and had to earn money to replace it? Or do you want them toremember that all stories are built around solving characters problems, and thatgood readers focus on this when they summarize what they have read?

Using good reader behaviors as a starting point helps a teacher or coachfocus on the process of reading as opposed to focusing on the content of selec-tions. The reading teacher or coach does not primarily teach content; that is, thefirst goal of the lesson is not for students to learn how a character built tin canstilts or why another character dug up earthworms. Reading teachers andcoaches teach students how to read, and must therefore concentrate on the pro-cesses real readers . . . engage in as they read (Allington, 1994, p. 23). Contentis important, but it is not the principal goal. For example, a teacher or coach maywant students to learn about butterflies. However, even when facilitating contentlearning, the teacher or coach still focuses on processon how the students read,synthesize, and report their knowledge of butterflies. The thinking processesneeded to identify words, and to comprehend and remember selections, are thefocal points of reading instruction.

The good reader behaviors are not completely separate. They tend to over-lap. For example, good readers determine importance and summarize. How cansummarizing be completely separate from noting importance? Good readers askquestions and read to find the answers. Very often, finding the answer involvesmaking an inference.

Should we worry about the fact that good reader behaviors are not separateactivities? I do not think so. The reading process is too complex to be dividedinto discrete parts. We made that mistake in the past with our scope andsequence charts, and we are making it again with our long and detailed lists ofstudent expectations. The good reader behaviors are guidelines that help you as ateacher or coach choose an instructional focus. Chances are that in developingone good reader behavior, you are developing others. If you show students howto monitor their comprehension, you are probably also helping them to self-question. Good reader behaviors provide you with a manageable frame of refer-ence for designing instruction and assessing reading performance in terms ofprocess as opposed to content. They also provide you with a manageable frameof reference for dealing with standards and expectations. In short, they driveinstruction and assessment!

Step 2: Collect Evidence That Is Related to the GoodReader Behaviors

If you know what good reader behaviors you are teaching and assessing, you havea better chance of choosing the proper evidence. Evidence should be relevant;that is, the evidence should be clearly related to the good reader behaviors youhave chosen as your focus. If you are teaching students to draw inferences, you


should collect evidence on their ability to do just that. If you are focusing onincreasing reading fluency, your evidence will be something that shows fluency,such as examples of oral reading.

You also need to know how you will record this evidence during the assess-ment process and afterward. The classroom or tutoring session is a busy place! Asa teacher or coach, you have to pay attention to many things at once. You cannottrust your memory. If your evidence comes from observation of students activi-ties or comments, you need to set up some manageable system for recordingwhat they say and do, so you wont forget it. Without some form of documenta-tion, much of what has been seen will be either forgotten, or, more likely,remembered in a form that scarcely resembles the original artifact or event(Wolf, 1993, p. 519). You also need to maintain an ongoing record of individualstudent progress. The following account of Jaynes instruction and assessmentsuggests how all this may be accomplished.

Jayne chose the following good reader behavior as the focus of her lesson:Good readers recognize the structure of the text. She decided to use focuseddiscussion to develop her fourth-grade students understanding of story struc-ture. After reading a story, Jayne gave each student four cards. One card con-tained the word Character. Problem and Solution were written on the second andthird cards. The final card contained Steps. As Jayne and her students discussedthe story, each child held up a card to indicate what he or she wanted to say. Ifstudents held up the Character card, they were expected to talk about the charac-ter; if they held up the Problem card, their comments were expected to focus onthe problem; and so on. Jayne modeled comments on these elements of storystructure and reminded her students that using story parts would help them toremember and retell a story more effectively.

During the discussion, Jayne also collected evidence about her studentsunderstanding of story structure. She used a simple checklist to keep a record oftheir comments. The checklist had four columns marked Character, Problem,Solution, and Steps, and a fifth column with no heading. If a student held up theCharacter card and talked about a character, Jayne placed a check mark in theCharacter column. If a student identified a characters problem or commentedabout it, Jayne placed a check mark in the column marked Problem. If a studenttalked about the steps that were involved in solving the problem, or its solution,she checked Steps or Solution. If students made comments that were not directlyrelated to the structure of the story, or if they had difficulty matching their com-ments to the card they held up, Jayne noted this by making check marks in thefifth column. During the discussion, Jayne was both instructing her studentsin, and assessing their understanding of, components that are common to all sto-ries.

During the discussion, Jayne observed that Paola was staring out the win-dow and that Lorne was attempting to grab Hallies pencil. What did they tell



Jaynes Checklist

Good Reader Behavior Recognize Structure of TextDate

Selection How Beetles Got Their Beautiful CoatsActivity Whole-Class Discussion

Character Problem Solution Steps

Joseph

LaKendra

Lorne ?

Marva

Willie

Juan

Shane

Paola ?

Rosa

Hallie

Raoul

Kendall

Brett

Altonia

Brenda

DeAndra

Jesse

Tiffany

Jackie

Allie

Jon

Detura

Jayne about the students understanding of story structure? Perhaps nothing. Onthe other hand, perhaps lack of involvement signaled lack of understanding, andJayne noted this on her checklist. She placed question marks next to Paolas andLornes names in the fifth column.

Jayne realized that discussion has limitations in assessing all students. Someare shyer or more reticent than others and do not participate eagerly in discus-sion. Both the amount of time allowed for discussion and the content of thestory limit the number of times a student can contribute, as well as what can besaid. Only one student can talk at a time, and that students contribution may beidentical to what others would have said if they had been given the chance.Therefore, Jayne collected additional evidence on her students understanding ofstory structure. She asked them to choose one of the characters in the story andplan a short story of their own. They should imagine another problem the char-acter might have, devise a solution, and tell how the character solved it. Shehoped that this activity might shed insight on the extent to which individual stu-dents such as Paola and Lorne understood the role of basic story elements.

Jayne was also concerned about the performance of individual students dur-ing the discussion. On the checklist, she highlighted in yellow the names of stu-dents who did not participate in the discussion, or whose comments were unre-lated to story structure elements or to the cards they held up. Was the discussionperformance of these students due to lack of understanding of story elements?Perhaps they just did not have enough chances to participate during the discus-sion. Perhaps they chose not to contribute because of uncertainty or shyness.Jayne used her highlighted checklist as a guide for paying more attention to thesestudents and perhaps offering them more opportunities for talking in the future.She also used it as a guide when she held conferences with them about theirindependent reading. If Jayne had not kept some record of the discussion, shemight not have noticed these studentsor, if she did, she might have easily haveforgotten. Again, the classroom is a busy place, and forgetting occurs very easilyin the best of circumstances!

In summary, Jayne followed the first two steps of the assessment process. Sheidentified what she was teaching and assessing by selecting a good reader behav-ior as her focus and choosing appropriate instructional activities to develop it.She then collected evidence that was related to the good reader behavior andused a checklist to record student performance.

Since the first edition of this book was published, teachers and coaches haveshared with me various ways in which they use and adapt Jaynes checklist.Marva uses it during the word-sorting activity in her primary classroom. As herstudents sort word cards to demonstrate similar lettersound patterns, sheemploys the checklist to keep track of their progress. She has three headings: aplus mark, a check mark, and a minus sign. She assigns a plus mark if a studenthas no difficulty in forming meaningful word sorts. If the student needs some


small help or suggestion, she assigns a check mark. Students who are unable tosort receive a minus sign, and Marva pays particular attention to their needs dur-ing the next session. Often Marva groups her students for word sorting, but shestill uses the checklist to keep track of individual participation in the group activ-ity.

Dan uses the checklist during vocabulary discussion in a middle school lan-guage arts class. His headings are Yes, No, and Maybe. If students demonstrateunderstanding of word meaning, he marks them under Yes. If they offer an erro-neous definition or example, he checks no. In some cases, a students commentssuggest some knowledge of the word, albeit incomplete; in these cases, he assignsMaybe.

Robert uses the checklist when he teaches his fifth graders how to readexpository text by forming questions, noting topics, identifying importantdetails, and recognizing areas of misunderstanding or difficulty. Because the dis-cussion is quite fast-paced, Robert does not attempt to keep track of each ofthese elements. Recognizing that the good reader behaviors overlap, he onlychecks two possibilities: On target and Off target. If a student participates in thediscussion and demonstrates understanding of the text and the reader strategiesthat Robert is modeling, he or she receives a check mark for On target.

Anna uses the checklist during sustained silent readinga time when shespends time with individual students. She asks them to read a short portion oftheir chosen book and summarize the content up to that point. Her headings areFluency and Retelling, and she assigns a plus or minus as she evaluates each.

As I was preparing this second edition, a story appeared on the front page ofour local paper. It described an elementary school with low test scores thatimproved so much that it won an award from the U.S. Department of Educationas a Blue Ribbon School (Borsuk, 2007). How did this happen? Teachers learnedto focus on what the children were learning. They aligned their teaching to thedistrict goals and began to monitor student work systematically by keepingrecords of individual student performance and progress. I would not be surprisedif different versions of Jaynes checklist were in every classroom!

Step 3: Analyze the Evidence on the Good Reader Behaviors

Assessment is always interpretive (Johnston, 1992a, p. 60). In an attempt tomake sense of students reading performance, teachers and coaches must inter-pret the evidence they have collected about the good reader behaviors (Valencia,2007). What did Jaynes evidence reveal about the presence or absence of thegood reader behavior that was the focus of her lesson? Jaynes checklist recordedthe kinds of comments her students made, but what did it indicate about theirunderstanding of story structure? Jayne also asked her students to plan a story


involving one of the characters they had discussed. What did this plan tell Jayneabout individual students understanding of story elements? This is where arubric comes in.

What Is a Rubric?

A rubric provides criteria that describe student performance at various levels ofproficiency (ONeill, 1994, p. 1). In other words, it offers guidelines for analyz-ing the evidence. For example, in Chapter 1, we have talked about decidingwhether you should stop to fill your gas tank. The evidence you would collect isobviously your gas gauge. However, how do you analyze the state of your gauge?What guideline or criterion do you use for filling up? Do you fill your tank ifyou are at the halfway mark? Do you fill it if you are three-quarters down? I havea friend who never stops for gas until the gauge reads practically empty! On theother hand, I get very nervous when the gauge hits three-quarters empty. Each ofus is using different criterion for stopping for gas, and the criterion we are usingis a type of rubrica guideline for analyzing the evidence.

A rubric is a tool for examining student work and for judging the quality ofthat work. I remember assigning projects and, when it came time to give a grade,choosing the better students work to examine first. I then used their perfor-mance as a guide for evaluating the others. I suppose this was a form of a rubric,but not a very efficient or thoughtful one. Prior to assigning the project, I shouldhave listed what I wanted the students to include in the project and what Iexpected the project to look like. In other words, I should have constructed arubric. Then not only would I have had a guideline for evaluation, but I couldhave shared this with my studentswho then would have known exactly whatwas expected of them.

Why should a teacher or coach take the time to construct and use a rubric?First, a rubric clarifies the process of analyzing evidence of good reader behaviors.It provides guidelines for judging quality (Au & Raphael, 2007). Second, rubricsmake expectations explicit (Winograd, 1994, p. 421). If a rubric is shared withstudents, it can improve the students performance by making the teachers orcoachs expectations quite clear. Students can use the rubric as well as the teacheror coach. They can use the rubric to evaluate themselves, and self-assessment is apowerful instructional tool. In this way, a rubric both instructs and assesses!

A rubric can be written, but it does not have to be. My friend and I use men-tal rubrics or guidelines to determine whether we should stop for gas. I stopwhen the gauge reads three-quarters empty; she stops when it is on empty! Asanother example, think about preparing for a vacation. Some of us make lists ofwhat we must pack and what we must do before we leave. We check each itemoff. Did we notify the post office to stop mail? Yes. We check it off. Did we take a


bathing suit? Yes. We check it off. This list is a simple form of written rubric.Others of us perform the same process mentally, checking off in our heads whatwe have completed and what still needs to be done. This is an example of a men-tal rubric.

Constructing a Rubric

As a teacher or coach, how do you construct a written rubric? There is one pri-mary guideline: Keep it simple! Teachers and coaches rarely have time to con-struct involved rubrics. They do not have time to score long and complex ones.You want a rubric that is easy to use and is focused upon specific good readerbehaviors. If the rubric is too complex, you will eventually discard it (and proba-bly feel guilty for doing so!). Avoid the guilt and emphasize simplicity.

A rubric is made up of two elements: It lists criteria for a piece of work,[and] it also articulates gradations of quality for each criterion from excellent topoor (Goodrich, 19961997, p. 14). Lets construct a simple rubric so you canget an idea of how to work with these two components. Lets build a rubric forassessing a salesperson. We have all had experiences with very efficient and help-ful salespeople. We have also had shopping experiences ruined by bad-temperedsalespeople who would not or could not answer our questions or provide thehelp we needed.

First, we need some criteria for what makes a good salesperson. Think aboutsalespersons whom you would describe as good or excellent, and use what youremember about their behavior to generate some criteria. For example, a sales-person should be friendly. This seems self-evident; nobody appreciates a grouch!A salesperson also should know his or her merchandise. Nothing is more annoy-ing than to ask questions about a possible purchase and not receive an answer (orreceive the wrong answer). A salesperson should be attentive and not ignore youby chatting with other sales personnel or arranging merchandise. A salespersonshould listen to your needs and not try to talk you into something you dontwant. A salesperson should be helpfulthat is, willing to make suggestions andoffer you alternatives if the merchandise you originally wanted is unavailable. Wecould probably generate many more criteria, but lets keep it simple. We nowhave five criteria: A good salesperson is friendly, knows the merchandise, is atten-tive, is a good listener, and is helpful. We have the first element of our rubriccriteria for performance.

Lets move to the second partdegrees of quality. Each criterion needs to beranked in some way. The simplest ranking is a yesno rating; for example, thesalesperson was either friendly or not friendly. Another form of ranking involvesnumbers such as 4, 3, 2, and 1, with 4 standing for highest quality. Young chil-dren can understand yesno ratings and numbers very well. Other rankings are


more descriptive. Some rubrics use one or two statements to describe each rank-ing, and this is a point you may eventually reach. In the beginning, however,strive for simplicity. If you want to move beyond yesno ratings and numbers,use only one or two words to describe your gradations of quality. In assessing asalesperson, you might use these terms: Very effective, Effective, Somewhateffective, and Not effective. Suppose you are interested in assessing knowledgeand understanding. Your descriptors might be Full understanding, Someunderstanding, Incomplete understanding, and Misunderstanding. If youare interested in how often something occurs, you might use Usually, Fre-quently, Sometimes, and Never.

So, after listing criteria and deciding upon how to rank them, we put thetwo together to form a simple rubric for assessing a salesperson.

A good salesperson . . .

4 3 2 1

Is friendly

Knows the merchandise

Is attentive

Is a good listener

Is helpful

Lets review the process of constructing a rubric with a reading task. Teachersand coaches often ask students to give book talks. Each student chooses a favor-ite book and presents it to the class. What rubric would be appropriate for assess-ing the quality of that presentation? The first step is to list criteria. Just as welisted criteria that described a good salesperson, we have to list criteria for a goodbook talk. What do you want to see in the students work? First, tie the assign-ment to a good reader behaviorin this case, Good readers choose to read forenjoyment and enrichment. Given this good reader behavior, you might decidethat the following criteria should be included: an attention-grabbing introduc-tion, a description of a favorite part or character, an explanation of why the bookwas chosen, and a personal reflection on the value or worth of the book. Remem-ber to keep your rubric simple. In the interests of simplicity, fewer criteria arebetter than more. Once you have the criteria, move to the rating scale. Will ayesno rating be sufficient? Do you want to use numbers? What about words todescribe quality? Perhaps your rubric might look like this.


Book Talk Rubric

Name Book

Veryeffective Effective

Slightlyeffective

Noteffective

Grabbed audience attention

Described favorite part/character

Explained why the book was chosen

Talked about worth of book

Overall rating of book talk

How can you, as a busy teacher or coach, find the time to construct a rubric?You find the time by making it part of your instruction. You dont stay afterschool to create rubrics on your own! You dont construct them over the week-end! You create rubrics collaboratively with your students. This is a powerfulvehicle for sharing good reader behaviors. Explain the assignment to the stu-dents, and ask them what would make a good project. Guide them to choosesuitable criteria. Work with them to select realistic descriptors. Deciding uponimportant criteria and choosing descriptors for quality are effective activities fordeveloping the students understanding and appreciation of good reader behav-iors. Involving the students will also help you to keep the rubric simple. Youwant a rubric that the students understand as well as you do, because they aregoing to use it too. Once a rubric is created, use it again and again. If it needs tobe revised, involve the students in the revision. In fact, it has been my experiencethat students often take the initiative in revising or expanding rubrics.

Many districts have constructed their own rubrics, and teachers or coachesare required to use them. There are also published rubrics (Burke, 2005, 2006;Billmeyer, 2001; Glickman-Bond, 2006; Hill, Ruptic, & Norwick, 1998). Ifsuch measures are helpful, by all means use them. If they seem overwhelming,use parts of them. Start with what you can realistically handle. As you becomemore experienced in the instruction and assessment of good reader behaviors,your rubrics will naturally become more complex. Allow yourself to grow intothe process.

Jayne used two rubrics to assess her classs performance. As Jayne examinedthe checklist, she used a mental rubric. She decided that if 18 students, orapproximately 80% of the class, offered comments tied to story structure ele-ments, she would be well pleased. Someone else might have used a different cut-off. That is just fine. There are no absolute guidelines here! Establishing a cutoffis your decision. Try it out; you can always change it. Jayne also noted that most


of her students talked about the characters, their problems, and the solutionsthey reached. However, fewer than half commented about the steps that led toproblem resolution. She realized that she would need to focus more on this ele-ment with the entire class.

Jayne and her class worked together to construct a short individual rubric touse for evaluating the students story plans. She reminded them of their focuseddiscussion as a guide to choosing criteria. They decided on a simple yesno rat-ing. If the behavior was present, it would be checked. They agreed that four orfive checks would indicate a very good job in story planning. After the studentsfinished planning their story, they filled out the rubric and evaluated themselves.Jayne collected these filled-out forms and wrote her own evaluations on the samepapers. She then returned the papers to the students.

Jaynes Rubric

Name Date

Assignment

I chose a main character from the story to write about.

I matched the character to a problem.

I described the problem.

I listed at least two steps in solving the problem.

I described how the problem was solved.

Score

45 checks: I did a very good job!3 checks: I did okay.12 checks: I need to think harder about story parts.

Step 4: Make an Instructional Decision about the GoodReader Behaviors

You have now identified the good reader behaviors you are assessing; you havealso collected evidence, and you have analyzed the evidence by using a written ormental rubric. You know which good reader behaviors were evident and whichwere not. The fourth step is to decide what you are going to do about this. Howwill the evidence and your analysis of it indicate future instructional directions?Unfortunately, teachers and coaches often ignore this last step. After they haveanalyzed the evidence, they record it in a grade book and plan another lessontotally unrelated to what went on before!


Teachers and coaches make several kinds of instructional decisions. Onedecision involves choosing the focus of the next lesson. What good reader behav-ior will a teacher or coach emphasize next? Another decision relates to grouping.Will the next lesson be presented to the class as a whole? Does the class need tobe grouped in some way? What kind of grouping will best achieve the teachersor coachs objectives? Should

Documents

Assessment of Reading