Using Data to Make Using Data to Make Informed DecisionsInformed Decisions

Key Terms Overview and Test Drive•OAKS•RIT•Vertically Scaled Assessment

Presented by Jennifer Shotts

““We have wide range of complex We have wide range of complex achievement targets to assess. We need achievement targets to assess. We need all the tools we have at our disposal to all the tools we have at our disposal to do this job. We can not afford to throw do this job. We can not afford to throw any-including standarized tests-away. any-including standarized tests-away. Our challenge is to find ways to use all Our challenge is to find ways to use all the tools well and use them in the tools well and use them in balance…”balance…”

Stiggins, R.J. (1985, October) Improving assessment Stiggins, R.J. (1985, October) Improving assessment where it means in the most: in the classrooom. where it means in the most: in the classrooom. Educational Leadership, pp 69-74. Educational Leadership, pp 69-74.

Key Terms: What does Key Terms: What does that mean?that mean?

ACTIVITY: Picture ACTIVITY: Picture the WORDthe WORD

Use the Index Cards and markers on your tableUse the Index Cards and markers on your table Draw a word picture that helps you identify Draw a word picture that helps you identify

OAKS, RIT, VERTICALY SCALED. OAKS, RIT, VERTICALY SCALED. Add words that help you define it as we go Add words that help you define it as we go

along.along.Oregon Assessement of Knowledge Oregon Assessement of Knowledge and Skillsand SkillsSummative, Criterion-Based, Summative, Criterion-Based, Vertically Scaled RIT score, Vertically Scaled RIT score, Computer-AdaptiveComputer-Adaptive

Activity: Right Angle ThinkingActivity: Right Angle Thinking

What do you know about OAKS What do you know about OAKS Assessments?Assessments?

What questions puzzle you?What questions puzzle you? How can you explore this topic today? How can you explore this topic today?

On your graphic organizer, brainstorm the Facts (A), and Feelings /Associations (B) about our topic:

OAKS Assessments.

Oregon Assessment of Knowledge and SkillsOregon Assessment of Knowledge and SkillsUse and ConsequencesUse and Consequences

OAKS—used to measure the extent to OAKS—used to measure the extent to which students are learning the state’s which students are learning the state’s curriculum as described in the Oregon curriculum as described in the Oregon Academic Content StandardsAcademic Content Standards Also used to measure Essential skills of Oregon Also used to measure Essential skills of Oregon

Diploma Project for reading, mathematics and Diploma Project for reading, mathematics and part of writing part of writing

Serves dual purpose of meeting federal and Serves dual purpose of meeting federal and state accountability requirementsstate accountability requirements

Question 1Question 1

Scale scores for the OAKS exam are Scale scores for the OAKS exam are calledcalled MAX scoresMAX scores Mean scoresMean scores RIT scoresRIT scores Rater scoresRater scores

Measuring ImprovementMeasuring Improvement

What does the RIT Score mean?What does the RIT Score mean? Vertically-Scaled ScoreVertically-Scaled Score Margin of ErrorMargin of Error Challenge: How much improvement or lack Challenge: How much improvement or lack

of improvement is meaningful? of improvement is meaningful? Still relative—improvement compared to Still relative—improvement compared to

what? what?

Question 2Question 2

RIT scores can be used to do all of the RIT scores can be used to do all of the following except:following except: Determine if a student meets standardsDetermine if a student meets standards Determine if a student has grown in Oregon Determine if a student has grown in Oregon

content and skillscontent and skills Determine a student’s strengths and weaknesses Determine a student’s strengths and weaknesses

in a content areain a content area Determine a student’s intervention needsDetermine a student’s intervention needs

Scale Scores: RIT ScoresScale Scores: RIT Scores

Oregon reports test scores in a scale score, or Oregon reports test scores in a scale score, or RIT score.RIT score.

Scale ranges from 150 – 300Scale ranges from 150 – 300

Scaled to allow for student growth to be Scaled to allow for student growth to be measured over timemeasured over time

RIT scores are comparable within the same RIT scores are comparable within the same content area and grade across administrationscontent area and grade across administrations

Vertical ScaleVertical Scale

The continuous scale means that a student who improves by 10 points between 3rd and 4th grades (moving from 204 to 214) has improved just the same as a student who improves by 10 points between 5th and 6th grades (moving from 219 to 229).

This equal interval feature makes measuring growth of individual students easy and reliable.

Vertical ScaleVertical Scale

Reading/ Reading/ Literature Literature

Meets Meets Standards Standards RIT scoreRIT score

Math Math Meets Meets

Standards Standards RIT scoreRIT score

ScienceScience

Meets Meets Standards Standards RIT scoreRIT score

Grade 3Grade 3 204204 205205

Grade 4Grade 4 211211 212212

Grade 5Grade 5 218218 218218 225225

Grade 6Grade 6 222222 221221

Grade 7Grade 7 227227 226226

Grade 8Grade 8 231231 230230 234234

Grade 10Grade 10 236236 236236 240240

RIT Cut Scores for Meets StandardsRIT Cut Scores for Meets Standards

Meets Standards RIT Cut Scores for Subjects

180

190

200

210

220

230

240

250

3 4 5 6 7 8 10

Grade Level

RIT Score

Reading/Literature Math Science

A new source of information: A new source of information: growth measuresgrowth measures

Another way to look at student performance---Did Another way to look at student performance---Did your students grow academically during the year? your students grow academically during the year? RIT scores are scaled to allow for student RIT scores are scaled to allow for student

growth to be measured over timegrowth to be measured over time RIT scores are comparable within the same RIT scores are comparable within the same

content area and grade across administrationscontent area and grade across administrations Policy questions include Policy questions include

What constitutes expected annual growth?What constitutes expected annual growth? What constitutes value added?What constitutes value added?

OAKS Performance ReportsOAKS Performance Reports

OAKS Scavenger HuntOAKS Scavenger Hunt List the subjects/content areas that are List the subjects/content areas that are

reported on OAKS onlinereported on OAKS online What years of test results are available on What years of test results are available on

OAKS online?OAKS online? Name two point in time reports on OAKSName two point in time reports on OAKS Name an Improvement Report on OAKSName an Improvement Report on OAKS Name a Longitudinal Report on OaksName a Longitudinal Report on Oaks Name the levels of detail you can drill to Name the levels of detail you can drill to

on the OAKS system: on the OAKS system:

Read the notes in Read the notes in fine print fine print !!

Teachers should not base decisions and Teachers should not base decisions and evaluations solely on data displayed in this evaluations solely on data displayed in this online score reporting system. online score reporting system.

Comparative data reports are based on the Comparative data reports are based on the number of students tested number of students tested at the timeat the time and and may not be representative of official results. may not be representative of official results.

FERPA prohibits the release of any FERPA prohibits the release of any personally identifiable information. personally identifiable information.

Online Score Report TableOnline Score Report Table

General Overview of PerformanceGeneral Overview of Performanceby Grade and Subjectby Grade and Subject

SchoolSchool

How would you How would you respond to the respond to the following statement? following statement?

Visible Thinking Protocolhttp://www.pz.harvard.edu/vt/VisibleThinking_html_files/03_ThinkingRoutines/03c_Core_routines/UsedToThink/UsedToThink_Routine.htm

I used to think…about I used to think…about OAKS assessmentOAKS assessment

Visible Thinking Protocolhttp://www.pz.harvard.edu/vt/VisibleThinking_html_files/03_ThinkingRoutines/03c_Core_routines/UsedToThink/UsedToThink_Routine.htm

But now, I think…But now, I think…