Developing Successful Assessment

EDU 423: Educational Assessment and Decision Making

December 7, 2016

bit.ly/cofoanton

With the goals of assessment in mind, what are the most significant factors to consider when developing

and scoring assessments?

Pre

-Ass

ess

me

nt

Beginning of unit

Used to indicate readiness of students

Informs instruction

Form

ativ

e A

sse

ssm

en

t

Checkpoints throughout unit

Informs instruction at regular intervals

Contains smaller sub-sets of objectives and learning goals

Sum

mat

ive

Ass

ess

me

nt

End of unit

Contains all objectives and learning goals

Where do we start? (Types of Assessments to Design)

Design

Implementation

With the goals of assessment in mind, what are the most significant factors to consider when developing and scoring assessments?

Reliability

• Is an accurate representation when repeated over time of knowledge, understanding, and application

• Consistency

Validity

• Accurately indicates what students know, understand, and can do/apply (KUD) and Essential and Enduring Knowledge (EEK)

Objectivity

• Fair and equitable for all students and levels of learners

• Absence of Bias

How do we assure that we are meeting our assessment goals in a reliable manner?• “Good assessment is reliable. One meaning of this is that the

assessment will yield the same accuracy when repeated over time. We can’t always know this when designing classroom assessments each week, but it’s something to try to factor into their design.” (Wormeli 2006, p. 40)

• What might be some ways to check the reliability of an assessment?

How do we assure that we are meeting our assessment goals in a reliable manner?

Stability

• Consistency of results among different testing occasions

• Test-Retest

Alternate Form

• Consistency of results among two or more different forms of a test

Internal Consistency

• Consistency in the way an assessment instrument’s items function

• If an assessment is created to measure whether as student can solve quadratic equations, if they get one item correct, they should get similar items correct

Popham 2002, p. 28

How do we assure that we are meeting our assessment goals in a valid manner?

What does it tell you if your students are asking:

Why are we doing this?

When will we ever use this?

Is this going to be on the test?

How do we assure that we are meeting our assessment goals in a valid manner?• Is the assessment authentic?

• “If assessment is not authentic to how students learned and what they were supposed to learn, then all subsequent grades are questionable.” (Wormeli2006, p. 33)

• An important consideration for the teacher is to see if the test being constructed is representative of the content/objectives being taught. (Popham 2002, p. 49)

• The assessment is close to real-world application. (Wormeli 2006, 32)

• How might you prepare your students for each of the following types of formal, summative assessments in an authentic way?• Essay/Short Answer Tests• Multiple Choice (Standardized) Tests

How do we assure that we are meeting our assessment goals in a valid manner?

How do we assure that we are meeting our assessment goals in a valid manner?• “In order for assessment to be valid, it must be varied and done over

time. A student might know the material today, but to determine whether he or she has learned it, assess the student on the same material a while later.” (Wormeli 2006, p. 41)

• What are some examples of varied assessment types?

Formative, Informal Assessment StrategiesTodd Finley, “Dipsticks: Efficient Ways to Check for Understanding,” Edutopia: https://www.edutopia.org/blog/dipsticks-to-check-for-understanding-todd-finley

Formative, Informal Assessment Strategies:Research/Learning Walls

Formative, Informal Assessment Strategies:Create Your Own Quiz (CYOQ’s)

Quizziz

Kahoot!

“What Would Anton Ask?”

Summative, Authentic Assessment Strategies:Performance Events

Long-Term, “Over Time” Assessment: Interactive Notebooks

Christina Gil, “Interactive Notebooks: No Special Hardware Required,” Edutopia:https://www.edutopia.org/blog/interactive-notebooks-no-special-hardware-christina-lovdal-gil

Long-Term, “Over Time” Assessment: Student Blogs/Portfolios

Hannah Dickens “Ten Tools to Create Online Student Portfolios,” Getting Smart:http://www.gettingsmart.com/2016/01/10-tools-to-create-online-student-portfolios/

Peer and Self-Assessment: Providing Feedback for Students and Teachers• Why should students be offered the opportunity to peer and self-

assess?

How do we assure that we are meeting our assessment goals in an objective manner?

Ensure Equity by Providing Reasonable Accommodations

Limit Bias Using Scoring Guides and Rubrics

Ensuring Equity: Assessment Accommodations

• Four Categories of Assessment Accommodations• Presentation Accommodations allow students to access test directions or content in

ways that do not require them to visually decode standard print. Students with print disabilities (defined as an inability to visually decode standard print because of a physical, sensory, or cognitive disability) may require alternate visual, tactile, or auditory formats.

• Response Accommodations allow students to record responses to test questions in alternate ways or to solve or organize a response using some type of material or device.

• Timing/Scheduling Accommodations change the allowable length of testing time and may also change the way the time is organized.

• Setting Accommodations change the location in which an assessment is given or the conditions of the assessment setting.

Sandra J. Thompson, “Special Connections,” University of Kansas:http://www.specialconnections.ku.edu/~kucrl/cgi-

bin/drupal/?q=assessment/assessment_accommodations/teacher_tools

Ensuring Equity: Assessment Accommodations

Source: National Center for Educational Outcomes

Developing Valid, Reliable, and Objective Scoring Guides

Four Characteristics of a Quality Rubric (Stiggens et al. 2004, p. 220)

• Content (Measures Objectives)

• Clarity (Easily Understood by Teachers and Students)

• Technicality (Easy to Use for Teachers and Students)

• Fairness (Valid/Reliable)

Developing Valid, Reliable, and Objective Scoring Guides

Identify the Objectives and Content

Write the Descriptors with Desired Outcomes

Determine Type of Rubric

•Holistic: Big Picture, Less Detail, Less Feedback

•Analytic: Break Down Concepts, More Detail, More Feedback

Determine Scoring and Labels

Write Lower Descriptors

Developing Valid, Reliable, and Objective Scoring Guides

Identify the Objectives and Content

Write the Descriptors with Desired Outcomes

Determine Type of Rubric

•Holistic: Big Picture, Less Detail, Less Feedback

•Analytic: Break Down Concepts, More Detail, More Feedback

Determine Scoring and Labels

Write Lower Descriptors

Create/sketch a (basic) Rubric Using this Strategy and the Handout.

Submit photo on Twitter using hashtag:

#CofOAssessment

How do we assure that we are meeting our assessment goals in an objective manner?• “Seven Steps to Fair Assessment” (Suskie 2000)

1. Have clearly stated learning outcomes.

2. Match your assessment to what you teach.

3. Use many different measures and many different kinds of measures.

4. Help students learn how to do the assessment task.

5. Engage and encourage your students.

6. Interpret assessment results appropriately.

7. Evaluate the outcomes of your assessments.