Assessment in Schools

Complex Achievement:Scoring Performance Based Assessments

Question (choose the best answer)

Which statement provides the best description of an analytic scoring rubric?

A. Rating is based on the overall performance compared to exemplars.

B. Rating is based on a sum of scores for the individual parts of the performance.

C. Rating is based on the student’s analysis of their performance.

D. All the above.

Types of Performance

1. Alternative – something other than traditional paper and pencil tests requiring students to demonstrate

2. Authentic – practical application of a task in real world conditions/setting(usually only approximated/simulated)

Assessing Performance

Why use performances in assessment?

Why do we score/measure performance? Communication Comparison

Assessing Performance

All Claims about the value of performance assessments rest on the assumption that performance can be accurately observed and reliably rated.

Conducting Music Assessment

Scoring Challenge

No one correct or best answer/solution

Many different performances or solutions might be judged as excellent (or poor)

Requires expert judgment and clearly specified criterion to assess properly.

Scoring Limitations Scoring can be inconsistent (unreliable Rating)

To compare scores fairly, Task (learning outcome) must be clearly

defined and communicated to students Scoring criteria/rubrics must be well

defined. Time consuming to complete

Must have reasonable amount of time to do Limits the number of tasks that can be

done

Scoring Issues

What are you assessing?

Process – approach used, methods & procedures, instrument use, etc.

Product – complete performance or resulting artifact

Scoring Decisions

What assessment instruments will be used?

Rubrics, Rating scales, Checklists …

How will the results be used/reported?

What will you do to make sure the results are accurate (reliable)?

Scoring Decisions

Who will do the assessment?

Teacher, student, peers, others

How will they be trained?

Guidelines and Suggestions

1. Focus on the learning outcomes that require complex cognitive skills and performances

2. Select tasks that represent important content and skills

3. Minimize the dependence of irrelevant skills not directly related to learning outcome

Guidelines and Suggestions

5. Provide scaffolding as needed6. Construct task directions that

clearly explain what students are expected to do

7. Clearly communicate performance expectations (how performance will be judged)

Scoring Issues

What are your expectations?

Criteria – ideas about what is good or desirable when we judge adequacy; also used to defend that judgment.

Scoring Criteria Issues

Floating Criteria – wait until you see the performance to determine acceptability

Ask yourself – Do you know what your are looking for? Can you define and describe the quality of a

performance (both good and bad)? Can you provide a defensible basis for

rating good and bad performance?

Scoring Criteria Issues

Criteria – define what is acceptable and unacceptable

in ways the student can understand communicate the goal or standards not useful when vague or ambiguous make public what is considered important

[9-30] characteristics

Instruments

Rubrics Rating Scales Checklists

Scoring Rubrics

Rubrics are a set of guidelines that explain the criteria by which performance will be judged or rated (may include a rating scale).

Rubrics outline performance standards

Scoring Rubrics

Rubrics can be

Analytic – individual aspect of the task are judged and used to determine overall score

Holistic – the performance or product is judged as a whole, compared to models or exemplars

[see chapter 10]

Scoring Rubrics

Rubrics typically provide a description of how the rater should determine the quality of various performances at specific levels.

Examples [9-29,9-33, pg 272]

[9-32 rubric development]

Group Task

Create an rubric you might use to rate or score the Leading Music performance.

Rating Scales

They provide a convenient recording method, common frame of reference, and focus the raters attention on specific important aspects of the performance

Used for (limited to) make quality judgments Requires additional information regarding

performance expectations Examples [9-34, pg 274]

Rating Scales

These take many forms (numerical and descriptive) but are used to provide a uniform way to score performances along a continuum (at least ordinal, preferably interval).

[9-42 types of rating scales]

How often do you (meant to record

frequency)

Response scale 1

Daily 2-3 times per week Once a week 2-3 times a month Once a month Less than once a

month

Response scale 2

Once a day Once a week More than once a

day More than once a

week As seldom as

possibleNot Ordinal, Not IntervalOrdinal but Not Interval

Question (choose the best answer)

Which of the following is NOT a good principle for constructing a graphic rating scales?

A. Characteristics should be directly observable.

B. Use 3 to 7 points on the scale.C. Points on the scale must form an

ordinal continuum.D. Each point on the scale must be

defined clearly.

Rating vs. Ranking

Ranking requires a person to place in relative order

Raters assigns a specific score

Why might you rank instead of rate?

Checklists More appropriate for analytic rubrics where

you can easily divide the task into a series of specific actions that must be present.

Reduces the amount of subjectivity in the judgment (dichotomous decision)

Can be problematic when aspects of the performance are valued but not represented in the criteria. (e.g., esthetically pleasing, interesting)

Examples [pg 282]

Common Rating Errors

Personal Bias – Generosity error – too easy, grade inflation Severity error – too hard, no perfect papers Central tendency – rating everyone about average Halo Effect – general impression of individual

(positive or negative) influences an individual rating Logical error – rating alike or different based on the

belief that factors are related (e.g., studious and able) [see 9-44, pg. 277]

Effective Rating Review

Focus on educationally significant outcomes

Characteristics should be directly observable

Clearly define key points on scale Select most appropriate type of

instrument Use an appropriate scale (# of points)

Effective Rating Review

Rate all performances on one task before going on to next.

When possible rate performances without knowing the raters name

If the assessment has significant impact, several ratings should be used.

Example practice [7-22,7-23]