Student Learning Objectives Day Five: Setting Performance Targets for SLO Learning Goals

Student Learning Objectives

Day Five: Setting Performance Targets for SLO Learning Goals

Re-Introductions Center for Transforming Learning and Teaching

Catalyzing and co-creating the transformation of learning environments through the use of assessment so that all are engaged in learning and empowered to positively contribute in a global society.

www.ctlt.org

Facilitator/Trainer:Julie Oxenford O’Brian

Coach/Trainer:Mary Beth Romke

[email protected] [email protected]

© CTLT 2014 www.ctlt.org

http://www.ctlt.org/

mailto:[email protected]

mailto:[email protected]

Day Four

SLO ComponentsLearning Goal

Learning Goal

Standards Reference

Rationale

Success Criteria

Measures

Evidence Sources

Alignment of Evidence

Collection and Scoring

Performance Targets

Baseline Data

Performance Groups

Performance Targets

Rationale for Targets

Progress Monitoring

Check Points

Progress Monitoring Evidence Sources

Instructional Strategies

SLO Results Student Performance Results

Targets Met

Teacher Performance

Day One

Day Two

Day Three

Day Four

Day Five

Day Five


Your Feedback (Parking Lot)Pluses

!!!

Change in my practice

Your Feedback (Parking Lot)

Change for Training ???

Survey ?? folks responded

Topics on which more than ½ reported they “got it” or “could teach someone else”

Topics on which more than ½ reported they need help or more practice

Survey Results (continued)

Learning from posted SLO Learning Goals For many folks, it didn’t appear as if the Learning Goal sub-

components had been revised based on earlier feedback.

Confusion was evident about what information to include in different section of the SLO Form (e.g., the difference between Evidence Sources for the end of the instructional interval and Progress Monitoring Evidence Sources).

In some cases, descriptions of assessment instruments/tasks were vague (e.g. I’ll use a performance task). They need to be specific (the following prompt will be/was used, “Describe what it is like to eat lunch at school”).

Often, Learning Progressions (learning target statements moving towards the learning goal across the instructional interval) were missing in the “Check Points” row.

Progress Monitoring Evidence sources were not associated with the Check Points learning targets.

Check-InFull Group:

Posting Revised SLO Learning Goals Accessing feedback Using feedback Do you know what evidence sources (assessment instruments or

tasks) you will use to measure student learning towards your Learning Goal at the end of the instructional interval?

With your table group, discuss: Did you post an SLO Learning Goal and receive feedback? Have

you made any changes to your Learning Goal as a result? If you didn’t post your SLO Learning Goal, what kept you from doing so?

How did it go identifying the learning progression associated with your SLO Learning goal? Were you able to identify check points and progress monitoring evidence sources along that progression?


Review: Assessment in the SLO Process

We use a body of evidence (data collected from a variety of assessment instruments) at various points during the SLO Process to “reason from evidence” about:

Student learning in relationship to our Learning Goal at the beginning of the instructional interval (Performance Targets, Baseline Data).

Student progress towards the Learning Goal during the instructional interval (Progress Monitoring, Evidence Sources).

Student learning in relationship to our Learning Goal at the end of the instructional interval (Measuring and Scoring, Evidence Sources).

In the SLO Form we identify:

The assessment instruments or tasks included in the body of evidence.

Actual student assessment results.


SLO Components DescriptionStudent Learning Goal

Learning Goal: Standards Reference: Rationale for the Learning Goal: Success Criteria:

Measures and Scoring

Evidence Sources (Measures or Assessments):

Alignment of Evidence to Learning Goal:

Collection and Scoring (attach scoring guide/rubric): Performance Targets

Baseline Data:

Performance Groups:

Performance Targets:

Rationale for Targets:

Progress Monitoring

Check Points:

Progress Monitoring Evidence Sources:

Instructional Strategies:

Results Student Performance Results:

Targets Met:

Teacher Performance:

SLO FORM

SLO Checklist A supplement to the SLO Form and Component

Descriptions.

Intended to be used to check for completion of the SLO Form, not quality of what is provided in each section.

Work with a partner to review this checklist.

Identify any questions/items that are unclear.

Full group opportunity for clarification.

Clarified where Component Descriptions were vague or misleading (revised SLO Component Descriptions).

Using the SLO Checklist Take out your partially completed SLO Form

Sections that should be completed: Class/Course and Student Population SLO Learning Goal Measures and Scoring (except for aggregating

scores) Performance Targets, (Baseline Data only) Progress Monitoring

Use the checklist to determine if you have completed these sections of your SLO Form.

Your Turn Have you completed each of the sections

of the SLO Form (listed on the prior slide)?

Yes Find another participant who has also

completed his/hers and trade forms.

Use the checklist to determine if your partner has provided complete information each required component of the SLO Form.

No – Take this time to complete the identified sections of your SLO form.

Performance Targets Materials

Session Five SLOs

Engage in learning activity during in-person session

Do follow-up readings and tasks

Try-out practices in your context.

Determine what will constitute evidence of “proficient” student performance on the SLO Learning Goal.

Specify how a body of evidence will be used to evaluate the degree to which students have met the SLO learning goal.

Explain the process of establishing SLO Performance Targets and information to include in the SLO Form.

Interpret baseline data to define student “performance groups”.

Describe expected targets for each performance group.

Understand how student student results can be translated into teacher ratings.

Complete and share the description of an SLO for your class/course using the SLO Form.

Activity: Monitoring your learning Turn to Progress Monitoring and review the learning objectives for

today (note catcher, page 3).

Make notes about what each learning objective means to you.

Create a bar graph which describes where you currently believe you

are in relationship to each learning objective. Leave reflections blank for now.

Learning ObjectiveI don’t know what this Is

I need more practice

I’ve got It

I can teach this

Reflections

Determine what will constitute evidence of “proficient” student performance on the SLO Learning Goal.

This means: I can describe how I will score different assessment instruments, and combine information from multiple evidence sources to determine my students’ performance.

SLO Session Five Agenda

Scoring Assessment Instruments

Using a Body of

Evidence

Reviewing Baseline

Data

SLO Target Setting Process

Building Capacity for SLOs in APS

Setting Performance

Targets

Evidence Sources Measuring student performance in relationship to the

Learning Goal at the end of the instructional interval.

Use at least three evidence sources

Types of Evidence Sources that you could list on your SLO form: Performance or Extended Written Assessment Tasks (scored

within the classroom, scoring depends on degree of quality)

Assessment Instruments with multiple, and possibly different types of tasks/items (scored within the classroom, scoring involves identifying correct/ partially correct responses)

Assessment instruments scored outside of the classroom (interpretation of scores).

Today

Scoring Different Types of Assessment Instruments or Tasks

Degrees of quality One right answerTypes of tasks included

Extended Written Response, Performance

Selected Response and Short Constructed Response (some).

Tools RubricChecklistRating Scale

Answer KeyScoring Guide

Scoring Includes Describe the degree to which quality was attained in the response.

Identify if the response to each item/task was correct.

Describe the performance over multiple aspects of quality and assign a rating.

Aggregate information about the correct vs. the incorrect responses.

Scoring Evidence Sources Last time:

Performance or Extended Written Tasks scored using high-quality rubrics

Today:Assessment Instruments with multiple

items/tasks scored using scoring rules/guides

Assessment Instruments scored outside of the classroom (interpreting scores)

Scoring Assessment InstrumentsHow do/will you score assessment instruments that include:

A mix of different types of tasks/items (some with one right answer)?

Multiple learning targets?

Some learning targets that are not part of the SLO Learning Goal?

Table buzz. . .What is proficient?

What does proficient performance look like on a given assessment instrument?

Would two different teachers across APS identify the same performance as “proficient”?

How do you currently equate your expectations with your colleagues?

Do students who score proficient on classroom assessment instruments also receive a proficient rating on externally administered assessments? On TCAP? On Acuity?

Scoring for SLOs Specifies what learning is being assessed by

each item (on the assessment instrument).

Utilizes an assessment map (remember day 3).

Aggregates data based on each learning target being assessed.

Provides evidence of student learning relative to the SLO learning goal overall.

Aggregating Assessment Results to Different Levels

Multiple Students'

Scores (Teacher Rating)

Multiple Assessment Instruments/Tasks (Individual Student Learning relative to the SLO Learning

Goal)

An assessment Instrument/Task (Individual Student Learning relative to part of the SLO

Learning Goal/ or one source of evidence for the whole SLO Learning Goal)

Items (Student Learning relative to Learning Targets, part of the SLO Learning Goal)

Scoring for SLOs Utilizes Performance Levels Performance level ratings – a way of

summarizing (aggregating) student performance across items/tasks

Commonly used: Unsatisfactory

Partially Proficient

Proficient

Advanced

Do you use other performance levels in scoring student work in your classroom?

Scoring Assessment Instruments1. Fully describe the Performance Levels that will be used

– for each item/task determine what will constitute evidence that students are proficient, define other levels of performance as appropriate (e.g. can students get an item partially correct?).

2. Summarize (aggregate) student performance by learning target (across multiple items/tasks).

3. Summarize (aggregate) performance Relative to the SLO Learning Goal (across multiple targets, but including only targets that correspond to the SLO Learning Goal).

Define/Fully Describe Performance Levels

A. Determine what learning targets are being assessed by the instrument.

B. Determine how much information is being gathered about each target and the SLO Learning Goal.

C. Define what the evidence (from the assessment instrument) will demonstrate.

D. Operationalize performance level definitions (for each learning target and SLO Learning Goal).

Table Discussion Turn to Scoring Assessment Instruments for

Learning (Tools, p. ??).

Review the steps in this process.

Identify any questions you have (or believe your colleagues would have) about the steps in this process.

Defining Performance Levels Example1. Work with a partner, to review the examples for scoring

a 5th Grade Math Assessment Instrument.

2. Take out: Scoring Assessment Instruments for Learning Process

(Tools, p. ??)

Math Assessment Map Example (Tools, p. ??)

Example Math Assessment Performance Level Descriptions (Tools, p. ??)

3. Using the Scoring Assessment Instruments for Learning Process, compare the steps in the process to this example.

Example Math Assessment

With your table group discuss the following questions: How were performance level descriptions (unsatisfactory,

partially proficient, proficient, advanced) operationalized in terms of task/item level performance for each learning target?

How were performance level descriptions operationalized across targets as one source of evidence to evaluate student learning on the SLO Learning Goal at the end of the instructional interval?

Do you agree with the way this instrument was scored or is there another way this instrument could have been scored?

Full group questions/concerns.

Your Turn: Practice Work with a partner. Take out the following:

An assessment map(s) that one or both of you brought to the session.

Scoring Assessment Instruments for Learning (Tools, p. 35).

Defining Performance Levels Worksheet (Tools, p. 41).

Define/fully describe performance levels for an assessment instrument that you have mapped, for each learning target being assessed.

Does this instrument measure student learning in relationship to your SLO Learning goal?

Implications of Scoring for LearningBuzz with a partner. . .

If a score is provided for an entire test (with multiple items), what does that score mean?

What if the test includes more than one learning target? Addresses multiple standards?

When would a summary score for an entire test be meaningful (in terms of learning)?

At what other level(s) can assessment results be summarized?

Summarizing at different levels Summarizing student performance at the

Learning Target Level.

Summarizing student performance across relevant learning targets to determine a score/performance rating that can be used as one source of evidence to evaluate student learning in relationship to the SLO Learning Goal.

Partner Buzz: How could this same approach be used to summarize performance by standard for standards-based grading?

Reporting Student Results for SLOsFor assessment instruments used to measure student learning in relationship to the SLO Learning Goal:

Student assessment results (scores) should also be reported at the SLO Learning Goal Level.

Teachers must generate a score/performance rating relative to the SLO Learning Goal for each student for the instrument.

Student performance across multiple assessment instruments (evidence sources) can then be used to determine if the student is proficient on the SLO Learning Goal.

Scoring Assessment Instruments for Learning What are the benefits beyond SLOs?

Builds the foundation for standards-based grading.

Organizes assessment data to inform instruction.

What does it facilitate for students?Meaning-making of summative tests.

Tracking the progress of their learning over time.

Assessment Instruments Scored outside the Classroom. . .Talk with a partner about. . .

What assessment resources are available from sources outside the classroom?

What scores are provided?

When are scores available?

How are the results used currently?

Could they be used for SLO baseline data?

Could they be used as an evidence source to measure student learning in relationship to the learning goal at the end of the instructional interval?

Examples

Interim/Benchmark AssessmentsAcuity

Early Literacy AssessmentsDRA2

DIBELS

Post-Secondary Readiness AssessmentsPLAN/ Explore

Using Externally Administered/Scored Assessment Instruments. . .

In evaluating student performance at the end of the instructional interval. . .

Check the alignment of the assessment instrument (or a portion of the assessment instrument) with your SLO Learning goal.

Determine what scores are provided and if the available scores align with your SLO Learning Goal.

Using Externally Administered/Scored Assessment Instruments. . .

As baseline data. . .

More likely useful for this type of evidence source.

Determine what scores are provided and the degree to which the provide information related to your SLO Learning Goal.

Next Steps



Using a Body of

Evidence

Reviewing Baseline

Data



Setting Performance

Targets

SLO Form

Assessments, Scoring and Criteria

What assessment instruments will be used at the end of the instructional interval to measure student learning in relationship to the SLO Learning Goal?

How will the results across multiple assessment instruments/tasks be combined to determine student performance?

Aggregating Assessment Results to Different Levels

Multiple Students'


Multiple Assessment Instruments/Tasks (SLO Learning

Goal)

An assessment Instrument/Task (evaluates student learning relative to part of the SLO


Items (Learning Targets)

Evaluating Student Performance on a Body of EvidenceAt the end of the instructional period:

How should you consider multiple evidence sources to evaluate each students’ performance in relationship to the SLO learning goal?

What scoring rules will you use to combine student results across multiple assessment instruments/tasks.

Example

Turn to “Student Learning Objectives Performance Targets Example” (Tools, p. ?).

Consider the Proficient Performance description across three assessment instruments

How did this teacher combine information across multiple measures?

Developing Rules for Combining scores across Evidence Sources (tools, p. ?)

1. Determine whether the content, cognitive processes and the DOK range in the evidence source identified matches closely with the SLO learning goal. [Note: You should have done this when selecting the instruments/tasks.]

2. Identify the number of student performance levels for each evidence source. If this is missing, work in teams to determine appropriate cuts for scores that correspond to a performance level (develop a rubric to capture this scoring rule).

3. Report (provide information about) the results and performance outcomes achieved for each student in reference to each evidence source considered.

4. Describe performance levels for your SLO learning goal.

5. Develop the scoring rules in consultation with other teachers to consider information across the body of evidence.

6. Capture information from steps 2-5 in a “Combining Scores Graphic Organizer” (Tools, p. ?).

Combining Scores Graphic Organizer for SLOs

The Combining Scores Graphic Organizer is a method of displaying results of assessments or other measures used for SLOs.

Combining Scores Graphic Organizer Overview

For each evidence source, indicate each of the SLO performance levels as shown in the grid below.

Unsatisfactory Partially Proficient

Proficient Advanced

The grid can be expanded to include more than one assessment instrument or task.

Assessment Instrument or

Task


Proficient Advanced

Combining Scores Graphic Organizer

Combining Scores Graphic Organizer A summation line lists values for each performance level based on the number of measures considered.

Assessment Instrument

or Task


Proficient Advanced

1 2 3 4

2 measures 2 3 4 5 6 7 8

3 measures 3 4 5 6 7 8 9 10 11 12

4 measures 4 5 6 7 8 9 10 11 12 13 14 15 16

5 measures 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20


In the following examples:

All assessment instruments/tasks are equally weighted (same points given).

Points are summed to determine student performance level for the SLO learning goal.

The total points in the summation line demarcates each performance level on the SLO learning goal.

This methodology of using a matrix to combining scores is not new. Notice the format of the continuums in one assessment (DRA2). This combines scores across subtopics of the assessment to get to an over-all rating.

Combining Scores


How does the Combining Scores Graphic Organizer work?


• Identify the assessment instruments and/or tasks (measures) to be used.

• All measures should align with the SLO learning goal.

Combining Scores Graphic OrganizerIndividual Student Evaluation

Student at end of 4th gradeAssessment

Instrument or TaskUnsatisfactory Partially

ProficientProficient Advanced

Performance Task

1 2 3 4

Benchmark

Reading 1 2 3 4

Choose assessment instruments to be used.In this scenario, a teacher is determining whether a 4th grade student is developing comprehension skills. A teacher made performance task and benchmark reading assessment scores are used.


Ranges or performance descriptors are added to each evaluation level for each measure.

Student at end of 4th grade

Assessment Instrument or Task


Proficient Advanced

Performance Task 1-5 6-10 11-16 17+

1 2 3 4

Benchmark Unsat. Part. Prof Prof. Adv.

Reading 1 2 3 4

The cut points set for the performance task that has a total of 20 points are mapped to the four SLO performance categories. This division into point ranges is not based on statistical derivations but based on agreement with ELA teachers in the same grade level.

The evaluation levels for the interim test can map onto the SLO performance levels.


Find the evaluation level for each measure. Circle the numbers for the levels.

Student at end of 4th gradeAssessment Instrument or

Task


Proficient Advanced

Performance Task

1-5 6-10 11-16 17+

1 2 3 4


1 2 3 4

2 measures 2 3 4 5 6 7 8

In this example, the student scored 14 on the Performance Task and Partially Proficient on the Benchmark test.

Combining Scores Graphic OrganizerIndividual Student Evaluation To combine scores for these two evidence sources we add the

circled numbers and circle the sum on the summation line.

The sum is 5, which falls in the partially proficient performance level for the SLO learning Goal

Final judgment: student is at the partially proficient level by the end of the instructional period on the SLO learning goal.

Student at end of 4th gradeAssessment

Instrument or TaskUnsatisfactory Partially Proficient Proficient Advanced

Performance Task 1-5 6-10 11-16 17+

1 2 3 4


1 2 3 4

2 measures 2 3 4 5 6 7 8


9th grade student-End of yearAssessment Instrument or

Task


Proficient Advanced

SRI Below Basic Basic Prof. Advanced

1 2 3 4

End of Course Unsat. Part. Prof Prof. Adv.

1 2 3 4

Class final 0-49% 50-69% 70-85% 86-100%

1 2 3 4

3 measures 3-4 5-7 8-10 11-12

In this example, the 9th grade student was proficient on the End of Course assessment and the Class final. The SRI score was at the basic level. The summation places this student in the proficient performance category for the SLO Learning Goal overall.

Interpret an example. . . Consider the “Annotated Example: Combining

Scores for SLOs Graphic Organizer”

What evidence sources were used? How many were used?

What scores were provided for each evidence source?

How were student scores across the evidence sources combined into an over-all performance level rating for the SLO?

Combining Scores Graphic OrganizerInconsistent Results. . .

5th grade student-End of yearAssessment

Instrument or TaskUnsatisfactor

yPartially

ProficientProficient Advanced

Everyday Math% of goals mastered

0-49% goals mastered

50-74% 75-89% 90-100%

1 2 3 4

Benchmark Math Unsat. Part. Prof Prof. Adv.

1 2 3 4

2 measures 2-3 4-5 6-7 8

In this example, the 5th grade student was a Proficient on the Benchmark Math Assessment. However, the student mastered only 13 Everyday Math goals which is 45% all math goals for grade five. The teacher set a goal of 75% of the EM goals should be mastered to be proficient.

Additional analysis is needed. . .

Combining Scores Graphic OrganizerInconsistent Results

5th grade student-End of yearAssessment Instrument or

Task


Proficient Advanced

Everyday Math% of goals mastered

0-49% goals mastered

50-74% 75-89% 90-100%

1 2 3 4

Benchmark Math Unsat. Part. Prof Prof. Adv.

1 2 3 4

Unit Test 0-49 50-74% 75-89% 90-100%

1 2 3 4

3 measures 3-4 5-7 8-10 11-12

Using only one or two measures is helpful, but if more data is available and used, the final judgment could be better.

Consider the effect of adding one other measure, in this cases, a unit test.

A profile becomes a bit clearer.


Continuum

Discrete categories

AdvancedProficientPartially ProficientUnsatisfactory

1 2 3 4

1 2 3 4

The reality is, though, that most assessment results lie on a continuum.

Looking at where in a range the results occur is helpful.

It may be helpful to consider categories as whole entities. That is, each category is discrete unto itself.


Assessment Instrument or

Task


Proficient Advanced

EM % of goals mastered

0-49% 50-74% 75-89% 90-100%

1 2 3 4Benchmark Math Unsat. Part. Prof Prof. Adv.

1 2 3 4EM Unit Tests 0-49 50-74% 75-89% 90-100%

1 2 3 4

3 measures 3 4 5 6 7 8 9 10 11 12

On the grid, stars are placed at the approximate locations of the observed scores. We notice that the % of goals mastered is almost at the next highest level and the Unit tests measure is almost at the next lowest level.

This does make a final evaluation of slightly below Proficient seem to make more sense. It also, however, should prompt the teacher to examine the methodology of determining if a EM goal is met. It seems inconsistent with the other evidence sources.

Combining Scores Graphic OrganizerTroubleshooting

There are a few “red flags” that prompt further analysis or drilling down into the data a bit further whether you are evaluating an individual student or a group of students. Examples include, but are not limited to:

For 2 measures, results are highly divergent.

For 3 or more measures, results span 3 evaluation levels.



Proficient Advanced

X

1 2 3 4

X

1 2 3 4

If, when considering 2 measures, the student’s scores fall into non-congruent performance levels, investigate to identify the reasons for this anomaly.


Proficient Advanced

X

1 2 3 4

X

1 2 3 4



Proficient Advanced

X1 2 3 4

X1 2 3 4

X1 2 3 4

If, when considering 3 or more measures, the student’s scores fall into more than 2 performance levels, investigate to identify the reason for this anomaly.


Where does the score fall within a range? Look at the measures used. Do they measure the same content or

goals? Were the assessments used administered during the same time

frame? Was the assessment administered properly? Are the scores valid and reliable? Were there extraneous reasons for anomalous scores (personal or

environmental distractions)? Does the student’s learning style affect performance on a particular

measure? Does the student have difficulty accessing a particular measure?

Reasons for anomalies could include, but not be limited to the following:

Group Discussion How would you refine the scoring rules? Would you weight

(give more points) to some assessments/data sources over others? Why?

What to consider if we know students are growing within a performance level, but we cannot capture these movements through this approach?

How will you define scoring rules to establish performance levels for your SLOs?

Your Turn How will you combine the scores across all of

your evidence sources to measure student learning in relationship to your SLO Learning Goal?

What rules will you establish for addressing examples when student scores across two or more evidence sources is inconsistent?

Next Steps



Using a Body of

Evidence

Reviewing Baseline

Data



Setting Performance

Targets

Terminology Consider the SLO Terminology

Work with a partner to answer the following questions:

1. What are “performance levels”? What performance levels are used in APS?

2. How would you describe a student “performance group”?

3. What are examples of “performance targets”?

SLO Performance Targets

Identify the expected outcomes by the end of the instructional period.

May differ for different student performance groups (i.e. students who started the class/course at different places with regard to their learning in relationship to the Learning Goal).

Describe the expected ending point.

SLO Performance Targets Include . .

Turn to SLO Form, SLO Component Descriptions and SLO Rating Scale

Answer these questions with a partner:What will you record in the SLO Form about

Performance Targets?

What does an Performance Target of “acceptable quality” include?

SLO Performance Targets. . . Build upon scoring at the individual student

level for the identified assessment instruments/tasks.

Include using a body of evidence (multiple measures) at both the beginning and the end of the instructional interval.

Aggregate individual student performance to student performance groups.

Determine how good is good enough at the student performance group level.

SLO Performance Targets Consider The student population

The period of time (instructional interval)

The SLO Learning Goal

How student learning in relationship to the learning goal will be measured: Evidence Sources (Assessments)

Scores/Metrics

How individual student performance will aggregated to the student performance group (what metrics)

How good is good enough at the student performance group level.

Rationale for performance targets by student performance group

Target Setting Process Turn to the “SLO Target Setting Process” (Tools p. ?)

Mark each step in the process using this legend: + = I’ve got it.

? = I have a question about this step.

√ = I could do this with some help.

Talk with a partner to: clarify questions and identify questions for the full group.

Share questions.



Using a Body of

Evidence

Reviewing Baseline

Data



Setting Performance

Targets

SLO Form

Component: Performance Targets

Information: Determining student performance groups and their performance at the beginning of the instructional interval.

Reviewing baseline dataBaseline data Student learning data collected before or at the

beginning of the instructional interval.

Measures of student learning that relates to the SLO Learning Goal

Could include: State assessment results (by student) from last year for current

students. District interim/benchmark assessment results from beginning of

the year. Results from other district - or school-wide assessments Results from classroom assessments

Why analyze baseline data? Evaluate how much initial student performance

varied at the beginning of the instructional period.

Determine if students can/should be put into more than one group based on their initial performance.

Establish a “baseline” from which student learning growth can be measured for different student performance groups.

Not to establish an initial score for every student.

What evidence sources?Talk with a partner. . .

What sources of evidence (assessment results) do you have about student performance relative to the SLO Learning Goal at the beginning of the instructional interval?

How many evidence sources have you identified? Triangulation (consider three sources if possible)

What student-level scores or metrics are available for each evidence source?

In what format do you receive or record student results?

Analyzing Baseline DataFor each evidence source:

1. Determine what scores or metrics are available from the evidence source.

2. Describe the performance of the student population or the class using each evidence source (e.g. 80% of the students were proficient; 15% were partially proficient; and 5% were unsatisfactory on TCAP in math from the prior year).

3. Consider the range of student performance (low to high). Is the variability in student performance enough to form more than one group of students based on their performance across the different evidence sources?

Across evidence sources

4. Combine data across evidence sources to describe the performance of the two to four student performance groups.

5. Establish rules for assigning students to performance groups if the following conditions apply:

a. Individual student performance is inconsistent across evidence sources.

b. All evidence sources are not available for the student.

Tool Turn to “Analyzing Baseline Data to Assign

Performance Groups” (Tools, p. ?).

Review the steps described by this tool.

Talk with a partner:Are each of the steps in analyzing baseline

data and assigning performance groups clear?

Do you have questions/concerns about any step in this process?

Assigning Performance Groups

Consider the Assigning Performance Groups Example.

This is a simpler way to combine data than the one we talked about earlier?

How was evidence combined across evidence sources to describe each performance group?

Your Turn Use “Analyzing Baseline Data to Assign

Performance Groups” (Tools, p. ?).

Consider the student results from your baseline data sources.

Develop student performance group descriptions incorporating all of your baseline data sources.



Using a Body of

Evidence

Reviewing Baseline

Data



Setting Performance

Targets

SLO Form

Component: Performance Targets

Information:Performance target for each student

performance group

Rationale for performance targets

Determining Performance Targets

Bookmark:SLO Target Setting Process (Tools, p. ?)

SLO Setting Performance Targets Worksheet (Tools, p. 15)

We are on step #6.

Setting Performance Targets Start with the SLO Learning goal and the proficient

performance level across all evidence sources.

Consider the baseline performance for each student performance group.

Determine what percent or number of the students in the performance group could realistically reach proficient performance (or better) by the end of the instructional period.

Partner Buzz: Would performance less than proficient on the SLO Learning Goal be appropriate for some students in one or two performance group(s)? What would be the rationale?

Example of Setting Expected Targets

Turn to SLO Performance Targets Example (Tools, p. ?).

Review this example with a partner.

What questions do you have about how this teacher identified performance targets for each student performance group?

How would you know the targets were attainable and rigorous?

Structuring this work. . . Work with colleagues to refine and finalize

performance targets.

Establish initial performance targets independently.

Engage with colleagues to determine if performance targets are ambitious but attainable.

Consider the SLO Rating Scale criteria for performance targets.

Your Turn: Set Performance Targets

Aggregating Data for Teacher Ratings

Multiple Students'


Multiple Assessment Instruments/Tasks (SLO Learning

Goal)

An assessment Instrument/Task (evaluates student learning relative to part of the SLO


Items (Learning Targets)

Student Results to Teacher Ratings How do student results get translated into teacher

ratings?

Still being determined in APS.

This includes. . . Determine if/how many performance targets were met. Assign a teacher rating/points related to teacher

contribution to student learning growth (including collective and individual attribution)

Assign an over-all rating.

Example: Hawaii SLO Rating Rubric

Turn to: Hawaii SLO Rating Rubric

Four teacher ratings: highly effective, effective, developing, and ineffective.

Teacher ratings depend on the number of students in the class/course.

Partner Buzz: How will student performance translate into teacher ratings in

Hawaii?

What might be difficult for teachers to understand about this kind of approach?

Jeffco Individual Educator Goals Turn to: Jeffco Individual Educator Goals

Handbook excerpt.

For 2013-14, individual educator goals contributed 15 out of 100 points of each educators evaluation.

Read: Determine Point Calculation

How did student performance in relationship to the performance targets translate into educator ratings in Jeffco?



Using a Body of

Evidence

Reviewing Baseline

Data



Setting Performance

Targets

Closing thoughts: Reasons for Optimism With the growth of the technological tools, there is:

Increased opportunities for sharing assessment instruments and practices

Increased opportunities for sharing information and providing support regarding assessment decisions

Increased opportunities for state-supported assessment tools (in contrast to state-mandated assessments).

Large-scale assessment has reached a saturation point. It will become obvious that large-scale assessment, alone, cannot meet the assessment demands of the Common Core.

Therefore…

Failure is not an option Regardless of how teacher evaluation evolves,

teachers making informed decisions about instruction, setting rigorous yet realistic goals for students, and making accurate judgments about student performance (i.e., the SLO process) is the essence of education.

If we cannot reach the point where we have confidence in the accuracy of performance information generated at the classroom and school level then what is the point…

Capturing your thinking. . .

Reflect and Consider your Learning

Return to your Progress Monitoring Form, turn to Performance Targets

Did you move to the right in your self-assessment? Add to your graph.

Make any notes about your own learning in the “reflections” column.

Give us Feedback!! Oral: Share one ah ha!

Written: Use sticky notes

+ the aspects of this session that you liked or worked for you.

The things you will change in your practice or that you would change about this session.

? Question that you still have or things we didn’t get to today

Ideas, ah-has, innovations

Leave your written feedback on the parking lot.

References Bell. B., & Cowie, B. (2001). Formative assessment and science education. Dordrecht,

Netherlands: Kluwer Academic Publishers

Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2003). Assessment for learning: Putting it into practice. Buckingham, UK: Open University Press.

Black, P., & Wiliam, D. (1998b). Inside the black box: Raising standards through classroom assessment. Phi Delta Kapan.

Bloom, B. S. (1984). The search for methods of group instruction as effective as on-to-one tutoring. Educational Leadership. 41(8): 4-17.

Bransford, J., Brown, A., & Cocking, R. (Eds). (1999). How People Learn: Brain, Mind, Experience, and School. Washington DC: National Academy Press

Meisels, S., Atkins-Burnett, S., Xue, D., Bickel, D. & Son, S. (2003). Creating a system of accountability: The impact of Instructional assessment on elementary children’s achievement test scores. Educational Policy Analysis Archives. 11(9).

Popham, W. J. (2008). Transformative assessment. Alexandria VA: ASCD.

Rodriguez, M. C. (2004). The role of classroom assessment in student performance on TIMSS. Applied Measurement in Education. 17, 1024.

Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science. 18, 119 – 144.

Stiggins, R. J., Arter, J. A., Chappuis, J., & Chappuis, S. (2006, 2004). Classroom assessment for student learning: Doing it right – using it well. Assessment Training Institute.

Documents

Student Learning Objectives Day Five: Setting Performance Targets for SLO Learning Goals