91
1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning

1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

  • View
    216

  • Download
    0

Embed Size (px)

Citation preview

Page 1: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

1

Oregon Reading FirstOregon Reading First

Institute onBeginning Reading VII:

Evaluating and Planning

Institute onBeginning Reading VII:

Evaluating and Planning

Page 2: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

2

AcknowledgmentsAcknowledgments

Oregon Department of Education

Institute for the Development of Educational

Achievement, College of Education, University of

Oregon

U.S. Department of Education,

Office of Special Education Programs

Page 3: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

3

Content developed by:

Edward J. Kame’enui

Carrie Thomas Beck

Scott K. Baker

Nicole Sherman Brewer

Ben Clarke

Hank Fien

Barbara Gunn

Rachell Katz

Trish Travers

Jennifer Walt

Content DevelopmentContent Development

Additional support:

Katie Tate

University of Oregon

Page 4: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

4

CopyrightCopyright

All materials are copy written and should not

be reproduced or used without expressed

permission of Dr. Edward J. Kame’enui or

Dr. Deborah C. Simmons. Selected slides

were reproduced from other sources and

original references cited.

Page 5: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

5

IntroductionIntroduction

Thomas Beck © 2004

Page 6: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

6

Objectives of Reading FirstObjectives of Reading First

(1) To provide assistance to State educational agencies and local educational agencies in establishing reading programs for students in kindergarten through grade 3 that are based on scientifically based reading research, to ensure that every student can read at grade level or above not later than the end of grade 3.

(2) To provide assistance to State educational agencies and local educational agencies in preparing teachers, including special education teachers, through professional development and other support, so the teachers can identify specific reading barriers facing their students and so the teachers have the tools to effectively help their students learn to read.

Page 7: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

7

Objectives of Reading First (cont.)

Objectives of Reading First (cont.)

(3) To provide assistance to State educational agencies and local educational agencies in selecting or administering screening, diagnostic, and classroom-based instructional reading assessments.

(4) To provide assistance to State educational agencies and local educational agencies in selecting or developing effective instructional materials (including classroom-based materials to assist teachers in implementing the essential components of reading instruction), programs, learning systems, and strategies to implement methods that have been proven to prevent or remediate reading failure within a State.

Page 8: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

8

Objectives of Reading First (cont.)

Objectives of Reading First (cont.)

(5) To strengthen coordination among schools, early literacy programs, and family literacy programs to improve reading achievement for all children.

Page 9: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

9

What will Reading First look like in the

years to come?

What will Reading First look like in the

years to come?

Page 10: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

10

Oregon Reading First: Year 03

Oregon Reading First: Year 03

No IBRs are planned for the third year of implementation. Cohort A Leadership Strand continues:

• Leadership Sessions in October and March

• Coaches’ Training in Sept, Nov, Feb, and April

Schools receive funds for the replacement of consumables and wear of instructional materials.

Funds provided for early literacy narrative and expository text for classroom and school libraries.

Full time Mentor Coach Professional Development:

• school-based

• substitute time for collaboration

• some statewide PD opportunities

Page 11: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

11

Oregon Reading First: Year 03

Oregon Reading First: Year 03

ongoing refinement of Summary of CSI Maps classroom observations (three in Year 03) lesson progress reports regular grade level team meetings ongoing data-based decision making end-of-year evaluation and planning

Page 12: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

12

Oregon Reading First: Year 03

Oregon Reading First: Year 03

Cohort A: Levels of Support Model

Based on the reports from RF districts and schools, subsequent technical and professional development support will be based on at least two factors:

1. Student reading data2. Level of implementation of major RF components

Page 13: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

13

aver

age

read

ing

perf

orm

ance

/

high

impl

emen

tatio

n

stro

ng re

adin

g pe

rfor

man

ce /

low

impl

emen

tatio

n

stro

ng re

adin

g pe

rfor

man

ce /

high

impl

emen

tatio

n

poor

read

ing

perf

orm

ance

/

high

impl

emen

tatio

n

aver

age

read

ing

perf

orm

ance

/

low

impl

emen

tatio

n

poor

read

ing

perf

orm

ance

/

low

impl

emen

tatio

n

OutsideIn

Support

InsideOut

Support

Figure 1: District/School Support Needs

IntensiveSupport

ModerateSupport

Supportas Needed

Page 14: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

14

Data-Based Leadership ModelData-Based Leadership Model

w

Page 15: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

15

Q1: What are our goals?

Q1: What are our goals?

Page 16: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

16

Reading Action Plan 04-05Reading Action Plan 04-05

Review progress toward meeting goals on 04-05 Reading Action Plan.

Goal 1:

Goal 2:

Goal 3:

Page 17: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

17

Q2: How are we doing?Q2: How are we doing?

Fien © 2004

Page 18: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

18

Two Approaches for Evaluating Effectiveness

Two Approaches for Evaluating EffectivenessI. Outcomes Criterion

95% of all students achieve the early literacy goal.

II. Adequate Progress Criteria – are all students making adequate progress? Core Curriculum and Instruction: Benchmark students

make adequate progress and achieve goals

Supplemental Support: Strategic students make adequate progress and achieve goals

Intensive Intervention: Intensive students make adequate progress and achieve goals or at least reduce risk.

Page 19: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

19

I. Platinum StandardI. Platinum Standard

I. Outcomes Criterion

95% of all students achieve the early literacy goal. For example, 95% of students in Kindergarten

reach established on PSF Goal of 35 in the Spring. If so,

Platinum Standard is MET!!! If not, continue with Adequate Progress

Criterion for Benchmark, Strategic and Intensive Students

Page 20: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

20

II. Silver Standard of Adequate Progress

II. Silver Standard of Adequate Progress

A school will meet silver standard if they are in the top third compared to other schools.

Page 21: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

21

Criterion Type of StandardMeets

Standard?

Index of

StrengthSchoolwide

System Outcome

Platinum Standard95% of all students meet goal

Benchmark Adequate Progress:

Meet Goal

Silver StandardUpper 3rd of Benchmark

Middle 3rd of Benchmark

Lower 3rd of Benchmark

Yes

Step-by-Step: Evaluating the Effectiveness of theBenchmark Support or Secondary Prevention

SchoolwideSystem

StrengthNo

YesNo

YesNeeds

SupportNo

YesNeeds

SubstantialSupport

Relative Strength

Page 22: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

22

Criterion Type of StandardMeets

Standard?

Index of

StrengthSchoolwide

System Outcome

Platinum Standard95% of all students meet goal

Strategic Adequate Progress:

Meet Goal

Silver StandardUpper 3rd of Strategic

Middle 3rd of Strategic

Lower 3rd of Strategic

Yes

Step-by-Step: Evaluating the Effectiveness of theStrategic Support or Secondary Prevention

SchoolwideSystem

StrengthNo

YesNo

YesNeeds

SupportNo

YesNeeds

SubstantialSupport

Relative Strength

Page 23: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

23

Criterion Type of StandardMeets

Standard?

Index of

StrengthSchoolwide

System Outcome

Platinum Standard95% of all students meet goal

Intensive Adequate Progress:

Meet Goal

Silver StandardUpper 3rd of Intensive

Middle 3rd of Intensive

Lower 3rd of Intensive

Yes

Step-by-Step: Evaluating the Effectiveness of theIntensive Support or Secondary Prevention

SchoolwideSystem

StrengthNo

YesNo

YesNeeds

SupportNo

YesNeeds

SubstantialSupport

Relative Strength

Page 24: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

24

Activity 1: Year 2 Grade Level Outcomes: By grade and in each classroom, how are students

performing at the end of the year on the essential components of RF?

Activity 1: Year 2 Grade Level Outcomes: By grade and in each classroom, how are students

performing at the end of the year on the essential components of RF?

Materials Needed Grade-level histograms Oregon Reading First histograms Breakout Forms

Directions

1. Discuss activity as a large group

2. Break into grade level teams.

3. For each grade level, document student outcomes on grade-appropriate DIBELS measures.

4. Compare your grade level outcomes with the DIBELS Platinum Standard (95% Established)

5. Compare your grade level outcomes with Oregon Reading First Schools’ overall outcomes.

Page 25: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

25

1. Spring Goal: What is the primary instructional goal for the spring of K-3rd

grade?

1. Spring Goal: What is the primary instructional goal for the spring of K-3rd

grade?

Core Component or Big Idea:_____________

DIBELS Measure: ______________________

Goal Skill Level: _______________________

Page 26: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

26

DIBELS Spring Benchmark Goals

DIBELS Spring Benchmark Goals

Grade K Goals: • 35 CSPM (correct sounds per minute) on Phoneme

Segmentation Fluency

• 25 CSPM (correct sounds per minute) on Nonsense Word Fluency

First Grade Goals:

• 40-60 CWPM (correct words per minute) on Oral Reading Fluency

• 50 CSPM (correct sounds per minute) on Nonsense Word Fluency (in Winter of first grade)

Page 27: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

27

DIBELS Spring Benchmark Goals

DIBELS Spring Benchmark Goals

Second Grade Goals:

• 90 CWPM (correct words per minute) on Oral Reading Fluency

Third Grade Goals:

• 110 CWPM (correct words per minute) on Oral Reading Fluency

Page 28: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

28

2.Spring Outcomes: At the end of the year, on the essential RF component, what percent of your students are:

2.Spring Outcomes: At the end of the year, on the essential RF component, what percent of your students are:

Established/ Low Risk:_____

Emerging/ Some Risk:_____

Deficit/ At Risk:_____

Page 29: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

29

First Grade Example

Page 30: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

30

2.Spring Outcomes: At the end first grade, on ORF, what percent of your students are:

2.Spring Outcomes: At the end first grade, on ORF, what percent of your students are:

Low Risk: 62%

Some Risk: 20%

At Risk: 17%

Did this school reach the absolute criteria of 95% established? No

Page 31: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

31

3. Compare your grade level outcomes with Oregon Reading First Schools’ overall outcomes.

3. Compare your grade level outcomes with Oregon Reading First Schools’ overall outcomes.

Insert Project wide histograms

Page 32: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

32

Take 10 minutes to Complete Activity #1 in

Grade Level Teams

Take 10 minutes to Complete Activity #1 in

Grade Level Teams

Page 33: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

33

Activity #2: Were our grade level instructional plans (Summary of CSI Maps) effective in

supporting adequate progress for students with benchmark, strategic and intensive needs?

Activity #2: Were our grade level instructional plans (Summary of CSI Maps) effective in

supporting adequate progress for students with benchmark, strategic and intensive needs?

Materials Needed: Summary of Effectiveness Reports by School, Summary of Effectiveness

Reports by Class, Oregon Reading First Schools’ Summary of Effectiveness Reports, Adequate Progress School Norms

Directions:1. Discuss activity as a large group2. Document adequate progress for each grade level.3. Compare grade-level adequate progress with National Norms4. Compare grade-level adequate progress with Oregon Reading

First schools

Page 34: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

34

DIBELS Summary of Effectiveness Reports4 Ways to Achieve Adequate Progress

DIBELS Summary of Effectiveness Reports4 Ways to Achieve Adequate Progress

Time 1 (e.g., winter)

Time 2 (e.g., spring)

At-Risk Intensive 1. Some Risk 2. Low Risk

At-Risk Strategic Some Risk 3. Low Risk

At-Risk Benchmark Some Risk

4. Low Risk

Page 35: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

35

How are we doing at the systems level (school)?

How are we doing at the systems level (school)?

School: Were our grade level instructional plans (Summary of CSI Maps) effective in supporting adequate progress for students with benchmark, strategic and intensive needs?

For each grade level: What percent of benchmark students reach established (low risk) on

the spring benchmark goal? What percent of strategic students reach established (low risk) on the

spring benchmark goal? What percent of intensive students reach established (low risk) on the

spring benchmark goal? What percent of intensive students reach emerging (some risk) on the

spring benchmark goal?

Data Source: Summary of Effectiveness by School and Summary of Effectiveness by Class

Page 36: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

36

1. Initial Skills:1. Initial Skills:

In the winter of kindergarten, what percentage of students were:

Benchmark: ____

Strategic: ____

Intensive: ____

Page 37: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

37

Page 38: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

38

2. Adequate Progress Of Benchmark Students:

2. Adequate Progress Of Benchmark Students:

Of the students who were Benchmark in the middle of kindergarten, what percent achieved the PSF goal of 35 for the end of kindergarten? ___________

Page 39: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

39

Page 40: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

40

2. Adequate Progress Of Benchmark Students:

2. Adequate Progress Of Benchmark Students:

Of the students who were Benchmark in the middle of kindergarten, what percent achieved the PSF goal of 35 for the end of kindergarten? 60%

Page 41: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

41

3. How would you rate the effectiveness of the core curriculum

and instruction?

3. How would you rate the effectiveness of the core curriculum

and instruction?

Relative Strength- 96% to 100% of benchmark students achieve PSF goal.

Needs Support- 86% to 95% of benchmark students achieve PSF goal.

Needs Substantial Support- 0% to 85% of benchmark students achieve PSF goal.

Page 42: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

42

4. Adequate Progress Of Strategic Students:

4. Adequate Progress Of Strategic Students:

Of the students who were Strategic in the middle of kindergarten, what percent achieved the PSF goal of 35 for the end of kindergarten? ___________

Page 43: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

43

Page 44: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

44

4. Adequate Progress Of Strategic Students:

4. Adequate Progress Of Strategic Students:

Of the students who were Strategic in the middle of kindergarten, what percent achieved the PSF goal of 35 for the end of kindergarten? 32%

Page 45: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

45

5. How would you rate the effectiveness of the school-wide system of strategic

support:

5. How would you rate the effectiveness of the school-wide system of strategic

support:

Relative Strength- 75% to 100% of strategic students achieve PSF goal.

Needs Support- 50% to 74% of strategic students achieve PSF goal.

Needs Substantial Support- 0% to 49% of strategic students achieve PSF goal.

Page 46: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

46

6. Adequate Progress Of Intensive Students:

6. Adequate Progress Of Intensive Students:

Of the students who were Intensive in the middle of kindergarten, what percent made adequate progress towards the PSF goal of 35 for the end of kindergarten? ___________

Page 47: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

47

Page 48: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

48

6. Adequate Progress Of Intensive Students:

6. Adequate Progress Of Intensive Students:

Of the students who were Intensive in the middle of kindergarten, what percent made adequate progress towards the PSF goal of 35 for the end of kindergarten? 77%

Page 49: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

49

7. How would you rate the effectiveness of the school-wide system of intensive

support:

7. How would you rate the effectiveness of the school-wide system of intensive

support:

Relative Strength- 80% to 100% of intensive students achieve PSF goal or are emerging.

Needs Support- 57% to 79% of intensive students achieve PSF goal or are emerging.

Needs Substantial Support- 0% to 56% of intensive students achieve PSF goal or are emerging.

Page 50: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

50

Some other information on the Summary of Effectiveness Report:

% of students for each instructional recommendation that do not make adequate progress or

Page 51: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

51

Track individual students’ progress

D. Mariah an intensive student that made adequate progress to established

H. Adrian a benchmark student that did not make adequate progress

Page 52: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

52

8. Compare your school’s adequate progress with other Oregon Reading

First schools. Based on this comparison:

8. Compare your school’s adequate progress with other Oregon Reading

First schools. Based on this comparison:

Discuss areas of strength. (e.g., Compared to the project wide data, our grade has a higher percentage of strategic students meeting spring time benchmarks.)

Discuss areas of needed student support. (e.g., Compared to the project wide data, our grade has a lower percentage of intensive students meeting spring time benchmarks).

Page 53: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

53

Oregon Reading FirstProject Level Summary of

Effectiveness2004-2005: Beginning of First Grade to

Middle of First Grade

Oregon Reading FirstProject Level Summary of

Effectiveness2004-2005: Beginning of First Grade to

Middle of First Grade

Page 54: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

54

Take 30 minutes to Complete Activity #2 in

Grade Level Teams

Take 30 minutes to Complete Activity #2 in

Grade Level Teams

Page 55: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

55

Activity #3: Discussion PointsActivity #3: Discussion Points

How did the Fall, Winter and Spring Benchmarking Assessment plan work this year? Any ideas for next year?

How did the Progress Monitoring Plan work this year? Any ideas for next year? If the teachers were not involved this year, how could they get involved next year?

Do we need brush-up DIBELS training next fall? Could we train our EA’s, parent volunteers? How will we train new staff?

What will we do to ensure continued reliable DIBELS data collection? Could we utilize the Assessment Integrity Checklist?

Could we refine how we use data in grade level team meetings?

Page 56: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

56

Activity #4: (Optional)Choose 1 Activity from a-c

Activity #4: (Optional)Choose 1 Activity from a-c

a) Transitioning ActivityMaterials Needed: Class reports, Summary of

Effectiveness Reports Kindergarten Team join with the First grade Team and discuss

Kindergarten students transitioning to first grade. Second Grade Team join with Third Grade Team and discuss second

graders transitioning to third grade. For each, discuss possible grouping strategies for fall that could

maximize instructional intensity.

b) Disaggregating ActivityMaterials Needed: Distribution Reports Discuss adequate progress for sub-groups of students (ELL, Sped,

transfer) relevant to your school.

c) Yearly Progress ActivityMaterials Needed: Cross Year Box Plots Discuss Year 2 RF outcomes compared to Year 1.

Page 57: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

57

Activity #5: How are we doing at the systems level (district)? (optional)

Activity #5: How are we doing at the systems level (district)? (optional)

District: Across the district, were our grade level instructional plans (Summary of CSI Maps) effective in supporting adequate progress for students with benchmark, strategic and intensive needs?

For each school and at each grade level: What percent of benchmark students reach established

(low risk) on the spring benchmark goal? What percent of strategic students reach established

(low risk) on the spring benchmark goal? What percent of intensive students reach established

(low risk) on the spring benchmark goal? What percent of intensive students reach emerging

(some risk) on the spring benchmark goal? Data Source: Summary of Effectiveness by District

Page 58: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

58

Page 59: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

59

Q3: How do we get there?Q3: How do we get there?

Thomas Beck, Sherman Brewer, Katz © 2004

Page 60: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

60

Linking Q2: How are we doing? with

Q3: How do we get there?

Linking Q2: How are we doing? with

Q3: How do we get there?

How are we doing with our core implementation? What do the data tell us?

How is the combination of current programs/strategies working for our lower performing students?

So what do we do next year?

Page 61: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

61

1. • How are we doing with our core implementation?

• What do the data tell us?

1. • How are we doing with our core implementation?

• What do the data tell us?

Activity # 1A: Evaluation of core implementationMaterials Needed: Activity #2 from Assessment Section Revisit the Summary of Effectiveness data from Activity #2

of the Assessment section. In particular, examine the number of benchmark students who met benchmark goals over the course of the year. Are most benchmark students remaining on track? What are the trajectories like for benchmark students?

Page 62: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

62

Questions to Consider with Core Implementation:

Questions to Consider with Core Implementation:

INFRASTRUCTURE:

Are all necessary teacher and student materials available in each classroom?

Is a sufficient amount of time allocated? Is a sufficient number of staff allocated? Are students grouped based on program

recommendations? Are group sizes for large and small group activities appropriate?

Page 63: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

63

Questions to Consider with Core Implementation:

Questions to Consider with Core Implementation:

QUALITY OF IMPLEMENTATION:

Are important activities taught/stressed (e.g., red checks, targets, etc.)? Keep in mind the differing priorities for K vs. 1 vs. 2 vs. 3. Revisit the instructional maps.

Were study groups formed to focus on particular big ideas in beginning reading?

Have the grade level teams worked together to systematically enhance the program as necessary?

How much time are students spending on independent activities? What types of independent activities are provided?

Is the in-program assessment administered regularly and results used to inform instruction?

Are students meeting projections for lesson progress? Is the program implemented with fidelity?

Page 64: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

64

Activity # 1B: Fidelity of ImplementationMaterials Needed: Fidelity checklists completed by

coaches in the Fall, Winter, and Spring (including documentation collected for alternate observation formats).

Coach summarizes common issues related to core implementation based on fidelity observations conducted in the Fall, Winter, and Spring. What are common trends in K, 1, 2, 3? What are teachers implementing well? Are there particular activities with which teachers have difficulty (e.g., phonics, PA, etc.)?

1.• How are we doing with our core implementation?

• What do the data tell us? (cont.)

1.• How are we doing with our core implementation?

• What do the data tell us? (cont.)

Page 65: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

65

To improve quality of core program implementation, consider the following:

To improve quality of core program implementation, consider the following:

Is ongoing, high quality training provided? Are teachers receiving support from the RF coach? in the

classroom? outside the classroom? Are regular inservice sessions developed around

implementation issues identified by coach? Do teachers have opportunities to observe model lessons

- from coach, from peers, from other schools? Is our school a candidate to participate in program-

specific training through WRRFTAC? Are program-specific consultants brought in to observe

in classrooms and provide ongoing support and training? Are new teachers provided the necessary program

training?

Page 66: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

66

So what do we do next year?

So what do we do next year?

Activity # 1C: Plan for Fall

Materials Needed:

• Winter to Spring CSI Maps

• blank Fall to Winter Summary of CSI Maps

• DIBELS data from assessment activities 1 and 2

• sample schedules and CSI maps from exemplary schools

Complete grade-level Summary of CSI Maps to plan instruction for benchmark students in the Fall. To prepare for this activity, complete the following procedures:

Page 67: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

67

To Prepare:To Prepare:

Review progress of benchmark students in each grade level. Identify grade level systems that will require support.

K:

1:

2:

3:

Page 68: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

68

To Prepare:To Prepare:

For each grade level system that will require support, list relevant infrastructure and quality of implementation issues:

K:

1:

2:

3:

Page 69: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

69

To Prepare:To Prepare:

Using the relevant infrastructure and quality of implementation issues as a guide, identify reading action plan goals for each grade level. Note that if benchmark students are making adequate progress within a grade level, it is important to consider how you will sustain the system over time.

Page 70: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

70

Grade Level RAPsGrade Level RAPsInstitute on Beginning Reading VII

Grade Level Reading Action Plan

2005-2006

School:

Grade:

GOAL 1:

Who:

When:

GOAL 2:

Who:

When:

GOAL 3:

Who:

When:

Page 71: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

71

Professional Development NeedsProfessional Development Needs

Determine the professional development that will be necessary for staff to meet the grade level Reading Action Plan goals. List below:

Page 72: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

72

To Prepare:To Prepare:

Examine sample schedules and Summary of CSI Maps from exemplary RF schools.

Are there ideas you can apply to your grade-level instructional plans?

Page 73: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

73

So what do we do next year?So what do we do next year?

Now, you are ready to fill out your maps. Break into grade-level teams and use the information from the activities that were just completed to develop grade-level Summary of CSI Maps to plan instruction for benchmark students in the Fall of 2005.

Page 74: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

74

Summary of CSI Map

TimePeriod

InstructionalRecommend-

ation

Participation in Core Supplemental andIntervention Programs/

Strategies

Supplemental andIntervention

Program Delivery

Frequency ofDIBELSProgress

Monitoring

Determining InstructionalEffectiveness

benchmark: Who:

When:

Activities:

Group Size:

Who:

When:__ w/in 90 minutes__ outside of 90 min

Time:

Group Size:

Who:

How Often:

Criteria:

strategic: Who:

When:

Activities:

Group Size:

Who:

When:__ w/in 90 minutes__ outside of 90 min

Time:

Group Size:

Who:

How Often:

Criteria:

Fall toWinter

intensive: Who:

When:

Activities:

Group Size:

Who:

When:__ w/in 90 minutes__ outside of 90 min

Time:

Group Size:

Who:

How Often:

Criteria:

Page 75: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

75

Activity # 2A: Evaluation of Grade-Level DataMaterials Needed: Activity #1 and #2 from Assessment

section, Winter to Spring CSI Maps

Revisit Winter to Spring DIBELS summary tables from Activity #1 of assessment section and summary of effectiveness reports from Activity #2. Use this data to examine the effectiveness of instruction provided to intensive and strategic students from Winter to Spring. Specifically, be prepared to address the following items:

2. How is the combination of current programs/strategies working for our lower performing students?

2. How is the combination of current programs/strategies working for our lower performing students?

Page 76: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

76

Evaluation of Grade-Level Data (cont.)

Evaluation of Grade-Level Data (cont.)

Using spring histograms, discuss how current instructional practices are addressing students needs in the following areas:• Phonemic Awareness• Alphabetic Principal• Fluency with Connected Text

Using the summary of effectiveness reports, examine adequate progress of intensive and strategic students from winter to spring.

Page 77: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

77

Evaluation of Grade-Level Data for Strategic and Intensive Systems

Evaluation of Grade-Level Data for Strategic and Intensive Systems

IF . . . THEN

If grade level system meets the platinum standard . . .

proceed to activity #2C to evaluate performance of individual students. Address the issue of sustainability.

If strategic/intensive system is a relative strength (silver) . . .

consider QUALITY OF IMPLEMENTATION questions on following slide. Follow up by examining performance of individual students (activity #2C).

If strategic/intensive system needs support or needs substantial support . . .

begin by considering INFRASTRUCTURE questions on following slide. Complete activity #2B.Then address QUALITY OF IMPLEMENTATION questions.

Page 78: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

78

Questions to Consider with Supplemental and Intervention

Implementation:

Questions to Consider with Supplemental and Intervention

Implementation:

INFRASTRUCTURE:• Are appropriate reading programs and materials being used to teach the full range of students across K-3? (see next slide)• Is adequate instructional time scheduled for students who are struggling? Within the 90-minute block? Outside the 90-minute block?• Have staff been assigned in a way such that reading instruction can be delivered to the full range of students each day?• Were students grouped appropriately for reading instruction?

QUALITY OF IMPLEMENTATION:• Have staff received professional development on programs used in classrooms prior to implementation and at least twice after initial training?• Were the reading programs and materials implemented with fidelity? Are efforts to improve fidelity working?• Did teachers focus on the general features of instruction that are critical for students who are struggling? (e.g., modeling, opportunities for practice, etc.)• Did grade level teams regularly analyze student reading data, plan instruction based on data, and regroup students based on the data?

Page 79: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

79

2. How is the combination of current programs/strategies working for our lower

performing students? (cont.)

2. How is the combination of current programs/strategies working for our lower

performing students? (cont.)

Activity #2B: Examine whether appropriate reading programs and materials are being used to teach the full range of students.Materials Needed: Oregon Reading First’s list of approved supplemental and intervention programs and corresponding reviews, sample Summary of CSI Maps from exemplary RF schools

Are we implementing too many supplemental and intervention programs with our lower performing students?

Do we need to focus on providing enhancements/”boosters” to the core program? Have we selected the strongest intervention programs to implement with our intensive

students?

Are there other intervention programs that may better address our students’ needs? List here.••••

How will the S-I programs fit within your overall grade-level and school-level reading plan?

Page 80: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

80

2. How is the combination of current programs/strategies working for our lower performing students? (cont.)

2. How is the combination of current programs/strategies working for our lower performing students? (cont.)

Activity # 2C: Evaluation of Progress Monitoring Data (OPTIONAL)Materials Needed: Progress monitoring booklets for strategic and

intensive students in each grade level, master list of student groups, Winter to Spring CSI Maps, Alterable Variables Chart

Working in grade-level teams, identify one instructional group of intensive students and one instructional group of strategic students. Gather progress monitoring booklets for these students. If you haven’t already done so, draw in the aim line for each student on each measure. Using the 3-point rule, evaluate the effectiveness of the intervention provided from winter to spring.

Page 81: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

81

So what do we do next year?

So what do we do next year?

Activity # 2D: Plan for FallMaterials Needed: • Winter to Spring CSI Maps• blank Fall to Winter Summary of CSI Maps• DIBELS data from assessment activities 1 and 2• list of approved supplemental and intervention programs• program reviews for adopted supplemental and intervention

programs• sample schedules and CSI maps from exemplary schools

Complete grade-level Summary of CSI Maps to plan instruction for strategic and intensive students in the Fall. To prepare for this activity, complete the following procedures:

Page 82: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

82

To Prepare:To Prepare:

Review progress of strategic and intensive students in each grade level. Identify grade level systems that will require support.

K:

1:

2:

3:

Page 83: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

83

To Prepare:To Prepare:

For each grade level system that will require support, list relevant infrastructure and quality of implementation issues:

STRATEGIC INTENSIVE

K

1

2

3

Page 84: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

84

To Prepare:To Prepare:

Using the relevant infrastructure and quality of implementation issues as a guide, identify reading action plan goals to address the needs of strategic/intensive students for each grade level. Note that if strategic / intensive students are making adequate progress within a grade level, it is important to consider how you will sustain the system over time.

Page 85: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

85

Grade Level RAPsGrade Level RAPsInstitute on Beginning Reading VII

Grade Level Reading Action Plan

2005-2006

School:

Grade:

GOAL 1:

Who:

When:

GOAL 2:

Who:

When:

GOAL 3:

Who:

When:

Page 86: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

86

Professional Development NeedsProfessional Development Needs

Determine the professional development that will be necessary for staff to meet the grade level Reading Action Plan goals. List below:

Page 87: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

87

To Prepare:To Prepare:

Examine sample schedules and Summary of CSI Maps from exemplary RF schools.

Are there ideas you can apply to your reading plan?

Page 88: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

88

To Prepare:To Prepare:Review important instructional components of Oregon Reading First’s Schoolwide Beginning Model:

EVALUATION CRITERIA DOCUMENTATION OF EVIDENCE

V. Differentiated Instruction/Grouping/Scheduling - Instruction optimizes learning for all students by

tailoring instruction to meet current levels of knowledge and prerequisite skills and organizing instruction

to enhance student learning.

1. Student performance is used to determine

the level of instructional materials and to select

research-based instructional programs.

2. Instruction is provided in flexible

homogeneous groups to maximize student

performance and opportunities to respond.

3. For children who require additional and

substantial instructional support, tutoring (1-1) or small

group instruction (< 6) is used to support teacher-

directed large group or whole class instruction.

4. Group size, instructional time, and

instructional programs are determined by and adjusted

according to learner performance (i.e., students with

greatest needs are in groups that allow more frequent

monitoring and opportunities to respond and receive

feedback).

5. Cross-class and cross-grade grouping is

used when appropriate to maximize learning

opportunities.

Page 89: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

89

So what do we do next year?So what do we do next year?

Now, you are ready to fill out your maps. Break into grade-level teams and use the information from the activities that were just completed to develop grade-level Summary of CSI Maps to plan instruction for strategic and intensive students in the Fall of 2005.

Page 90: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

90

Summary of CSI Map

TimePeriod

InstructionalRecommenda

-tion

Participation in Core Supplemental andIntervention Programs/

Strategies

Supplemental andIntervention

Program Delivery

Frequency ofDIBELSProgress

Monitoring

Determining InstructionalEffectiveness

benchmark: Who:

When:

Activities:

Group Size:

Who:

When:__ w/in 90 minutes__ outside of 90 min

Time:

Group Size:

Who:

How Often:

Criteria:

strategic: Who:

When:

Activities:

Group Size:

Who:

When:__ w/in 90 minutes__ outside of 90 min

Time:

Group Size:

Who:

How Often:

Criteria:

Fall toWinter

intensive: Who:

When:

Activities:

Group Size:

Who:

When:__ w/in 90 minutes__ outside of 90 min

Time:

Group Size:

Who:

How Often:

Criteria:

Page 91: 1 Oregon Reading First Institute on Beginning Reading VII: Evaluating and Planning Institute on Beginning Reading VII: Evaluating and Planning

91

Next Steps: Prioritize PD NeedsNext Steps: Prioritize PD Needs

FALL WINTER SPRING

K

1

2

3