stu.westga.edustu.westga.edu/~ksharp1/8480_evalreport_kas.docx · Web viewOne of the reasons the assessments took so long was a result of the inaccuracies with the Word ... detailed

Running Head: EVALUATION REPORT 1

Fountas & Pinnell Benchmark Assessment System

Prepared by Kimberly A. Sharp

Prepared for Yvonne Frey and Ivy Creek Elementary School

November 1, 2010

EVALUATION REPORT 2

Executive Summary

Background Information

Ivy Creek Elementary School is a seven year old school located in Buford, Georgia. It is

comprised of 886 students in grades PreK-Fifth Grade with the following demographics: 58%

Caucasian, 20% Hispanic, 10% African – American, 8% Asian and 4% Multi-Racial. The

majority of students come from families with an average socioeconomic status.

The school was recently invited to pilot the Fountas & Pinnell Benchmark Assessment

System largely as a result of the efforts of the reading coach at Ivy Creek Elementary School.

The Fountas & Pinnell Benchmark Assessment Program is designed to measure student’s

reading levels with the use of benchmark books. The information gained from a personalized

assessment session with each student provides the teacher with valuable insight into the reading

ability of the student and the concepts that need to be addressed during reading instruction

(Fountas & Pinnell, 2008.)

Purpose

The administrative staff at Ivy Creek Elementary School requested that this program be

evaluated to determine the impact of the assessment system on reading instruction and student

achievement in reading. This assessment system is closely aligned with the district’s views of

reading instruction. The Language Arts Department feels confident that the Fountas & Pinnell

Benchmark Assessment Program meets the needs of reading teachers.

Now that this formative evaluation project is complete, the leaders at Ivy Creek

Elementary School can examine how the Fountas & Pinnell Benchmark Assessment System is

impacting classroom teachers and their reading instruction. Additional professional learning

opportunities will be offered to teachers based on the needs outlined in this evaluation.

EVALUATION REPORT 3

Evaluation Questions

What impact do professional learning sessions have on teachers as they

implement the Fountas & Pinnell Benchmark Assessment System?

How does the Fountas & Pinnell Benchmark Assessment System impact reading

instruction?

How does the Fountas & Pinnell Benchmark Assessment System compare to the

assessment program used last year?

Should the Fountas & Pinnell Benchmark Assessment System be adopted for use

at Ivy Creek Elementary School next year?

Methods

All 37 certified staff members who administer the Fountas & Pinnell Benchmark

Assessment System at Ivy Creek Elementary were asked to participate in this study. Qualitative

data was gathered from open-ended and closed-ended questions in surveys (Appendix A) and

one member from each grade level was interviewed using the interview guide approach

(Appendix B.) Informal student assessment data was used to determine if students appear to be

progressing in the area of reading achievement. Because the formal assessment was not

administered before the end of this project, the evaluator examined student work and relied on

teacher observation to determine if students are showing signs of achievement.

Key Findings

The results of the survey indicated that teachers were pleased overall with the training

they received and that the initial professional development session during pre-planning was the

most helpful. Several teachers indicated that they especially appreciated watching the video of

an expert conducting an actual assessment and comparing the expert results with their own. All

EVALUATION REPORT 4

of the teachers interviewed expressed a desire to spend more time with the supplemental

materials in the kit.

According to the survey, teachers found that the Fountas & Pinnell Benchmark

Assessment Program was implemented smoothly. In each of the subsequent interviews, teachers

shared that the results of their assessments were extremely accurate. They were pleased with the

program in that it has enabled them to form solid reading groups and paint a truer picture of each

student’s reading ability. Notably, they described that the assessment period was originally

scheduled to last for two weeks, but it actually took four weeks to complete. Teachers also

explained that they believed this process would be faster next year because all of the students

who were tested this year would begin with a consistent starting point.

One of the reasons the assessments took so long was a result of the inaccuracies with the

Word Lists. This was a significant source of frustration for many teachers. However, the

teachers did like the recording forms as a communication tool to use during parent-teacher

conferences. They especially found that having copies of the recording forms and assessments

available in the teacher workrooms was helpful for logistical reasons.

The majority of teachers found that they liked the Fountas & Pinnell Benchmark

Assessment Program as much as or more than programs they have used in the past. They cited

the accuracy of the results and the amount of information gathered for each student as a benefit.

They also liked the fact that the assessment books were shorter with more complex text.

The survey showed that teachers like this program enough to agree with adopting the

program at Ivy Creek and within Gwinnett County. There were no noticeable differences in the

responses of teachers when cross-referenced by age, teaching experience and degree.

EVALUATION REPORT 5

Recommendations

Training and Professional Development

Professional development instructors should take into consideration the meeting schedule

as well as the number of days available to prepare for the beginning of school. Training should

be adjusted to account for experiential differences by allowing teachers to attend a beginning,

intermediate or advanced session based on their needs. When possible, instructors should use an

actual assessment as a model for teachers attending training.

Future professional learning sessions should be focused on the contents of the assessment

kits and the use of the supplemental materials provided by the Fountas & Pinnell Benchmark

Assessment Program. It would be appropriate to offer a variety of scheduled sessions (i.e.

before or after school, summer, etc.) in an effort to accommodate everyone.

Implementation and Reading Instruction

Additional professional learning on how to practically use the error information in

reading instruction would be valuable to teachers. In order to overcome the time constraints of

teachers trying to assess all students in a timely fashion, perhaps classes could be covered by

support staff or substitute teachers for a period of time so that teachers’ schedules were free to

conduct assessments during the first two weeks of school. Professional development could also

be offered to share practical strategies for conducting assessments throughout the school day.

The issue of mid-term grading is a concern that should also be addressed by grade levels.

A common strategy for reporting student progress needs to be developed. Informal assessments

revealed that students had improved at least one or two levels since the beginning of the school

year. However, it is unclear if this was because of the strategies implemented as a result of the

EVALUATION REPORT 6

Fountas & Pinnell Benchmark Assessment Program, the efforts of the teachers conducting

quality reading instruction or a combination of both.

Resource materials that accompany the program need to be adjusted as well. The

recording forms and assessments need to be modified so that they are easier to read. It was

suggested that these documents be placed on the Share Drive for all teachers to easily access in

addition to providing copies in the teacher workrooms.

Finally, Ivy Creek should invest in three more kits to share with intermediate grade

levels. This will eliminate the need for sharing kits amongst grade levels during the limited time

available for assessments to be completed.

Comparison to Other Programs

While teachers liked the variety of texts offered by this program, they would like to have

several options available so that students are looking at a new text for each of the upcoming

assessments. This may be beyond the control of the local school or county officials, but it is a

suggestion worth offering to the company producing the Fountas & Pinnell Benchmark

Assessment Program materials.

Adoption for Ivy Creek and Gwinnett

Overall, teachers recommended that the Fountas & Pinnell Benchmark Assessment

Program be adopted by Ivy Creek Elementary and by Gwinnett County. There was some

confusion about this portion of the survey that will need to be addressed with teachers during the

presentation of the findings of this evaluation report. I believe that the confusion was due to the

teachers’ familiarity with the work of Fountas & Pinnell. However, contrary to the opinion of

some, Ivy Creek had not utilized the Fountas & Pinnell Benchmark Assessment Program in the

past.

EVALUATION REPORT 7

Conclusion

The key findings of this evaluation report on the Fountas & Pinnell Benchmark

Assessment Program were overwhelmingly positive. Teachers at Ivy Creek Elementary School

indicated that they were pleased with the professional development sessions, the quality of the

materials, the impact the program had on reading instruction and student achievement.

Generally, they felt that this assessment program was as good as or better than programs they

had previously used and they recommended that the Fountas & Pinnell Benchmark Assessment

Program be adopted for continued use at Ivy Creek Elementary School and within Gwinnett

County. While the initial indications are very positive, by following the recommendations

mentioned above, the school has an opportunity to make the Fountas & Pinnell Benchmark

Assessment Program even more beneficial to teachers.

Based on the findings in this report, I believe that the Fountas & Pinnell Benchmark

Assessment Program has the potential for improving reading instruction and student achievement

at Ivy Creek Elementary. Professional development will be more focused on the specific needs

of teachers now that the general instruction is complete. As teachers grow more comfortable

with utilizing the program, I believe they will become more efficient in conducting their

assessments and they will begin to apply what they learn about student errors to their reading

instruction. Ultimately, both the students and teachers will benefit from the implementation of

this program.

EVALUATION REPORT 8

Evaluation Report: Fountas & Pinnell Benchmark Assessment System

Introduction


Ivy Creek Elementary School is located in northeastern Gwinnett County in Buford,

Georgia. Founded in 2003, the school is comprised of 886 students in grades PreK-Fifth Grade.

At a recent faculty meeting, the principal of Ivy Creek, Yvonne Frey, shared statistics about the

students attending the school. Demographically, the student population is 58% Caucasian, 20%

Hispanic, 10% African – American, 8% Asian and 4% Multi-Racial. Approximately 29% of

students receive free or reduced price lunch while the majority of students come from families

with average socioeconomic status (Y. Frey, personal communication, May 17, 2010).

Gwinnett County prides itself on becoming a system of world-class schools. As a

member of the largest school district in the state of Georgia, Ivy Creek Elementary has enjoyed

the support of an active Language Arts Department. The school was recently invited to pilot the

Fountas & Pinnell Benchmark Assessment System largely as a result of the efforts of the reading

coach at Ivy Creek Elementary School.

The administrative staff at Ivy Creek Elementary School requested that this program be

evaluated to determine the impact of the assessment system on reading instruction and student

achievement in reading. This evaluation was designed to provide valuable information to the Ivy

Creek Elementary School Stakeholders: Principal, Assistant Principals, Reading Coach,

Teachers and Staff. In addition, the results of the evaluation are available for review by the

county office staff. These data may have an impact on whether the program is recommended to

all elementary schools in the county. As the media specialist at Ivy Creek Elementary School

EVALUATION REPORT 9

and one of the few teachers not directly involved with administering the Fountas & Pinnell

Benchmark Assessment System, I conducted an objective evaluation of the program.

Evaluation Program

Fountas and Pinnell have spent more than twenty years establishing a leveling system for

educators to use when evaluating students’ reading ability. Teachers are able to determine

whether a student is meeting the standards set for each grade level by using leveled books and

measuring a student’s reading fluency and comprehension. Levels follow a twenty-six point

scale from the easiest level (A) to the most challenging level (Z.) These alphabetical levels

correspond with reading ability levels ranging from Kindergarten to 8th Grade (Fountas &

Pinnell, 2008.)

The Fountas & Pinnell Benchmark Assessment Program is designed to measure student’s

reading levels with the use of benchmark books. These leveled books are used during a one-on-

one session between the teacher and the student. During these sessions, the student reads the

book aloud and discusses the book while the teacher observes and codes the behavior of the

reader. This system of coding allows the teacher to identify student errors with regards to

fluency and comprehension. The information gained from this personalized assessment session

provides the teacher with valuable insight into the reading ability of the student and the concepts

that need to be addressed during reading instruction (Fountas & Pinnell, 2008.)

Purpose

The purpose of conducting an evaluation of the Fountas & Pinnell Benchmark

Assessment System at Ivy Creek Elementary School was to determine the impact it has on

reading instruction and student achievement. The program is designed so that teachers

administering the assessment are able to use student errors to guide instruction. As a result,

EVALUATION REPORT 10

reading instruction is more focused and allows students to progress more rapidly through the

stages of reading.

This assessment system is closely aligned with the district’s views of reading instruction.

Specifically, the program correlates with the use of guided reading and reading workshops as

methods of delivery. In a program study of a reading initiative designed to improve reading

achievement, Al Otaiba and Hosp recommended that assessment programs should align with

district reading programs (2008.) The Language Arts Department feels confident that the

Fountas & Pinnell Benchmark Assessment Program meets the needs of reading teachers.

Unfortunately, complete student achievement data was unavailable during the time

constraints for this project. While there is some indication of student progression in reading,

official midterm and year-end assessments will not be administered until January and again in

May. Therefore, the focus of this project was on professional learning related to implementing

the assessment system and the impact this program is having on teacher instruction.

Now that this formative evaluation project is complete, the leaders at Ivy Creek

Elementary School have a true picture of how the Fountas & Pinnell Benchmark Assessment

System is impacting classroom teachers and their reading instruction. Additional professional

learning opportunities will now be offered to teachers based on the needs outlined in this

evaluation. This training will be provided to teachers prior to the administration of the midterm

and year-end assessments. The data collected during the course of this project has also provided

professional learning instructors with specific feedback on how to further guide teachers in

administering effective reading instruction.

These steps reinforce the ideas suggested by Engstrom and Danielson (2006) in their

review of a professional development program on Howard Gardner’s multiple intelligence


theory. They found that teachers need ongoing support when implementing a new program.

Additionally, they recommended that action research be conducted throughout program

implementation so that professional development could be focused on the needs of teachers. It is

also critical for teachers to have sufficient time and resources to understand the intent of new

standards and the structure of new curriculum (Obara & Sloan, 2009).

Once the needs of teachers have been identified, professional development opportunities

can be designed to address those needs. It is important to narrow the focus of these sessions to

the most relevant topics while integrating specific content and teaching ideas. It is also

necessary to provide teachers with networking time and long-term support (Hutchins, Arbaugh,

Abell, Marra, & Lee, 2008) to ensure that the implementation of the Fountas & Pinnell

Benchmark Assessment Program is done correctly.

After teachers have administered the mid-term and post-evaluations, the evaluator and the

client plan to review student reading achievement data; CRCT scores, Guided Reading Levels,

and Reading Standards. These hard data should serve to reinforce the qualitative data obtained

as a result of this evaluation project (Kaufman, Guerra, & Platt, 2006.) Although these data will

be obtained beyond the scope of this evaluation, it will be necessary to include student

achievement results in order to complete the full program evaluation. This summative evaluation

is scheduled for completion by the end of the 2010-11 school year.

Evaluation Questions

In order to provide meaningful information to Ivy Creek Elementary School stakeholders

through this evaluation, the following questions were answered:

What impact do professional learning sessions have on teachers as they implement

the Fountas & Pinnell Benchmark Assessment System?


How does the Fountas & Pinnell Benchmark Assessment System impact reading

instruction?

How does the Fountas & Pinnell Benchmark Assessment System compare to the

assessment program used last year?

Should the Fountas & Pinnell Benchmark Assessment System be adopted for use at

Ivy Creek Elementary School next year?

MethodsParticipants

All certified staff members who administer the Fountas & Pinnell Benchmark

Assessment System were asked to participate in this study. This group included thirty-three

classroom teachers in Grades 1-5, the reading coach, one intervention specialist and two gifted

reading teachers.

Design and Procedures

A variety of activities were designed to address all of the objectives outlined in this

evaluation report. A qualitative research approach was used in that data was gathered from

teacher surveys (Appendix A) and interviews (Appendix B). Questions were designed to

evaluate teachers’ perceptions of their professional learning experience as related to the Fountas

& Pinnell Benchmark Assessment System. Additional questions addressed the impact the

program had on the way teachers delivered reading instruction. Comparing the Fountas &

Pinnell Benchmark Assessment System to the reading programs used in years past was an

important factor in determining whether or not to proceed with the adoption of the Fountas &

Pinnell model. Qualitative data was gathered from open-ended and closed-ended questions in

both surveys and in-person interviews.


Instruments

A comprehensive survey was developed by the evaluator for approval by the client.

Because only one survey was administered for the purpose of this project, a select group of three

teachers was asked to review the survey for clarity. All certified staff members responsible for

reading instruction were asked to complete the survey. Questions were both open and closed-

ended. Twenty-eight of the thirty-three teachers responded to the survey for a response rate of

85%.

Once the surveys were complete, in-person interviews were conducted with five teachers

who were willing to participate. One member from each grade level was interviewed using the

interview guide approach (Appendix B.) This qualitative data collection method allowed for

topics and issues to be specified in advance while allowing for variance in the sequence and

wording of the questions as the interview progressed. This method kept the interview

conversational which perhaps elicited more meaningful data (Johnson & Christensen, 2007).

Finally, informal student assessment data was used to determine if students appeared to

be progressing in the area of reading achievement. Because the formal assessment was not

administered before the end of this project, the evaluator examined student work and relied on

teacher observation to determine if students were showing signs of achievement.

Objective Survey Interview Informal Student Assessment

What impact do professional learning sessions have on teachers as they implement the Fountas & Pinnell Benchmark Assessment System?

x x

How does the Fountas & Pinnell Benchmark

x x


Assessment System impact reading instruction?How does the Fountas & Pinnell Benchmark Assessment System compare to the assessment program used last year?

x x x

Should the Fountas & Pinnell Benchmark Assessment System be adopted for use at Ivy Creek Elementary School next year?

x x x

Summary of Key Findings

What impact do professional learning sessions have on teachers as they implement the Fountas & Pinnell Benchmark Assessment System?

Pre-Plan

ning

Grade L

evel P

lanning

One-On-O

ne Trai

ningDVDs

Adequate

Train

ing012345 4.32 3.89 3.93

3.144.39


Survey Question

Liker

t Sca

le

The results of the survey indicated that teachers were pleased overall with the training

they received and that the initial professional development session during pre-planning was the

most helpful. Several teachers indicated that they especially appreciated watching the video of

an expert conducting an actual assessment and comparing the expert results with their own. The

training sessions also offered detailed instructions on completing running records. While this

was helpful to teachers new to the process, at least three teachers commented that this session


should have been optional for those who were already proficient at completing running records.

Another classroom teacher suggested that it might have been helpful to hear a “short and sweet”

version of the training including a sample assessment and details about how this program was

different from the programs used in the past. Some of the information presented was redundant.

All of the teachers interviewed expressed a desire to spend more time with the

supplemental materials in the kit. They commented that many materials available to them have

been left unused. They would like time to explore these resources and to dig deeper into what

the Fountas & Pinnell Assessment System has to offer. At the time of the survey, many had not

watched the DVDs. However, at the time of the individual interviews, they had seen them and

found them to be helpful.

Comments in both the surveys and interviews indicated that additional training should be

centered around using the coding system within running records (MSV) and on how to proceed

with grouping students once you have determined student levels and how they progress into

higher groups.

Beginning of the Year

Summer Throughout the Year

0

1

2

3

4

54.11

2.323.07

Timing for Professional Development

Survey Question

Liker

t Sca

le


By far, the majority of teachers prefer to receive this type of training at the beginning of

the school year. Several comments in the interviews revealed concern over trying to attend to

such important training during pre-planning. Because of the furlough days, there was less time

to prepare for the first day of school and that was a distraction for many trying to learn how to

implement the Fountas & Pinnell Benchmark Assessment Program. Some teachers indicated

they like to receive training throughout the school year. Additionally, some teachers expressed a

preference for half-day planning while others asked for training in smaller chunks of time

throughout the year. These questions received the widest range of responses, but overall the

survey showed that teachers disagreed with attending training during the summer.

How does the Fountas & Pinnell Benchmark Assessment System impact reading instruction?

User Fr

iendly

Accurat

e Rea

ding Leve

l

Guides Instr

uction

Understan

d Erro

rs

Use Stu

dent E

rrors

Word Lists

Record

ing Form

s

Copies in

Workr

oom012345 4.11 4.07 4.39 3.96 4.21

3.32 3.714.61


Survey Question

Liker

t Sca

le

According to the survey, teachers found that the Fountas & Pinnell Benchmark

Assessment Program was implemented smoothly. In each of the subsequent interviews, teachers

shared that the results of their assessments were extremely accurate. They were pleased with the

program in that it has enabled them to form solid reading groups and paint a truer picture of each

student’s reading ability. Teachers found that the levels of text were appropriate with


challenging vocabulary. Those interviewed indicated that all of their students had improved one

or two levels since the initial testing at the beginning of the school year.

The survey results also indicated that teachers are using the results of the Fountas &

Pinnell Benchmark Assessment Program to guide instruction and that it helps them understand

student errors and incorporate those errors into their guided reading instruction. However, in the

interviews, the teachers expressed a desire to use the assessment information in this way, but

they had not effectively implemented this concept. Notably, they described that the assessment

period was originally scheduled to last for two weeks, but it actually took four weeks to

complete. One teacher commented, “I am concerned with the amount of instructional time it

took to conduct 27 assessments. I started all of them too low initially, and will need to make

adjustments for next year.” Another expressed concern over grading. “Mid-term grades are due

and there are few grades due to the amount of time it takes to implement the reading workshop

program and to test the students’ independent reading levels.” Teachers explained that they

believed this process would be faster next year because all of the students who were tested this

year would begin with a consistent starting point.

One of the reasons the assessments took so long was a result of the inaccuracies with the

Word Lists. This was a significant source of frustration for many teachers. Comments included,

“The Word Lists were off by at least two levels for almost every student,” and “The Word Lists

do not adequately equate with the student comprehension level. Many times students can word

call from a list but when they read that particular level book, they struggle.” Because the Word

Lists were used as a tool for determining on what level to begin assessing a student, the process

took longer than intended to complete each individual assessment.


The teachers liked the recording forms as a communication tool to use during parent-

teacher conferences. It took some getting used to the layout of the forms, but they found that the

fluency rubric was easy to code and was more objective than previous programs. They

especially found that having copies of the recording forms and assessments available in the

teacher workrooms was helpful for logistical reasons. The only complaint about these materials

was that the font size was too small.

Another concern shared by second, third and fourth grade teachers was that they really

needed access to both the upper and lower grade kits. Some students were reading significantly

above or below grade level and they found themselves asking other teachers to borrow materials

in order to properly assess their students. Those teachers needed their materials, so it was

difficult to coordinate sharing them so that all assessments could be completed in a timely

fashion.

How does the Fountas & Pinnell Benchmark Assessment System compare to the assessment program used last year?

More Infor-mation

More Accurate Prefer F&P0

1

2

3

4

53.89 3.86 3.93

Comparison with Other Programs

Survey Question

Liker

t Sca

le


Assessment Program as much as or more than programs they have used in the past. They cited


the accuracy of the results and the amount of information gathered for each student as a benefit.

They were also pleased with the fact that the books are shorter with more complex text. One

teacher indicated that this program was better because it did not just test the student’s stamina for

reading. Another shared that the variety of text was a nice change of pace, but she wished that

there was more than one book offered at each level so that students were exposed to different

texts during subsequent testing. Another teacher liked that the reading levels in this program

stretched well above grade level so that her students were assessed more accurately.

Several teachers expressed that the Fountas & Pinnell Benchmark Assessment Program

was more time-consuming to administer than other programs they had used. One shared that she

didn’t think it was any better or worse. “It's the same type of assessment with a different

company's name on it.”

Should the Fountas & Pinnell Benchmark Assessment System be adopted for use at Ivy Creek Elementary School next year?

Adopt at ICES Adopt in Gwinnett0

1

2

3

4

54.11 4.14

Recommendation for Adoption

Survey Question

Liker

t Sca

le

This question seemed to prompt some confusion amongst survey participants and yet they

obviously like this program enough to agree with adopting the program at Ivy Creek and within


Gwinnett County. However, some believed that the program had been adopted in 2003 when

reading workshop was implemented during Ivy Creek’s first year. Others were concerned that

we had spent money on a program that might not be used in the future. “I would like for us to

find ONE program and use it consistently. It seems like we are continuously changing mid

stream which makes it difficult to use the information in a meaningful way. The constant change

also has an effect on teacher moral because we feel like we are always learning how to

implement a "new and better" method.”

Recommendations


Teachers found that receiving initial training at the beginning of the school year during

pre-planning was most beneficial as the assessments needed to be completed as soon as the

school year began. However, those leading professional development should take into

consideration the meeting schedule for the week as well as the number of days available to

prepare for the beginning of school. In the event of a shortened pre-planning week, it is possible

that teachers could receive training during the first week of school.

Professional development instructors should also keep in mind that teachers will have

varying degrees of experience with conducting the assessments and running records. Training

should be adjusted to account for these differences by allowing teachers to attend a beginning,

intermediate or advanced session based on their needs. These sessions should include any

differences between the assessments that teachers are currently using and the Fountas & Pinnell

Benchmark Assessment Program. When possible, use an actual assessment as a model for

teachers attending training.


Future professional learning sessions should be focused on the contents of the assessment

kits and the use of the supplemental materials provided by the Fountas & Pinnell Benchmark

Assessment Program. Teachers expressed an interest in find out more information about

effectively using the coding system within running records (MSV), grouping students and using

errors for instruction. Preferences as to the length and timing of these sessions varies widely. It

would be appropriate to offer different options in an effort to accommodate everyone.


Survey results indicated that teachers are effectively implementing the Fountas & Pinnell

Benchmark Assessment Program. However, personal interviews with teachers from each grade

level revealed that while they have a desire to utilize student errors in their lesson planning and

reading instruction, the reality is that this is not happening. I believe that the discrepancy is a

result of teachers answering the questions with what they feel they are supposed to be doing as

opposed to what they are actually doing. To overcome this discrepancy, additional professional

learning on how to practically use the error information in reading instruction would be valuable

to teachers.

One of the biggest challenges with implementing the Fountas & Pinnell Benchmark

Assessment Program was the time it took for teachers to conduct the assessments. While

teachers believe that this process will go more quickly next year, it is important to address this

issue. Perhaps classes could be covered by support staff or substitute teachers for a period of

time so that teachers’ schedules were free to conduct assessments during the first two weeks of

school. Professional development could also be offered to share practical strategies for

conducting assessments throughout the school day.


The issue of mid-term grading is a concern that should also be addressed by grade levels.

Because of the length of time needed to complete the assessments, some teachers found it

difficult to have enough grades to accurately report reading progress. A common strategy for

reporting student progress needs to be developed. During the interviews, teachers did indicate

that all of their students had improved at least one or two levels since the beginning of the school

year. However, it is unclear as to the specific reason for the improvement. It could have been

because of the strategies implemented as a result of the Fountas & Pinnell Benchmark

Assessment Program, but it is more likely the efforts of the teachers conducting quality reading

instruction or a combination of both. On teacher remarked that this assessment program was just

another “tool in the toolbox” to reach students and teach them how to read.

Resource materials that accompany the program need to be adjusted as well. While the

Word Lists were a critical component to finding a starting point for assessments this year,

teachers should be able to rely on end of year assessment levels as a starting point for next year.

This would make the obstacles encountered with the Word Lists obsolete next year. However,

the recording forms and assessments need to be adjusted so that they are easier to read. It was

suggested that these documents be placed on the Share Drive for all teachers to easily access in

addition to providing copies in the teacher workrooms.

Finally, Ivy Creek should invest in three more kits to share with intermediate grade

levels. Second grade teachers need a kit for Grades 3-5 while third and fourth grade teachers

need kits for Grades K-2. This will eliminate the need for sharing kits amongst grade levels

during the limited time available for assessments to be completed.


Comparison to Other Programs


Assessment Program as much as or more than programs they have used in the past. While they

liked hearing a variety of texts read by each student, they would like to have several options

available so that students are looking at a new text for each of the upcoming assessments. This

may be beyond the control of the local school or county officials, but it is a suggestion worth

offering to the company producing the Fountas & Pinnell Benchmark Assessment Program

materials.

Adoption for Ivy Creek and Gwinnett

Overall, teachers recommended that the Fountas & Pinnell Benchmark Assessment

Program be adopted by Ivy Creek Elementary and by Gwinnett County. There was some

confusion about this portion of the survey that will need to be addressed with teachers during the

presentation of the findings of this evaluation report. I believe that the confusion was due to the

teachers’ familiarity with the work of Fountas & Pinnell. The leveled book room at Ivy Creek

has been organized using the Fountas & Pinnell leveling system, but teachers used a variety of

assessment programs (i.e. DRA, Rigby) to determine student reading levels. A correlation chart

that matched student reading levels in other programs to the Fountas & Pinnell levels was

provided so that teachers could easily locate materials in the leveled book room. However, there

was previously no consistency in the leveling systems used to assess student reading levels

across grade levels and Ivy Creek had not utilized the Fountas & Pinnell Benchmark Assessment

Program in the past.


Conclusion

The key findings of this evaluation report on the Fountas & Pinnell Benchmark

Assessment Program were overwhelmingly positive. Teachers at Ivy Creek Elementary School

indicated that they were pleased with the professional development sessions, the quality of the

materials, the impact the program had on reading instruction and student achievement.

Generally, they felt that this assessment program was as good as or better than programs they

had previously used and they recommended that the Fountas & Pinnell Benchmark Assessment

Program be adopted for continued use at Ivy Creek Elementary School and within Gwinnett

County. While the initial indications are very positive, by following the recommendations

mentioned above, the school has an opportunity to make the Fountas & Pinnell Benchmark

Assessment Program even more beneficial to teachers.

Based on the findings in this report, I believe that the Fountas & Pinnell Benchmark

Assessment Program has the potential for improving reading instruction and student achievement

at Ivy Creek Elementary. Professional development will be more focused on the specific needs

of teachers now that the general instruction is complete. As teachers grow more comfortable

with utilizing the program, I believe they will become more efficient in conducting their

assessments and they will begin to apply what they learn about student errors to their reading

instruction. Ultimately, both the students and teachers will benefit from the implementation of

this program.

As we conduct future evaluations of programs at Ivy Creek Elementary, it is my hope that

teachers will feel more and more comfortable sharing honest opinions with evaluators so that we

can fine tune the instructional practices that work best at the school.


References

Al Otaiba, S., Hosp, J., Smartt, S., & Dole, J. (2008). The challenging role of a reading coach, a

cautionary tale. Journal of Educational & Psychological Consultation, 18(2), 124-155.

Retrieved from ERIC database.

Engstrom, M., & Danielson, L. (2006). Teachers' perceptions of an on-site staff development

model. Clearing House: A Journal of Educational Strategies, Issues and Ideas, 79(4),

170-173. Retrieved from ERIC database.

Fountas, I. & Pinnell, G. (2008). Assessment guide: A guide to the benchmark assessment

system. Portsmouth: Heinemann.

Hutchins, K., Arbaugh, F., Abell, S., Marra, R., & Lee, M. (2008). A consumer guide to

professional development. Science Scope, 31(8), 16-19. Retrieved from ERIC database.

Johnson, B., & Christensen, L. (2007). Educational research: Quantitative, qualitative, and

mixed approaches (3rd ed.). Boston: Allyn and Bacon.

Kaufman, R., Guerra, I., Platt, W. (2006). Practical evaluation for educators: finding what

works and what doesn’t. Thousand Oaks: Corwin Press.

Obara, S., & Sloan, M. (2010). Classroom experiences with new curriculum materials during the

implementation of performance standards in mathematics: A case study of teachers

coping with change. International Journal of Science and Mathematics Education, 8(2),

349-372. Retrieved from ERIC database.

Pinnell, G. & Fountas, I. (2007). The continuum of literacy learning: A guide to teaching.

Portsmouth: Heinemann.

Purdue University Online Writing Lab (OWL) (May 5, 2010). APA formatting and style guide.

Retrieved from http://owl.english.purdue.edu/owl/resource/560/01/.

http://owl.english.purdue.edu/owl/resource/560/01/


Appendix A

Survey – Fountas & Pinnell Benchmark Assessment Program

The following survey is designed to evaluate the Fountas & Pinnell Benchmark Assessment Program (Fountas & Pinnell Program.) It should take approximately 5-10 minutes to complete. Your answers will provide valuable information in determining the effectiveness of the program as it relates to reading instruction and professional development. No names will be used at any time when reporting data for this evaluation, so please answer honestly!


1. How old are you?a. 20-29b. 30-39c. 40-49d. 50-59e. 60+

2. How long have you been teaching?a. 0-5 yearsb. 6-10 yearsc. 11-15 yearsd. 16-20 yearse. 21 or more years

3. What degree level do you currently hold?a. Bachelorsb. Mastersc. Specialistd. Doctoratee. Other, please specify

4. Have you used the Fountas & Pinnell Benchmark Assessment Program this year?a. Yesb. No

5. Have you previously used the Fountas & Pinnell Benchmark Assessment Program?a. Yesb. No



Please indicate how you strongly you agree or disagree with each of the following statements:

1 2 3 4 5Strongly Disagree

Strongly Agree

6. The one-hour pre-planning training session was beneficial to me.

1 2 3 4 5 N/A

7. The two 40-minute professional learning sessions during the first two weeks of school were

beneficial to me.

1 2 3 4 5 N/A

8. The one-on-one coaching provided by the reading coach/intervention coach was beneficial to

me.

1 2 3 4 5 N/A

9. The training DVDs were beneficial to me.

1 2 3 4 5 N/A

10. I received adequate training on implementing the Fountas & Pinnell Program.

1 2 3 4 5

11. I prefer receiving training at the beginning of the school year.

1 2 3 4 5

12. I prefer receiving training during the summer.

1 2 3 4 5

13. I prefer receiving training throughout the school year.

1 2 3 4 5


14. Please add any additional comments about the nature of the Training and Professional

Development as they relate to the Fountas & Pinnell Program:


Please indicate how you strongly you agree or disagree with each of the following

statements:


Strongly Agree

15. I find the Fountas & Pinnell Program to be user-friendly.

1 2 3 4 5

16. I believe that the Fountas & Pinnell Program accurately measures my students’ reading

level.

1 2 3 4 5

17. I use the Fountas & Pinnell Program to guide my reading instruction.

1 2 3 4 5

18. I have a greater understanding of student errors as a result of working with the Fountas &

Pinnell Program.

1 2 3 4 5

19. I use student errors as identified by the Fountas & Pinnell Program to guide my reading

instruction.

1 2 3 4 5


20. The word lists assist me in my reading instruction.

1 2 3 4 5

21. The recording forms assist me in my reading instruction.

1 2 3 4 5

22. Having copies of the support documents in the teacher workroom is beneficial to me.

1 2 3 4 5

23. Please add any additional comments about the implementation and reading instruction as

they relate to the Fountas & Pinnell Program:

Fountas & Pinnell Program vs. Other Reading Assessment Programs

Please indicate how you strongly you agree or disagree with each of the following

statements:


Strongly Agree

24. The Fountas & Pinnell Program provides me with more information about my students

reading abilities than previous reading assessment programs.

1 2 3 4 5

25. The Fountas & Pinnell Program is more accurate than previous reading assessment

programs.

1 2 3 4 5

26. I prefer using the Fountas & Pinnell Program over previous reading assessment programs.

1 2 3 4 5


27. Please add any additional comments about how the Fountas & Pinnell Program compares to

other reading assessment programs you have used:

Recommendation for Adoption

28. I would like to see the Fountas & Pinnell Program adopted for use at Ivy Creek Elementary

School.

1 2 3 4 5

29. I would like to see the Fountas & Pinnell Program adopted for use in Gwinnett County.

1 2 3 4 5

30. Please add any additional comments about the Fountas & Pinnell Program. Is there

anything you would like to see done differently next year?

If you would be willing to participate in a short interview session to further share your opinions

about the Fountas & Pinnell Program, please include your name: _______________________.

Again, all identifying characteristics of individual responses will remain confidential.

Thank you for taking time to complete this survey.


Appendix B

Interview Guide – Fountas and Pinnell Benchmark Assessment Program

Tell me about your experience with using the Fountas and Pinnell Benchmark

Assessment Program.

What are the strengths of the Fountas and Pinnell Benchmark Assessment Program?

What are the weaknesses of the Fountas and Pinnell Benchmark Assessment Program?

What part of the training benefitted you the most?

Is there anything you feel should be done differently next year?

In what ways has the Fountas and Pinnell Benchmark Assessment Program impacted

your reading instruction?

In what ways has the Fountas and Pinnell Benchmark Assessment Program impacted

student achievement?

What kind of growth have you seen in your students’ reading levels?

How would you compare Fountas and Pinnell Benchmark Assessment Program to

programs you have used in the past?

What is the most valuable part of the Fountas and Pinnell Benchmark Assessment

Program?

What is the least valuable part of the Fountas and Pinnell Benchmark Assessment

Program?

Is there anything else you’d like to share?


Appendix C

Language Impact Questionnaire

1. Which course of action would the evaluation report best support for 2011-12?

a. Continue the Fountas & Pinnell Benchmark Assessment Program in the same

way as 2010-11

b. Modify the Fountas & Pinnell Benchmark Assessment Program

c. Discontinue the Fountas & Pinnell Benchmark Assessment Program

2. Based on the evaluation report, how would you describe the effectiveness of the Fountas

& Pinnell Benchmark Assessment Program?

a. Completely successful – meets all program goals

b. Partially successful – meets some goals, achievement of remaining goals is yet to

be determined

c. Partially unsuccessful – meets some goals, remaining goals were not achieved and

will not be achieved with the current approach

d. Unsuccessful – none of the program goals have been achieved

3. In comparison to other reading programs, how would you rate the Fountas & Pinnell

Benchmark Assessment Program based on the evaluation report?

a. Clearly superior

b. Somewhat better

c. About the same

d. Less effective

e. Definitely worse


4. According to the evaluation report, the main goals of the Fountas & Pinnell Benchmark

Assessment Program are…. (list as many as apply):

5. According to the evaluation report, the Fountas & Pinnell Benchmark Assessment

Program’s important strengths or accomplishments are…. (list as many as apply):


Program’s important limitations or deficiencies are…. (list as many as apply):


Program is perceived positively overall by what percentage of the users?

a. 90% or higher

b. 80%-89%

c. 70%-79%

d. 60%-69%

e. 50%-59%

f. 30%-49%

g. 0%-29%


8. According to the evaluation report, which statement best describes the likelihood that the

Fountas & Pinnell Benchmark Assessment Program will positively impact Ivy Creek

Elementary School’s performance?

a. Virtually certain – positive impact is already evident

b. Probably – trends so far are positive

c. Possible – program design is sound, but no evidence is available at this time

d. Unlikely – trends so far are negative and the basic program design is flawed

e. Program will not positively impact performance. No impact or negative impact is

already evident.

Questionnaire based on Table 7.1, p. 211-12 (Kaufman, Guerra & Platt, 2006)


Data Collection Organization Matrix

Questions Related Results

Level (Mega, Micro, Macro)

Required Data,

Measureable Indicators

Data Source

Data Collection Tools

& Procedur

es

Data Collected By (Date)

Primary Responsibili

ty

What impact do

professional learning sessions have on

teachers as they

implement the Fountas & Pinnell

Benchmark Assessment

System?

Teachers will be

confident and

successful reading

teachers.

MACRO

Teachers will

average a score of 4 or higher on the 5

point Likert scale

on professiona

l developme

nt questions.

Teachers will

express confidence

in implementi

ng the program.

Teachers will

indicate needs for additional

professional learning

with regards to

implementing the

program.

Teachers

Survey Results

Interviews

10/18/10

Kimberly Sharp

How does The MACR Teachers Teacher Survey 10/18/1 Kimberly


the Fountas & Pinnell


System impact reading

instruction?

program will

positively impact reading

instruction.

O will average a score of 4 or higher on the 5

point Likert scale

on implement-ation and reading

instruction questions.

Teachers will

indicate that they analyze errors

made by students in

order to develop

appropriate reading

instruction methods.

Teachers will

indicate that

students show

growth in their

reading achievement and that

reading levels are positively impacted by the use

of the

s Results

Interviews

Informal: Student Reading Levels

0 Sharp


program.How does

the Fountas & Pinnell


System compare to

the assessment

program used last

year?

Teachers will prefer

this program to previous

programs.

MACRO

Teachers will


point Likert scale

on the Fountas & Pinnell vs.

other reading

programs questions.

Teachers will

express that the

new program is superior to previous

programs.

Teachers

Survey Results

Interviews

10/18/10

Kimberly Sharp

Should the Fountas &

Pinnell Benchmark Assessment System be

adopted for use at Ivy

Creek Elementary School next

year?

Teachers will

recommend that this

program be adopted at Ivy Creek

and in Gwinnett County.

MACRO

Teachers will


point Likert scale

on the recommend-ation for adoption

questions...

Teachers

Survey Results

Interviews

Informal: Student Reading Levels

10/18/10

Kimberly Sharp

Matrix taken from Figure 4.3, p. 105 (Kaufman, Guerra, & Platt, 2006).

Documents

stu.westga.edustu.westga.edu/~ksharp1/8480_evalreport_kas.docx · Web viewOne of the reasons the assessments took so long was a result of the inaccuracies with the Word ... detailed