5

Click here to load reader

Balanced Literacy Proposal 1 - mvsc.k12.in.us · Balanced Literacy Proposal 1 Introduction & Background Information A balanced literacy school focuses on activities such as shared

  • Upload
    lynhi

  • View
    212

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Balanced Literacy Proposal 1 - mvsc.k12.in.us · Balanced Literacy Proposal 1 Introduction & Background Information A balanced literacy school focuses on activities such as shared

Balanced Literacy Proposal     1  

Introduction & Background Information

A balanced literacy school focuses on activities such as shared and guided reading and writing through the utilization of literacy, listening and writing centers within a print-rich environment. This research project proposes to evaluate the implementation of the Balanced Literacy Initiative, which began three years ago, and was spearheaded by the Lakeshore Alliance for Student Success (LASS) in Northwest Indiana. Currently 25 elementary schools are receiving the training and resources needed to implement the program in grades K-2. The Balanced Literacy program is being implemented with the assistance of literacy coaches who provide professional development training for the participating teachers. A balanced literacy school consists of certain characteristics that serve specific purposes, which relate to the development of literacy (e.g. reading, spelling and writing). Appendix A provides a descriptive summary of each balanced literacy characteristic and its specific purpose. Participants

This study will include the following schools in Northwest Indiana who are participating in the Balanced Literacy Initiative:

• Olive Township Elementary • Prairie View Elementary • Rolling Prairie Elementary • Fieler Elementary • Kenwood Elementary • Maywood Elementary • Joseph Hess Elementary • Thomas Edison Elementary • Thomas Jefferson Elementary • Franklin Elementary • Irving Elementary • Iddings Elementary • Jonas E. Salk Elementary • Edgar Miller Elementary • John Wood Elementary • Longfellow Elementary • Grissom Elementary • Hosford Park Elementary • Nathan Hale Elementary • Hebron Elementary • Lake Prarie Elementary • Morton Elementary • Harding Elementary • Oak Hills Elementary • Three Creeks Elementary

Page 2: Balanced Literacy Proposal 1 - mvsc.k12.in.us · Balanced Literacy Proposal 1 Introduction & Background Information A balanced literacy school focuses on activities such as shared

Balanced Literacy Proposal     2  

Research Design, Research Questions & Suggested Data Analysis

This evaluation study will incorporate multiple methods and designs in order to address the evaluation questions. The specific designs will include the mixed methods triangulation design, the quasi-experimental design (panel data with and without a comparison group) and a formal meta-analysis. The triangulation component of this evaluation study will be used to address the first two research questions and assist in the data validation process. The quasi-experimental component will be used to determine the degree to which (1) balanced literacy results in a statistically significant amount of growth on standardized achievement tests; (2) balanced literacy results in a narrowing of the achievement gaps and (3) balanced literacy results in an academic advantage as compared to comparison classrooms. Finally, the meta-analytic component will be used to identify specific school and/or classroom effects as well as implementation effects that moderate the effectiveness of the balanced literacy program on student achievement in reading, writing and spelling. The primary independent variable for this evaluation study will be participation in Balanced Literacy. Treatment fidelity will serve as an intervening factor. The primary dependent variable will include student standardized test performance. Table 1 below outlines each evaluation question, the proposed design and the suggested analysis for each question. Table 1. Evaluation Question and Evaluation Approach Linkage

Evaluation Question Design Suggested Analyses Evaluation question one: How well are the teachers implementing the balanced literacy approach?

Mixed methods triangulation

Descriptive and thematic analysis based on observations and surveys/interviews

Evaluation question two: To what extent is balanced literacy being implemented?

Mixed methods triangulation

Descriptive and thematic analysis based on observations and surveys/interviews

Evaluation question three: To what extent do the classrooms participating in balanced literacy show growth as measured by standardized achievement tests?

Quasi-experimental: panel design with and without a comparison group

Descriptive statistics Hierarchical linear modeling (value added)

Evaluation question four: What are the significant moderating factors with regard to the effectiveness of the implementation of balanced literacy?

Quantitative meta-analysis

Coding for moderators, computing effect sizes, testing for homogeneity of effect sizes and testing for significant moderators if effect sizes are heterogeneous

Page 3: Balanced Literacy Proposal 1 - mvsc.k12.in.us · Balanced Literacy Proposal 1 Introduction & Background Information A balanced literacy school focuses on activities such as shared

Balanced Literacy Proposal     3  

Data Collection

Student level achievement data will be collected every fall and spring of each academic year for a total of four years of data (baseline year plus three implementation years). The teacher observation data will be collected at various points throughout the study. A copy of the teacher observation checklist form is attached (see Appendix B). In addition, a survey/interview will be developed by the researcher that will ask teachers about their implementation of balanced literacy in the classroom. The variables that will be measured are listed in Table 2. The color key associated with Table 2 is as follows: Grey represents the identification variables, yellow represents the dependent variables, green represents the control variables (mediators and moderators) and brown represents the independent variable. The required column in Table 2 indicates whether or not the data is required for the evaluation study. Table 2. Research Variables

Variable Measure Required

Student identifier Random number Yes

Classroom identifier Random number Yes

School identifier Random number Yes

Reading Standardized achievement test (e.g. ISTEP) Yes

Writing Standardized achievement test (e.g. ISTEP) Yes

Spelling Standardized achievement test (e.g. ISTEP) Yes

Comparison subject Standardized achievement test (e.g. ISTEP) No

Student gender Student records Yes

Student race Student records Yes

Student disability status Student records Yes

Student SES Qualifies for free and reduced lunch Yes

Student attendance School records Yes

School: percent low income School records Yes School: percent limited English School records Yes

School: parental involvement School records No

Extent of implementation Observation & surveys Yes

Ability level (gifted) School records Yes

Retention status (yes/no) School records Yes

Page 4: Balanced Literacy Proposal 1 - mvsc.k12.in.us · Balanced Literacy Proposal 1 Introduction & Background Information A balanced literacy school focuses on activities such as shared

Balanced Literacy Proposal     4  

Kindergarten (1/2 day or full) School records Yes

Attended pre-school School records Yes

English as 2nd language School records Yes

Participated in PAT program School records Yes

Implementation fidelity Observation & surveys Yes

Type of assessment used School records Yes

Balanced Literacy Treatment group (experimental/control) Yes Procedure

School related data will be extracted directly from the Indiana Department of Education website (http://www.doe.in.gov/data/). Student and teacher level data will be obtained from the literacy coaches who will be collecting the data via teacher observation forms and student standardized achievement tests. However, not all of the schools are using the same student achievement assessment instruments. The teacher training will be implemented by Purdue University Calumet. A copy of the course syllabus is attached (see Appendix C). Data Quality Control Methods, Reliability and Validity

The data quality control methods that will be utilized in this evaluation study include the following:

1. Multiple raters will be used for establishing inter-rater reliability for all qualitative analyses resulting from the research survey/interview.

2. Formal reliability analyses will be conducted for all of the quantitative items on the research survey.

3. Multiple data sources will be used for triangulation purposes (e.g. student achievement data, teacher observations and teacher survey/interview).

4. Evaluator has designed the evaluation study such that potential threats to internal validity (history, selection bias, maturation and statistical regression to the mean) are minimized and threats to the statistical conclusion validity are minimized. Historical threats, maturational threats, selection bias and statistical regression to the mean are minimized due to the quasi-experimental component of the study. Threats to the statistical conclusion validity are minimized by carefully considering the implications of sample size when conducting quantitative analyses and by testing all of the statistical assumptions associated with each statistical analysis.

5. An on-line survey format will be used to minimize human error in data entry (e.g. SurveyMonkey or Zoomerang).

6. All quantitative data will be checked for accuracy. 7. Electronic back-up files of all data and relevant evaluation study

documents will be kept in a secure place.

Page 5: Balanced Literacy Proposal 1 - mvsc.k12.in.us · Balanced Literacy Proposal 1 Introduction & Background Information A balanced literacy school focuses on activities such as shared

Balanced Literacy Proposal     5  

Ethical Considerations The data will consist of archival school level data that is provided on the Indiana Department of Education website, teacher level implementation data and student level demographic and achievement data. All of the student level data used for this study consists of data that is already being collected and therefore no special data collection efforts will be made relative to student achievement. Teachers will be asked to complete a research survey/interview that will be completed on a volunteer basis and informed consent for completing the survey/interview will be required. Confidentiality will be maintained at all times by assigning an identifier to each teacher and student so that names are not maintained within the primary investigator’s database. This study poses absolutely no physical harm to participants. Furthermore, preliminary studies have shown positive academic affects for students who are exposed to the treatment associated with this study (Biancarosa, G., Byrk, A., & Dexter, E., 2008)1. Finally, this study contains no deception.

Significance of Study

Although there have been some recent and preliminary studies conducted on the value-added effects of balanced literacy through the use of literacy coaches, Biancarosa et al (2008) have indicated that there is still a “pressing need” to determine why the effects of the literacy coaches vary between teachers and schools. This study seeks to not only replicate the findings of Biancarosa, et al, but to replicate them based on a sample of students and schools in Northwest Indiana and to determine the reasons for the variability in the effectiveness of the implementation through the use of a meta-analytic design (refer to Table 1).

Source of Funding

This study is funded through the Lakeshore Alliance for Student Success (LASS).

                                                            1 Paper presented at the National Reading Conference on December 3, 2008 in Orlando, FL. Slides: Assessing the Value-Added Effects of Literacy Collaborative Professional Development on Student Learning