Between the Lions Preschool Literacy Project Evaluation Results 2005-2006

Between the Lions Between the Lions Preschool Literacy ProjectPreschool Literacy Project

Evaluation Results Evaluation Results

2005-20062005-2006

Project GoalProject Goal

To improve language development and To improve language development and other pre-literacy skills among low-income other pre-literacy skills among low-income children in child care centers children in child care centers

Project InterventionProject Intervention

Leaders in Literacy Leaders in Literacy training training

On-site technical assistance On-site technical assistance

Between the Lions Between the Lions materials materials

Evaluation QuestionsEvaluation Questions

How do the classroom environments of teachers How do the classroom environments of teachers who participate in the who participate in the BTLBTL intervention compare intervention compare with the classroom environments of teachers with the classroom environments of teachers who do not participate in the who do not participate in the BTLBTL intervention? intervention?

How do the literacy outcomes for children whose How do the literacy outcomes for children whose teachers participate in the teachers participate in the BTLBTL intervention intervention compare with the literacy outcomes of children compare with the literacy outcomes of children whose teachers do not participate in the whose teachers do not participate in the BTL BTL intervention? intervention?

Participants in 2005-2006Participants in 2005-2006

Fall 2005 InterventionFall 2005 Intervention– 4 Child Care Centers4 Child Care Centers

11 Classrooms11 Classrooms

Fall 2005 ControlFall 2005 Control– 2 Child Care Centers2 Child Care Centers

5 Classrooms5 Classrooms

Spring 2006 Spring 2006 Intervention Intervention – 7 Child Care Centers 7 Child Care Centers

(3 from Fall Intervention, (3 from Fall Intervention, 1 from Fall Control)1 from Fall Control)

9 Classrooms (All 9 Classrooms (All different from Fall)different from Fall)

Spring 2006 ControlSpring 2006 Control– 3 Child Care Centers3 Child Care Centers

5 Classrooms (All 5 Classrooms (All different from Fall)different from Fall)

InstrumentsInstruments

Classroom Environment AssessmentClassroom Environment Assessment– Early Language & Literacy Classroom

Observation (ELLCO) Toolkit

Early Literacy Assessment– Get Ready to Read Screening Tool –

Recommended for 4-year-olds– Test of Early Reading Ability-Third Edition

(TERA-3) – Designed exclusively for ages 3 ½ to 6

ELLCO ToolkitELLCO Toolkit

Literacy Environment ChecklistLiteracy Environment Checklist: Used in : Used in assessing the organization and contents of the assessing the organization and contents of the classroom.classroom.Classroom Observation & Teacher InterviewClassroom Observation & Teacher Interview: : Used to gather objective ratings of the quality of Used to gather objective ratings of the quality of the language and literacy environment and the language and literacy environment and experiences in a given classroom.experiences in a given classroom.Literacy Activities Rating ScaleLiteracy Activities Rating Scale: Provides : Provides summary information on the nature and duration summary information on the nature and duration of literacy-related activities observed.of literacy-related activities observed.

Get Ready to Read Screening ToolGet Ready to Read Screening Tool

Print knowledge – Assesses a child’s Print knowledge – Assesses a child’s understanding of books, printed letters, understanding of books, printed letters, and words.and words.

Emergent writing – Assesses a child’s first Emergent writing – Assesses a child’s first efforts to use print in a meaningful way.efforts to use print in a meaningful way.

Linguistic awareness – Assesses a child’s Linguistic awareness – Assesses a child’s understanding of how language works. understanding of how language works.

Test of Early Reading Ability,Test of Early Reading Ability,3rd Edition (TERA-3)3rd Edition (TERA-3)

Subscales – Normed average score of 10Subscales – Normed average score of 10– AlphabetAlphabet: Measures children’s knowledge of the : Measures children’s knowledge of the

alphabet and sound-letter correspondence. alphabet and sound-letter correspondence. – ConventionsConventions: Measures children’s familiarity with the : Measures children’s familiarity with the

conventions of print. conventions of print. – MeaningMeaning: Measures children’s ability to comprehend : Measures children’s ability to comprehend

the meaning of printed material.the meaning of printed material.

Reading Quotient – Normed average score of Reading Quotient – Normed average score of 100 – Derived from the sum of the three 100 – Derived from the sum of the three subscale scores subscale scores

Assessment TimelineAssessment Timeline

Fall 2005 Intervention Fall 2005 Intervention and Controland Control– Pretests - August 2005Pretests - August 2005

ELLCO, Get Ready to ELLCO, Get Ready to Read, TERA-3Read, TERA-3

– Posttests – Jan. 2006Posttests – Jan. 2006ELLCO, Get Ready to ELLCO, Get Ready to Read, TERA-3Read, TERA-3

– Delayed Posttests – Delayed Posttests – June 2006June 2006

ELLCO, Get Ready to ELLCO, Get Ready to Read, TERA-3Read, TERA-3

Spring 2006 Spring 2006 Intervention and Intervention and ControlControl– Pretests – January Pretests – January

20062006ELLCO, Get Ready to ELLCO, Get Ready to Read, TERA-3Read, TERA-3

– Posttests – June 2006Posttests – June 2006ELLCO, Get Ready to ELLCO, Get Ready to Read, TERA-3Read, TERA-3

Number and Percent of TERA-3 Participants

Below and Within the Acceptable Age Range

Fall 2005 Testing Spring 2006 Testing Intervention Control Intervention Control Number Percent Number Percent Number Percent Number Percent Below range 20 18.0% 6 15.0% 37 33.3% 15 33.3% Within range 91 82.0% 34 85.0% 74 66.7% 30 66.7%

Explanation of Statistical Explanation of Statistical Terminology UsedTerminology Used

N - Number of participants N - Number of participants

Mean - AverageMean - Average

p p - Level of significance - Level of significance (p < .(p < .05 indicates 05 indicates less than 5% probability that the effect was less than 5% probability that the effect was a chance occurrence; 95% probability that a chance occurrence; 95% probability that the change resulted from the intervention)the change resulted from the intervention)– Significance is determined by statistical tests Significance is determined by statistical tests

(e.g., t-tests)(e.g., t-tests)

Fall 2005 ClassroomFall 2005 ClassroomInitial Assessment ResultsInitial Assessment Results

Scale Intervention Group Change (Pretest to Posttest)

Control Group Change (Pretest to Posttest)

ELLCO Literacy Environment Checklist

Gain (significant)

Gain (not significant)

ELLCO Classroom Observation Scale

Gain (significant) Loss (not sig nificant)

ELLCO Literacy Activities Rating Scale

Gain (significant) Gain (not significant)

Fall 2005 Preschoolers’Fall 2005 Preschoolers’Initial Assessment ResultsInitial Assessment Results



TERA-3 Reading Quotient


Gain (significant)

TERA-3 Alphabet Subscale


TERA-3 Conventions Subscale

Gain (significant) Gain (significant)

TERA-3 Meaning Subscale


Gain (significant)

Get Ready to Read Total Score


Fall 2005 ClassroomFall 2005 ClassroomDelayed Posttest ResultsDelayed Posttest Results




Gain (significant)



Gain (significant) Loss (not sig nificant)



Fall 2005 Preschoolers’Fall 2005 Preschoolers’Delayed Posttest Results (June 2006)Delayed Posttest Results (June 2006)




Gain (significant)



Gain (significant) Loss (not significant)




Gain (significant)

Gain (s not isignifucant )



Spring 2006 ClassroomSpring 2006 Classroom Assessment Results Assessment Results




Gain (significant)



Gain (not significant) Loss (not significant)



Spring 2006 Preschoolers’Spring 2006 Preschoolers’ Assessment Results Assessment Results




Loss (not significant)

Loss (significant)




Loss (not significant) Loss (significant)





Gain (significant) Loss (not significant)

Limitations of 2005-2006 StudyLimitations of 2005-2006 Study

Brief intervention periods for both Fall and Brief intervention periods for both Fall and Spring (4 to 5 months) Spring (4 to 5 months)

Use of assessment instruments that were Use of assessment instruments that were not appropriate for younger aged childrennot appropriate for younger aged children

Notably smaller control group than Notably smaller control group than intervention group intervention group

High turnover of teachers and students High turnover of teachers and students

Recommendations for Recommendations for 2006-2007 Study2006-2007 Study

Longer intervention period Longer intervention period

Documentation of intervention to ensure Documentation of intervention to ensure fidelity to intended treatment fidelity to intended treatment

Consideration of revising the process of Consideration of revising the process of selecting intervention and control groups selecting intervention and control groups to match more closely in number to match more closely in number

Documents

Between the Lions Preschool Literacy Project Evaluation Results 2005-2006