http://ecr.sagepub.com/Journal of Early Childhood Research

http://ecr.sagepub.com/content/9/2/109The online version of this article can be found at:

 DOI: 10.1177/1476718X10387900

2011 9: 109 originally published online 14 March 2011Journal of Early Childhood ResearchKathy Sylva, Edward Melhuish, Pam Sammons, Iram Siraj-Blatchford and Brenda Taggart

benefitPre-school quality and educational outcomes at age 11: Low quality has little

  

Published by:

http://www.sagepublications.com

can be found at:Journal of Early Childhood ResearchAdditional services and information for    

  http://ecr.sagepub.com/cgi/alertsEmail Alerts:

 

http://ecr.sagepub.com/subscriptionsSubscriptions:  

http://www.sagepub.com/journalsReprints.navReprints:  

http://www.sagepub.com/journalsPermissions.navPermissions:  

http://ecr.sagepub.com/content/9/2/109.refs.htmlCitations:  

What is This? 

- Mar 14, 2011 OnlineFirst Version of Record 

- Jun 23, 2011Version of Record >>

at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from at Oxford University Libraries on November 7, 2013ecr.sagepub.comDownloaded from

Article

Journal of Early Childhood Research9(2) 109–124

© The Author(s) 2011Reprints and permission: sagepub.

co.uk/journalsPermissions.navDOI: 10.1177/1476718X10387900

ecr.sagepub.com

Corresponding author:Kathy Sylva, Department of Education, University of Oxford, 15 Norham Gardens, Oxford OX2 6PY, UKEmail: kathy.sylva@education.ox.ac.uk

Pre-school quality and educational outcomes at age 11: Low quality has little benefit

Kathy SylvaUniversity of Oxford, UK

Edward MelhuishInstitute for the Study of Children, Families and Social Issues, Birkbeck, University of London, UK

Pam SammonsUniversity of Oxford, UK

Iram Siraj-BlatchfordInstitute of Education, University of London, UK

Brenda TaggartInstitute of Education, University of London, UK

AbstractThis article reports the effects of pre-school quality on children’s cognitive and behavioural outcomes at age 11 using a large-scale longitudinal study of 3000+ children in England (EPPE/EPPSE). The ECERS-R and a curricular extension to it (ECERS-E) were used to assess the quality of provision in 141 pre-school settings attended by the children. The quality measures were derived from observations throughout the day of interactions and resources related to Literacy, Numeracy and Science learning, as well as observational/interview data related to how each centre catered to diverse needs of children. Multi-level modelling was used to investigate the effects of pre-school quality on children’s academic and social-behavioural outcomes at age 11. Pre-school quality significantly predicted most outcomes, after taking account of key child and family factors. More importantly, children who attended low quality pre-schools had cognitive and behavioural scores that were not significantly different from those of children with no pre-school experience. The methods and findings of this large-scale study are considered in terms of the strengths and limitations of ‘educational effectiveness’ designs. It is suggested that mixed methods designs can address many of the limitations.

Keywordslongitudinal effects, pre-school, quality, strengths and limitations, study design

110 Journal of Early Childhood Research 9(2)

Introduction

The context of this article is the debate about the importance of quality in early education. It has long been argued by child care researchers that quality of care relates to developmental outcomes (Belsky, 2001; Peisner-Feinberg et al., 2000; Vandell and Wolfe, 2000). However, measuring qual-ity is never straightforward; it has both structural properties that are important (Phillips et al., 2001) and process properties that matter as well for children’s development (NICHD Early Child Care Research Network [ECCRN], 1996, 2000a, 2000b; Phillips and Howes, 1987).

Generally, where quality has been considered, it has emerged as a weak but significant predictor of children’s developmental progress, particularly with regard to cognitive outcomes (Clarke-Stewart et al., 1994; Kwan et al., 1998; McCartney, 1984; NICHD ECCRN, 2005). In a compre-hensive literature review, Vandell (2004) concluded that findings are more conclusive when child care quality and child outcomes are related concurrently (e.g. Loeb et al., 2004; Love et al., 2003), rather than longitudinally (see Committee on Family and Work Policies, 2003). In longitudinal studies, relations are less evident when findings rely on a single assessment of quality (e.g. Deater-Deckard et al., 1996) than when relying on multiple assessments of quality (Burchinal et al., 2000). Two notable large-scale longitudinal studies which have assessed quantity, quality and care type simultaneously have shown convincing evidence of the importance of child care quality for children’s cognitive development (The Cost Quality and Outcomes study: Peisner-Feinberg and Burchinal, 1997; the NICHD ECCRN, 2000a, 2002).

Research on associations between quality of pre-school care and children’s socio-emotional outcomes is less conclusive. However, both the Cost Quality and Outcomes Study and the NICHD have provided evidence for longitudinal associations. The former found that, at second grade, children were more socially competent if they had been in better quality care programmes when they were younger. The latter produced evidence that quality of care is related to socio-emotional outcomes across different time points (between age two and four and a half): higher quality care predicted fewer problem behaviours, higher social competence, positive skilled peer interaction, and lower levels of impulsiveness (NICHD ECCRN, 2005).

The effective provision of pre-school and primary education (EPPE) in England In England, the short- and long-term effects of pre-school quality on children’s development have not, until recently, been studied in a systematic way, despite the fact that access to publicly funded early childhood education is now government policy intended to redress inequality and raise attainment (Inter-departmental Childcare Review, 2002; Sylva and Pugh, 2005). The Effective Pre-school and Primary Education (EPPE) project is the first British large-scale longitudinal study on the effects of pre-school provision, with more than 3000 children followed longitudi-nally. Using an ‘educational effectiveness’ design, EPPE set out to explore the effects of indi-vidual pre-school centres on children’s attainment and their social/behavioural development at entry to primary school at age five and the continuing effects at ages seven and 11, end of Key Stages 1 and 2 (Sylva et al., 2004, 2008). Now called EPPSE (The Effective Pre-school, Primary Education and Secondary Education), the project has been extended (2007–2011) in order to follow the same cohort to the end of Key Stage 3 at age 14. By studying the combined effects of pre-school, primary school and family factors, the study has provided new insights and evidence for the importance of early childhood experiences. For a full discussion of the findings, see Sylva et al. (2008, 2010a).

Sylva et al. 111

In order to assess pre-school quality the EPPE researchers focused particularly on process elements through the use of two well established observation instruments: the Early Childhood Environment Rating Scale (ECERS-R; Harms et al., 1998), and the recently revised ECERS-Extension (ECERS-E; Sylva et al., 2010). The ECERS-R has been widely used in research assess-ing child care quality (Burchinal et al., 2002; De Kruif et al., 2000; Gilliam, 2000; Jaeger and Funk, 2001; Phillipsen et al., 1997; Sheridan et al., 2009; Whitebook et al., 1989), whilst the ECERS-E was devised by the EPPE team (for details, see Soucacou and Sylva, 2010). It has strong associa-tions with children’s outcomes at age seven (Sylva et al., 2006).

The ECERS-E rating scale was devised by the EPPE team to provide a robust and relevant measure of quality related to cognitive development in children. As there have been significant changes to policy and practice in England, for example the revision and expansion of the curricu-lum in 2000, the ECERS-E was designed to reflect these changes, after wide consultation with experts and practitioners. It covers three of the six areas of the English curriculum: Communication, Language and Literacy; Problem Solving, Reasoning and Numeracy; and Understanding the World. The remaining three areas of development (Creative Development; Physical Development; and Personal, Social and Emotional Development) were already adequately covered by the ECERS-R and so not included in the ECERS-E items. Both the ECERS-R and ECERS-E are based on a conceptual framework which takes account of pedagogical processes and curriculum, although the ECERS-E was more focused on practices that encouraged the development of ‘emergent’ cognitive skills.

Research aimsThe aim of this study was to explore the medium- to long-term impact of pre-school quality (as measured by the ECERS-E and ECERS-R) on children’s developmental outcomes. It posed the question, if quality matters, do different levels of quality have different impacts in the longer term? Assessments at age 11 were selected to include the national assessments tests common across the country, along with a rating scale completed by the classroom teacher on the child’s social, emotional and behavioural characteristics. Thus the study examined the effects of different levels of quality, as well as the effects of different measures of quality, on a range of educational and social outcomes at age 11. The overarching research question was: is pre-school quality measured on the ECERS-R and on the ECERS-E related to child outcomes at the end of primary school?

Method

Overview of the sample

This article focuses on children’s outcomes measured towards the end of primary school, particu-larly Key Stage 2 (age 11). Assessments of children’s cognitive skills, and their social/behavioural competences were collected at age three (pre-test) but also at age five (at entry to primary school) and at age seven. The analyses in this article focus on the child outcome information gathered at age 11 but also uses child assessment scores at age three and five.

SampleAs illustrated by Figure 1 the sample was drawn from a large-scale longitudinal study of 3000+ children which was broadly representative of England at the time of recruitment (1997–1998). Five

112 Journal of Early Childhood Research 9(2)

regions in England were sampled and these were strategically selected to represent urban, suburban and rural areas and also areas with social and ethnic diversity. A total of 2857 three-year-old children were recruited from 141 pre-school settings representing six different types of group provision. An additional sample of 315 ‘home’ children (who had not attended a pre-school setting) was identified at entry to primary school, for comparison with those who had attended pre-school. In this analysis, the sample included 2664 children who had valid data for both quality and outcome measures at age 11 (see Sammons et al., 2004, 2008a, 2008b; Sylva et al., 2008, 2010a). Also included in the analysis are the ‘home’ children (to whom children are compared) but their baseline measures are at age four-plus/five.

This longitudinal sample consisted of roughly equal numbers of girls and boys, with 74 percent of the sample reporting ethnicity as of White UK heritage, and 11 percent of the children had English as an additional language (EAL). With regard to family structure, 14 percent of the children lived in large families (defined as those with three or more siblings), and there was variation in the quality of the pre-school Home Learning Environment (HLE, measured by 1:1 interview) with 41 percent in the good or very good categories, while 9 percent had rather poor HLE. Family socioeconomic status (SES) varied, with over 30 percent of the families classified as being in the two highest (professional) occupational categories and 16 percent reported as unemployed. Furthermore, as a low income indicator, 16 percent of the children were found to be eligible for free school meals (FSM) (Sylva et al., 2008).

Measures1. Pre-school quality. Pre-school quality was measured using the ECERS-R (Harms et al., 1998); and the ECERS-E (Sylva et al., 2010b). The ECERS-R focuses on a broad range of quality dimensions

25 nursery classes

590 children

34 playgroups

610 children

31 private day nurseries

520 children

20 nursery schools

520 children

7 integrated centres

190 children

24 local authority day care nurseries

430 children

home

310 children

Reception Yr 1 Yr 2Pre-school Provision

(3+ yrs)

Key Stage 1600 Schools

Yr 5 Yr 6

Key Stage 21000 Schools

Figure 1. Design of EPPE 3–11

Sylva et al. 113

and consists of 43 items across seven subscales. Space and furnishing (e.g. indoor space, room arrangement for play, child-related display), Personal care routines (e.g. greeting/departing, meals/snacks), Language-reasoning (e.g. books/pictures, encouraging children to communicate), Activities (e.g. dramatic play, nature/science), Interaction (e.g. supervision of children, staff–child interactions, interactions among children), Program structure (e.g. free play, group time), and Parents and staff (e.g. provisions for parents–staff interaction). Detailed descriptions are provided for each item; item scores are 1 (inadequate) through to 7 (excellent). The ratings are based on a minimum of a two-hour observation in one classroom or with one group and a limited number of interview questions regarding aspects of the routine not visible during observation. It has been shown to possess good psychometric properties and good predictive validity as it significantly relates to children’s developmental outcomes (Gilliam, 2000; Jaeger and Funk, 2001; Peisner-Feinberg and Burchinal, 1997; Phillips et al., 1987).

The ECERS-Extension (ECERS-E; Sylva, 2010b) focuses on the pre-school ‘cognitive’ curricu-lum and consists of 18 items on four subscales: Literacy, Mathematics, Science and Environment, Diversity. It is based on the literature on ‘emergent’ skills in literacy, numeracy and science (Soucocou and Sylva, 2010).

2. Child assessments2.1. Cognitive assessments At age 11 children were assessed on National Assessment Tests in

English and Mathematics. These were administered in the same week according to standardized instructions and marked on a government marking scheme. The BAS (Elliot, 1983), was used as baseline at age three years.

2.2. Social and behavioural assessments Teacher rating scales were also used. The extended ver-sion of the Strengths and Difficulties Questionnaire (Goodman, 1997) was used by the classroom teacher to assess different features of children’s social/behavioural development. Confirmatory factor analysis yielded four underlying social behavioural dimensions measures for each child (self-regulation, positive social behaviour, hyperactivity and anti-social behaviour). The Adaptive Social Behaviour Inventory (ASBI: Hogan et al., 1992) served as baseline at age three.

3. The Home Learning Environment (HLE). A major element of the EPPE design is to identify the familial and educational processes that underlie change in the developmental trajectories of young children (Brooks-Gunn, 2003), because it is well established that children’s outcomes are a product of both home and pre-school. The level of educational support children received at home was established through an interview with parents on entry to the study. Because children entered the study when they began pre-school (or the ‘pre-school room’ of a child care centre), the pre-test assessment of the HLE was made when children were between three and four-plus years; note that the home group parents were interviewed when their children entered the reception class at age four-plus years. Parents were asked about learning and play activities they engaged in with their children at home. A summary index of the Home Learning Environment (HLE: Melhuish et al., 2008) was constructed, measuring the frequency of the following activities: reading, painting and drawing, library visits, and playing or teaching numbers/shapes, alphabet/letters, and songs/nursery rhymes.

Analytic strategyMulti-level modelling (Goldstein, 1995) was used to examine the influences of pre-school quality on children’s educational and social/behavioural outcomes at age 11. These analyses were used as they take account of the hierarchical structure of the data (i.e. pupils clustered within schools)

114 Journal of Early Childhood Research 9(2)

therefore producing more accurate estimates of the net effects of different predictors. The unique (net) contribution of quality to variations in children’s outcomes was established whilst other back-ground influences (related to the child as in the case of gender, or to the family as in the case of social-economic status) were statistically controlled.

Results

1. Cognitive outcomes

The effects of ECERS-R and ECERS-E scores on cognitive outcomes were tested independently in predictive models. These classified pre-school quality as ranging from ‘no quality’ (i.e. the ‘home’ group, approximately 10% of the sample) through low (15%), medium (52%) and high quality (23%).

Figure 2 shows that the experience of high (curricular) quality pre-school provision as measured by the ECERS-E shows a positive impact on educational attainment at age 11, although attending a medium quality centre also has a significant impact when compared to low quality. The patterns of influence are similar for English and Maths, although the effects are somewhat stronger for Maths. The difference between ‘home’ children and those who attended a low quality pre-school was non-significant, indicating that both ‘home’ children and the low quality pre-school group scored significantly lower at age 11 than children from medium or high quality pre-schools. Quality scores on the ECERS-R were not related to educational attainment at age 11.

Note that the scores of children in the ‘Home’ group (used as the comparators in the analyses in Figures 2–6) have been set to 0.

0.12

0.22

0.29

0.12

0.26

0.34

0.00

0.10

0.20

0.30

0.40

0.50

Low quality Medium quality High quality

EnglishMathematics

Eff

ect

size

Net effect of quality (ECERS-E) of pre-schoolon English and Mathematics at age 11

Reference group:Home children

Figure 2. The impact of pre-school quality (ECERS-E) on English and Mathematics in Year 6

Sylva et al. 115

2. Social behavioural outcomes

Both the ECERS-R and the ECERS-E had a statistically significant impact on all four social/behavioural outcomes, with the ECERS-R having a slightly stronger impact on ‘Pro-social’ and ‘Anti-social’ behaviour than the ECERS-E (see Figures 3–6).

Overall, children who attended medium and high quality pre-schools had higher levels of ‘Self-regulation’ at age 11 than others. Children who attended pre-school of any quality were rated by teachers as displaying significantly higher levels of ‘Pro-social’ behaviour relative to ‘home’ children, although the difference is most marked for those who attended high quality settings.

0.00 0.00

0.06

0.02

0.150.17

0.24 0.25

0.00

0.05

0.10

0.15

0.20

0.25

0.30

Pre-school Quality(ECERS-R)

Pre-school Quality(ECERS-E)

Eff

ect

Siz

es: S

elf-

reg

ula

tio

n

Home children-Reference group Low qualityMedium quality High quality

Figure 3. The impact of pre-school quality (ECERS-R and ECERS-E) on ‘Self-regulation’ in Year 6

0.00 0.00

0.16 0.160.15

0.18

0.28

0.23

0.00

0.05

0.10

0.15

0.20

0.25

0.30

Pre-school Quality(ECERS-R)

Pre-school Quality(ECERS-E)

Eff

ect

Siz

es: P

ro-s

oci

al B

ehav

iou

r

Home children-Reference group Low qualityMedium quality High quality

Figure 4. The impact of pre-school quality (ECERS-R and ECERS-E) on ‘Pro-social’ behaviour in Year 6

116 Journal of Early Childhood Research 9(2)

Children who attended high quality pre-schools also had the lowest levels of ‘Anti-social’ and ‘Hyperactive’ behaviour. ‘Home’ children were rated by teachers as displaying significantly lower levels of ‘Hyperactivity’ when compared to the low and medium quality groups but the high quality groups showed less hyperactivity at age 11 than the ‘Home’ group.

3. The combined impact of pre-school quality and Home Learning Environment (HLE) on cognitive outcomesFurther analyses were conducted to examine the combined effect of the Home Learning Environment (HLE) and pre-school quality (using the ECERS-E) outcomes at age 11. There was an interaction between these two predictors and children’s outcomes.

0.00 0.00

0.22 0.22

0.170.14

–0.01

0.02

–0.05

0.00

0.05

0.10

0.15

0.20

0.25

0.30

Pre-school Quality(ECERS-R)

Pre-school Quality(ECERS-E)

Eff

ect

Siz

es: H

yper

acti

vity

Home children-Reference group Low qualityMedium quality High quality

Figure 5. The impact of pre-school quality (ECERS-R and ECERS-E) on ‘Hyperactivity’ in Year 6

0.00 0.00

–0.05–0.08–0.08

–0.10

–0.23–0.22

–0.30

–0.25

–0.20

–0.15

–0.10

–0.05

0.00

Pre-school Quality(ECERS-R)

Pre-school Quality(ECERS-E)

Eff

ect

Siz

es: A

nti

-so

cial

beh

avio

ur

Home children-Reference group Low qualityMedium quality High quality

Figure 6. The impact of pre-school quality (ECERS-R and ECERS-E) on ‘Anti-social behaviour’ in Year 6

Sylva et al. 117

Figures 7 and 8 show that children with poor HLE scores who attended pre-school of any quality make significantly greater academic gains relative to ‘home’ children. However, the gains are highest for children who had experienced the highest quality at pre-school. Children with medium HLE scores also benefit from attending pre-school, although only if the pre-school is of medium or high quality. Children who have high HLE scores and attended a medium or high quality pre-school have the strongest positive long-term benefit in both English and Mathematics at age 11. These findings show that the quality of learning in the home interacts with the quality of the pre-school in shaping children’s development.

4. The combined impact of pre-school quality and HLE on social/behavioural outcomesOf the four social/behavioural outcomes, the only significant statistical interaction between pre-school quality and HLE was related to children’s ‘Self-regulation’. Overall, this finding indicates that a combination of high HLE and attendance at a medium or high quality pre-school are related to very high ‘Self-regulation’ scores at age 11. In addition, a high HLE seems to be a protective factor for children who do not attend pre-school, helping them to achieve higher levels of ‘Self-regulation’ in primary school. Similarly, attending high quality pre-school appears to protect against the disadvantage of a low HLE and promotes children’s later ‘Self-regulation’.

Figure 9 shows that the ‘Home’ children with high HLE scores have a higher ‘Self-regulation’ level at age 11 relative to ‘Home’ children with low and medium HLE scores. At the other end,

0

0.25 0.250.29 0.29

0.27

0.23

0.36

0.540.51

0.38

0.51

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Low Medium High

Early Years HLE

Eff

ect

Siz

e

No Pre School Low Quality Medium Quality High Quality

Reference Group:Low HLE and No Pre-School

Figure 7. The combined impact of early years HLE and quality of pre-school on attainment in Mathematics at Year 6

118 Journal of Early Childhood Research 9(2)

Low Medium High

Early Years HLE

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Eff

ect

Siz

e

0

0.29

0.370.33

0.25

0.37

0.19

0.34

0.61

0.44

0.36

0.58

Reference Group:Low HLE and No Pre-School

No Pre School Low Quality Medium Quality High Quality

Figure 8. The combined impact of Early years HLE and quality of pre-school on attainment in English at Year 6

0.00

–0.02

0.29

0.18

0.120.170.17

0.26

0.410.42

0.19

0.46

–0.20

–0.10

0.00

0.10

0.20

0.30

0.40

0.50

Low Medium High

Early years HLE

Eff

ect

Siz

es:

Sel

f-re

gu

lati

on

Home childrenLow Quality Pre-school

Medium Quality Pre-schoolHigh Quality Pre-school

Figure 9. The combined impact of early years HLE and pre-school quality (ECERS-E) on ‘Self-regulation’

Sylva et al. 119

children with low HLE scores who attended a high quality pre-school have significantly better outcomes in ‘Self-regulation’ at age 11 relative to children with low HLE but no pre-school experience. As expected, the highest scores on ‘Self-regulation’ come from the combined effect of medium or high pre-school quality and a high Home Learning Environment.

DiscussionThis study has explored the effects of process quality on the medium- to long-term developmental outcomes in children at the age of 11. It has shown that different kinds of quality relate to differ-ent outcomes; in fact, it would be surprising if this were not the case. Earlier reports showed that the ECERS-R is related to both social and cognitive outcomes at age seven, but only to social/behavioural outcomes when children reach age 11. In contrast, the ECERS-E continues to be a strong predictor of children’s cognitive outcomes at age 11. Most importantly of all, this study has shown that quality is very important in sustaining the effects of early education. Although at age seven, children in this study who had attended any form of pre-school did better than their peers who had no pre-school experience (Sammons et al., 2004; Sylva et al., 2004), by age 11 the beneficial effects of low-quality pre-school have begun to fade (Sylva et al., 2008). In fact, for some cognitive and social behavioural outcomes, attending a low quality pre-school was no better than children remaining at home.

This study is the first study to explore the combined effects of the Home Learning Environment and pre-school quality. High quality learning experiences at either home or pre-school setting can boost the development of children, thus acting as ‘protective’ factors. Previous experience of a high quality Home Learning Environment appears to act as a protective factor for children who had not attended pre-school (the home group) in terms of promoting higher levels of self-regulation. Similarly, past experience of high quality pre-school is predictive of later improved self-regulation for children who had experienced only a very poor Home Learning Environment. Thus the disad-vantage of not attending pre-school is countered if children have good learning experiences at home. Similarly, the disadvantage of a poor Home Learning Environment is ameliorated by high quality pre-school. Both aspects of early influence still show an impact on longer-term development up to the age of 11.

This article has shown that pre-school quality continues to have an effect on cognitive and social outcomes at age 11. Attendance at low quality pre-school led to fewer long-term benefits in cognitive and social development, whereas attendance at medium to high quality pre-school led to significantly greater gains. The policy implication of this study points to substantial improvement in the quality of early learning at pre-school if children, especially those from impoverished back-grounds, are to have a strong start at school. The importance of pre-school quality has been stressed in the Early Years Foundation Stage in England (EYFS; DCSF, 2007), a policy document which explicitly cites EPPE as a major part of its evidence base.

Reflections on the methods of this studyThis paper had two goals: first, to present the findings from this large-scale study as they relate to the impact of pre-school quality on children’s developmental outcomes. There was a second goal, however, and this was as important as the actual findings. The aim was to reflect on the methods and findings of EPPE in a critical way, to consider what such large-scale quantitative studies can tell us and what they cannot. It is important to place EPPE in the context of a vast intellectual ter-rain of methods for studying early childhood education.

120 Journal of Early Childhood Research 9(2)

What do such studies achieve? First, EPPE has demonstrated regular patterns of development in a large and representative sample of young children. Statistical control has enabled the ‘effects’ of the pre-school and the home learning environments to be estimated after rigorous control for individual characteristics such as gender and familial background such as parental education. In many of the analyses more than 40 percent of the variance in children’s outcomes has been explained by the variables (called predictors) entered into the statistical models. This means that the predictive variables in the EPPE study ‘explain’ a great deal about the developmental outcomes of a large and broadly representative sample. (Still, there is a lot they do not explain, and this will be discussed in the following section.)

Importantly, the EPPE findings on the effects of pre-school attendance, duration and quality of provision are net of other influences, including the baseline measures. This means that the quality effects reported here are in addition to (and net) of other influences such as the family or the child’s profile at entry to the study. Thus the effects of pre-school are ‘value added’ ones, after taking account of baseline profile and social background. The measure of progress during the pre-school period is a very precise one.

The measures of quality used in this study, however, are very broad-brush. The ECERS-R and E take only a day to administer, providing an estimate of quality that is robust but not deep. Indicators in these scales have to be ‘objective’ enough to allow inter-observer reliability, and ‘frequent’ enough to be picked up on a visit of a single day. More qualitative studies such as the EPPE case studies (Siraj-Blatchford et al., 2002, 2008) delve into the complexities of practice and the shared meanings (amongst staff) that underpin it. While such case studies illuminate the quantitative findings they also go beyond them.

And finally, studies such as EPPE establish differences in trajectories of whole groups of chil-dren, for example, those from large/small families, those who live in disadvantaged circumstances. The findings suggest that children from disadvantaged backgrounds, or those from large families, tend to do poorly. This will not hold true for each individual case; it is a generalization about groups and not an explanatory statement about individuals.

What are the limitations of large-scale quantitative research? This is an important question and the EPPE authors have never shied away from it. Understanding education and pedagogy requires both quantitative and qualitative methods. EPPE’s multi-level modelling does not pro-vide a deep explanation of the learning ‘process’, especially regarding supportive interactions between adults and children (sometimes called ‘scaffolding’). For this kind of understanding, smaller scale, more qualitative studies are ideal (see Siraj-Blatchford, 2002). Also, EPPE cannot explain ‘local’ circumstances of an individual pre-school centre; pre-schools exist in unique com-munities of people and institutions and some of this uniqueness no doubt contributes to develop-mental outcomes, which have not been included in the statistical models. No doubt the uniqueness of neighbourhoods and of cultural traditions within families contribute to individual children’s development; this fact is fully acknowledged but not accounted for in the statistical models.

EPPE does not ‘measure’ the whole child and so much that is important in development has been excluded from its statistical models. For example, EPPE measured language as an outcome, but it did not measure creativity or courage as outcomes (more’s the pity!). Large-scale studies are confined to measuring outcomes that are ‘easy’ to measure in a way that is reliable and valid – not difficult-to-measure constructs such as ‘cheerfulness’. Moreover, statistical studies such as EPPE can never explain the developmental trajectory of individual children; there are too many ‘personal’ experiences that contribute to the development of a single child for EPPE to measure and explain. Again, case studies make a vital contribution in this area and Siraj-Blatchford is carrying out case studies of individual EPPE children in secondary school.

Sylva et al. 121

Finally, and very importantly, the quantitative aspects of EPPE did not capture the ‘voices’ of thousands of children, families and practitioners. The ‘lived’ and ‘felt’ experiences of the many participants that have been included in the qualitative case studies are a vital part of EPPE. For this reason, it must always be stressed that EPPE is a mixed methods study in which individual voices are recorded and analysed (Sammons et al., 2005; Siraj-Blatchford et al., 2006) in the kind of rich detail not possible in broad-brush statistics.

Towards mixed methodsUsing a mixed methods approach EPPE studied many of the ‘individual’ aspects of practice not apparent in complex statistical models (see Sammons et al., 2005; Siraj-Blatchford et al., 2006). As Siraj-Blatchford et al. (2006, 70–71) explained:

The case studies provided a powerful means to link the qualitative and quantitative components of the study and enhance our understanding of quality. The overall aim of the qualitative case studies was to explore what made some centres more successful at achieving better child outcomes in particular domains. The aim was to tease out specific pedagogical and other practices associated with achieving ‘excellent’ outcomes as compared to ‘good’ outcomes. A stratified random sample of 12 centres was therefore selected to include those found by EPPE statistical analysis as having children who made ‘good’ progress...to ‘excellent’ progress...in one or more of the cognitive or social/behavioural outcomes, and with no negative effects on any outcome. The selection of the sample for the qualitative element of the study therefore relied on...the quantitative multilevel analyses of child outcomes in order to identify those settings where children achieved beyond expectations (p. 70–71).

The rich data on practice was therefore drawn from a robust quantitative sample. To summarize, EPPE used quantitative and qualitative methods in an iterative way to study development in fine and gross grain.

This article had two aims: 1) to demonstrate the effects of pre-school quality on children’s development, using a large-scale, quantitative study in England, and 2) to reflect on what such large-scale statistical studies can and cannot tell us. The merits of a mixed method approach are too many to be summarized here (but see Tashakkori and Teddlie, 2003). The qualitative studies of EPPE have been described by Siraj-Blatchford et al. (2006) as a dialectical procedure. They were not added on as a crowd-pleasing afterthought to ‘give flavour’; they were an integral part of the research design from the very beginning and their early findings informed the evolving quantitative methods.

Acknowledgements

This research was supported by the UK Department for Children, Schools and Families. We are grateful to the children, families and team members who contributed so generously to this study.

References

Belsky J (2001) Emanuel Miller Lecture – Developmental risks (still) associated with early child care. Journal of Child Psychology and Psychiatry and Allied Disciplines 42(7): 845–859.

Brooks-Gunn J (2003) Do you believe in magic? What we can expect from early childhood intervention programs. Social Policy Report 17(1): 3–16.

Burchinal MR, Howes C and Kontos S (2002) Structural predictors of child care quality in child care homes. Early Childhood Research Quarterly 17(1): 87–105.

122 Journal of Early Childhood Research 9(2)

Burchinal MR, Roberts JE, Riggins R Jr, Zeisel SA, Neebe E and Bryant D (2000) Relating quality of center-based child care to early cognitive and language development longitudinally. Child Development 71: 339–357.

Clarke-Stewart KA, Gruber C and Fitzgerald L (1994) Children at Home and in Day Care. Hillsdale, NJ: Lawrence Erlbaum.

Committee on Family and Work Policies (2003) Working Families and Growing Kids: Caring for Children and Adolescents. Washington, DC: National Academic Press.

De Kruif REL, McWilliam RA, Ridley SM and Wakely MB (2000) Classification of teachers’ interaction behaviours in early childhood classrooms. Early Childhood Research Quarterly 15(2): 247–268.

Deater-Deckard K, Pinkerton R and Scarr S (1996) Child care quality and children’s behavioural adjustment: A four-year longitudinal study. Journal of Child Psychology and Psychiatry 37(8): 937–948.

Department for Children, Schools and Families (2007) Early Years Foundation Stage. Nottingham: DCSF.Elliot CD, Murray DJ and Pearson L (1983) British Ability Scales. Windsor: NFER/Nelson.Gilliam WS (2000) The School Readiness Initiative in South-Central Connecticut: Classroom Quality, Teacher

Training, and Service Provision. Final report of findings for fiscal year 1999. Available at: http://nieer.org/resources/research/CSRI1999.pdf, accessed 1 July 2010.

Goldstein H (1995) Multilevel Statistical Models, 2nd edn. London: Arnold.Goodman R (1997) The strengths and difficulties questionnaire. Journal of Child Psychology and Psychiatry

38: 581–586.Harms T, Clifford M and Cryer D (1998) Early Childhood Environment Rating Scale, Revised Edition

(ECERS-R). New York: Teachers College Press.Hogan AE, Scott KG and Baurer CR (1992) The Adaptive Social Behaviour Inventory (ASBI): A new

assessment of social competence in high risk three year olds. Journal of Psychoeducational Assessments 10: 230–239.

Inter-departmental Childcare Review (2002) Delivering for Children and Families. London: Strategy Unit. Jaeger E and Funk S (2001) The Philadelphia Child Care Quality Study: An examination of quality in

selected early education and care settings. A technical report submitted to the Improving School Readiness project of the united way of south-eastern PA. Saint Joseph’s University, Department of Psychology, Philadelphia, PA.

Kwan C, Sylva K and Reeves B (1998) Day care quality and child development in Singapore. Early Child Development and Care 144(1): 69–77.

Loeb S, Fuller B, Kagan SL and Carrol B (2004) Child care in poor communities: Early learning effects of type, quality, and stability. Child Development 75(1): 47–65.

Love JM, Harrison L, Sagi-Schwartz A, Van Ijzendoorn MH, Ross C, Ungerer JA, et al. (2003) Child care quality matters: How conclusions vary with context. Child Development 74(4): 1021–1033.

McCartney K (1984) Effect of quality of day care environment on children’s language development. Developmental Psychology 20(2): 244–260.

Melhuish E, Sylva K, Sammons P, Siraj-Blatchford I and Taggart B (2008) Effects of the Home Learning Environment and preschool center experience upon literacy and numeracy development in early primary school. Journal of Social Issues 64(1): 95–114.

NICHD Early Child Care Research Network (1996) Characteristics of infant child care: Factors contributing to positive caregiving. Early Childhood Research Quarterly 11: 269–306.

NICHD Early Child Care Research Network (2000a) The relation of child care to cognitive and language development. Child Development 71(4): 960–980.

NICHD Early Child Care Research Network (2000b) Characteristics and quality of child care for toddlers and preschoolers. Applied Developmental Science 4(3): 116–125.

Sylva et al. 123

NICHD Early Child Care Research Network (2002) Early child care and children’s development prior to school entry: Results from the NICHD Study of Early Child Care. American Educational Research Journal 39(1): 133–164.

NICHD Early Child Care Research Network (2005) Child Care and Child Development: Results from the NICHD Study of Early child Care and Youth Development. New York: The Guilford Press.

Peisner-Feinberg ES and Burchinal MR (1997) Relations between pre-school children’s child care experiences and concurrent development: The Cost, Quality and Outcomes study. Merrill–Palmer Quarterly 43(3): 451–477.

Peisner-Feinberg ES, Burchinal MR, Clifford RM, Culkin ML, Howes C, Kagan SL, Yazejian N, Byler P, Rustici J and Zelazo J (2000) The Children of the Cost, Quality, and Outcomes Study Go to School: Technical report. University of North Carolina at Chapel Hill, Frank Porter Graham Child Development Center.

Phillips D and Howes C (1987) Indicators of quality child care: Review of the research. In: Phillips DA (ed.) Quality in Child Care: What Does Research Tell Us? Research Monograph of the National Association for the Education of Young Children, vol. 1, Washington, DC: NAEYC, 1–19.

Phillips D, McCartney K and Scarr S (1987) Child care quality and children’s social development. Journal of Applied Developmental Psychology 23(4): 537–543.

Phillips D, Mekos D, Scarr S, McCartney K and Abbott-Shim M (2001) Within and beyond the classroom door: Assessing quality in child care centers. Early Childhood Research Quarterly 15(4): 475–496.

Phillipsen LC, Burchinal MR, Howes C and Cryer D (1997) The prediction of process quality from structural features of child care. Early Childhood Research Quarterly 12(3): 281–303.

Sammons P, Siraj-Blatchford I, Sylva K, Melhuish E, Taggart B and Elliot K (2005) Investigating the effects of pre-school provision: Using mixed methods in the EPPE research. International, Journal of Social Research Methodology (special issue on Mixed Methods in Educational Research) 8(3): 207–224.

Sammons P, Sylva K, Melhuish E, Siraj-Blatchford I, Taggart B and Hunt S (2008a) Influences on children’s attainment and progress in Key Stage 2: Cognitive outcomes in Year 6. Research report DCSF-RR048.

Sammons P, Sylva K, Melhuish E, Siraj-Blatchford I, Taggart B and Jelicic H (2008b) Influences on children’s development and progress in Key Stage 2: Social/behavioural outcomes in Year 6. Research report DCSF-RR049.

Sammons P, Sylva K, Melhuish E, Siraj-Blatchford I, Taggart B, Elliot K and Marsh A (2004) The Effective Provision of Pre-School Education (EPPE) Project: Technical Paper 11 – Report on the Continuing Effects of Pre-School Education at Age 7. DfES/Institute of Education, University of London.

Sheridan S, Giota J, You-Me H and Jeong-Yoon K (2009) A cross cultural study of preschool quality in South Korea and Sweden: ECERS evaluations. Early Childhood Research Quarterly 24(2): 142–156.

Siraj-Blatchford I (2002) Effective provision of pre-school education: case studies (EPPE). Technical Paper 10, DFES, London.

Siraj-Blatchford, I., Sylva, K., Muttock, S., Gilden, R. and Bell, D. (2002) Researching Effective Pedagogy in the Early Years, DfES Research Report 365, London: HMSO, Queens’ Printer.

Siraj-Blatchford I, Sammons P, Taggart B, Sylva K and Melhuish E (2006) Educational research and evidence-based policy: The mixed-method approach of the EPPE Project. Evaluation of Research in Education 19(2): 63–82.

Siraj-Blatchford, I., Taggart, B., Sylva, K., Sammons, P. and Melhuish, E.C. (2008) Towards the transfor-mation of practice in early childhood education: the effective provision of pre-school education (EPPE) project, Cambridge Journal of Education, 38(1): 23–36.

Soucacou EP and Sylva K (2010) Developing instruments for assessing ‘difficult to measure’ aspects of quality in early childhood practice. In: Tucker E, Waljfrod G and Viswanathan M (eds) The SAGE

124 Journal of Early Childhood Research 9(2)

Handbook of Measurement: How Social Scientists Generate, Modify and Validate Indicators and Scales. Thousand Oaks, CA: SAGE: 61–86.

Sylva K and Pugh G (2005) Transforming the early years in England. Oxford Review of Education 31(1): 11–27.

Sylva K, Melhuish E, Sammons P, Siraj-Blatchford I and Taggart B (2004) The Effective Provision of Pre-School Education (EPPE) Project: Technical paper 12 – The final report. DfES/Institute of Education, University of London.

Sylva, K., Siraj-Blatchford, I., Taggart, B., Sammons, P., Melhuish, E., Elliot, K., & Totsika, V. (2006). Capturing quality in early childhood through environmental rating scales. Early Childhood Research Quarterly, 21(1): 76–92.

Sylva K, Melhuish E, Sammons P, Siraj-Blatchford I and Taggart B (2008) The Effective Pre-School and Primary Education 3–11 (EPPE 3–11) Project: Final report from the primary phase: Pre-school, school and family influences on children’s development during Key Stage 2 (age 7–11). DCSF/Institute of Education, University of London.

Sylva K, Melhuish E, Sammons P, Siraj-Blatchford I and Taggart B (2010a) Early Childhood Matters: Evidence from the Effective Pre-school and Primary Education Project. London: Routledge.

Sylva K, Siraj-Blatchford I and Taggart B (2010b) ECERS-E: The Early Childhood Environment Rating Scale Curricular Extension to ECERS-R. New York: Teachers College Press.

Tashakkori A and Teddlie CB (eds) (2003) Handbook of Mixed Methods in Social & Behavioural Research. London: SAGE.

Vandell DL (2004) Early child care: The known and the unknown. Merrill–Palmer Quarterly, Journal of Developmental Psychology 50(3): 387–414.

Vandell DL and Wolfe B (2000) Child care quality: Does it matter or does it need to be improved? Special report no. 78. Institute of Research on Poverty, University of Wisconsin-Madison.

Whitebook M, Howes C and Phillips D (1989) Who cares? Child care teachers and the quality of care in America. Final report of the National Child Care Staffing Study, Child Care Employee Project, Oakland, CA.