Diana Warren and John Haisken-DeNew ACER Research Conference 4 August 2013

Early Bird Catches the Worm: The Causal Impact of Pre-school Participation and Teacher Qualifications on Year 3

NAPLAN Cognitive Tests

Diana Warren and John Haisken-DeNew

ACER Research Conference4 August 2013

Funding for this research is gratefully acknowledged by the research partnership between the Melbourne Institute of Applied Economic and Social Research and the Victorian Department of Education and Early Childhood Development (DEECD). This paper uses unit record data from the Longitudinal Study of Australian Children (LSAC) Survey. The LSAC project was initiated and is funded by the Australian Government Department of FaHCSIA and is managed by the Australian Institute of Family Studies (AIFS). The findings and views reported in this paper, however, are those of the authors and should not be attributed to either FaHCSIA or AIFS.

www.melbourneinstitute.com

Key Research Questions

How does attendance at pre-school or kindergarten in the year prior to formal schooling affect NAPLAN Scores in Year 3?

Do the benefits from pre-school differ according to the qualification of the pre-school teacher?


Method and Key Findings

Data: The Longitudinal Study of Australian Children (LSAC).

Identification Strategy: Kernel Based Propensity Score Matching.

Key Findings:

Significant positive association between pre-school or kindergarten attendance and Year 3 NAPLAN Scores.

The impact of pre-school is substantial: Around 20 NAPLAN points, or 30% to 40% of one year of schooling.

Teacher qualifications are important: Children whose pre-school teacher had a degree or diploma specialising in early childhood education gained the most from pre-school.


The Importance of Early Childhood Education

The years from birth to age 5 have been identified as the most important developmental period during childhood.

Critical periods in early childhood during which particular skills and abilities are more readily acquired (Knudsen et. al., 2006).

Brain development in the first years of life lays the foundation for:

Language development and literacy acquisition

Cognitive processes

Emotional development

Self-regulation and problem-solving skills

and has a lasting impact on health, future learning and life success (McCain and Mustard, 1999; Shonkoff and Phillips, 2000).


Substantial Returns from Investment in Pre-school

As capacity for change in human skill development is highest early in life and decreases over time, the returns on public investment in high quality early childhood education are substantial (Heckman et. al 2006).

Returns to the individual in terms of increased earnings, higher education, improved physical and mental well-being.

Positive returns to society in terms of reduced crime and delinquency, public expenditure savings and increased tax revenues.

As learning is a cumulative process in which early skills facilitate further skill acquisition, the benefits of early interventions are larger and are enjoyed for longer (Heckman, 2006).

Early intervention programs are often more cost effective than later remediation (Carniero and Heckman, 2003).


Rate of Return Declines as Children Get Older

The rate of return from investment in human capital declines as children get older. Remedial programs in the adolescent and young adult years are more costly in producing the same level of skill attainment in adulthood (Heckman, 2006).

Source: Heckman (2006)


Benefits of Targeted Pre-school Programs

Significant benefits for children who attend high quality pre-school programs: Better intellectual development Higher levels of concentration, sociability and independence

Some of the best known evidence of the benefits of high quality early education experiences on later development comes from targeted early intervention programs undertaken in the United States.

These programs are intended for disadvantaged children, particularly those from single-parent or low-income families.


The HighScope Perry Pre-school Program

Targeted towards children from low-income households who were assessed to be at high risk of school failure.

A 2-year program of weekday morning pre-school routine combined with weekly home visits by program staff.

Teacher-child ratio of one to five, with teachers qualified in early childhood education.

Classrooms arranged to support children’s self-initiated learning activities as well as small-group and large-group activities.

Children who participated in the program were better prepared for school and significantly outperformed the no-program group on various intellectual and language tests from their pre-school years up to age 14, and also on literacy tests at ages 19 and 27.


The Abecedarian Project

An early intervention program administered to low-income children in the United States, with long-lasting benefits.

Infants were randomly assigned to either a treatment or control group.

All of the children received child care and health services at a centre with enriched resources.

The treatment group received intense intervention, consisting of a pre-school and school-age education program.

Statistically significant difference in the IQ of children in the treatment group at the end of pre-school and also at the ages of 8 and 12.

The children who participated in the intervention program completed more years of education, and were more likely to attend a four-year college.


Head Start

Targeted to local community needs and cater for several hundred thousand children across the United States.

Half-day, centre-based programcare for children from disadvantaged families in the two years before starting school.

Program participants may also receive related health, parenting and other child and family support services.

Significant short-term increases in test scores, as well as reductions in grade repetition and learning disability diagnosis, particularly for the most disadvantaged male children in the sample.

In the long-term, Male Head Start participants were less likely to be out of the labour

force Female Head Start participants were more likely to attend and

complete one year of college.


The Chicago Child Parent Centres (CPC)

A part-day public kindergarten program for children aged 3 and 4 who are at risk of academic underachievement.

Follow on program providing reduced class sizes and encouraging parental involvement during the first three years of school.

Run by teachers with college degrees and early childhood certification, emphasising basic skills in language and mathematics.

At age 28, CPC participants had:

Higher levels of educational attainment

Higher income levels

Significantly lower rates of drug and alcohol abuse

Lower rates of crime


Large Scale Public Pre-school Programs

Larger scale public programs often have weaker effects than targeted programs (Barnett, 1998; Dumas and Lefranc, 2010).

Mixed evidence about the long-term effects of typical pre-school programs: Long-term academic and social benefits for all children (Berlinski,

Galiani and Manacorda, 2008). Substantial gains for disadvantaged children (Fitzpatrick, 2008). Academic benefits of pre-school attendance tend to fade over time

(Goodman and Sianesi, 2005). Pre-school attendance may be associated with poorer behavioural

outcomes (Magnuson, Ruhm and Waldfogel, 2007).

No conclusive evidence that a pre-school teacher with a Bachelor degree will ensure better cognitive outcomes (Early et. al, 2007).


Early Childhood Education in Australia

Australian families are offered a diverse range of options.

Each state and territory offers non-compulsory pre-school education to children in the year prior to their first formal year of schooling.

Programs are provided in a mix of contexts, including stand-alone pre-schools, kindergartens, Long Day Care, and early learning centres.

Until recently, there were no nationally agreed or consistent standards for staffing across the child-care and pre-school sector.

As part of the 2008 COAG agreement, all children in the year before starting formal schooling now have access to an early childhood education program

Delivered by a degree qualified early childhood teacher

15 hours per week, 40 weeks per year


Costs and Incentives

Incentives provided by State and Federal governments:

Additional university places for early childhood education degrees

Removal of TAFE fees for diploma-level qualifications

Fee remissions for early childhood teachers willing to work in regional areas

Of the $1.4 billion expenditure by state and territory governments on ECEC in 2012-13, 80% ($1.1 billion) comprised expenditure on pre-school services (Productivity Commission, 2014).

Given the substantial cost of public pre-school programs, it is of particular policy interest to examine the longer-run impacts of these programs.

There are very few studies examining the impact of early childhood education and care in Australia.


Pre-school Literacy Project (PLP)

40 pre-schools across Victoria between 1996 and 1999.

Pre-school teachers encouraged to introduce literacy into their programs: Placing a writing table in the room along with appropriate resources Introducing literacy materials into the “home corner” Introducing a post box for letter exchanges Bringing the print around their room down to the children’s eye level

After one year of primary school, PLP students had: Significantly higher scores on reading and writing tests Higher level oral language skills More sophisticated phonological awareness

During the second year of school, the PLP students maintained their advantage in reading and oral language proficiency.


Child Care Choices Longitudinal Extension Study

A study of the child care and early school experiences of children in urban and rural New South Wales from 2002 to 2008 (Bowes et. al., 2009).

Hours of care, multiple and changeable care arrangements and the quality of the carer-child relationship were important predictors of children’s achievement.

Longer hours in early formal child care were found to be associated with poorer academic achievement.

Longer hours of early informal care had a positive effect on social behaviour.


Performance Indicators in Primary Schools (PIPS)

The Performance Indicators in Primary Schools (PIPS) program assesses the early literacy and numeracy skills of students.

PIPS is used to assess each student’s early literacy and numeracy skills and identify students who will benefit from intervention or enrichment programs.

Boardman (2005) compared the PIPS scores of 884 students across 38 schools in Tasmania, who began their preparatory year of school in January 2004.

Reading, Numeracy and overall test scores were significantly higher among children who had attended full-day sessions of kindergarten the previous year.


Australian Studies using the LSAC Data

Harrison (2009): Children who were attending an early childhood or pre-school

program at the age of 4 or 5 were more competent in language ability.

Vocabulary scores were negatively associated with longer weekly hours at child care or pre-school, particularly among children who were in care for more than 30 hours per week.

Claessens (2009): Examined the association between general cognitive ability and

socio-emotional skills at the age of four or five and academic achievement four years later.

Cognitive ability at the age of 4 or 5 is an important predictor of achievement in middle childhood.


The Longitudinal Study of Australian Children (LSAC)

Waves 1 and 3 of the Longitudinal Study of Australian Children (LSAC).

The first wave of interviews began in March 2004 and families are subsequently interviewed every two years.

Cognitive outcomes are measured using NAPLAN Scores for Numeracy, Reading, Spelling, Writing and Grammar.

Sample: 2229 children who had not yet begun formal schooling in 2004, with matched NAPLAN information for Year 3 in 2008.

Over 90% of children attended some type of pre-school or kindergarten program in the year before starting school.


NAPLAN Bands


Average NAPLAN Scores, by Pre-school Attendance


NAPLAN Score Distributions, Year 3 Numeracy

0.0

01

.00

2.0

03

.00

4.0

05

.00

6D

en

sity

200 300 400 500 600 7000 100 800

preschoolno preschool

Year 3 Numeracy

NAPLAN Band (%)1 2 3 4 5 6 Total

Pre-school1.6 5.5 18.7 26.9 25.5 21.0 100.0

No Pre-school*4.3 *7.5 29.0 23.2 25.7 10.4 100.0


NAPLAN Score Distributions, Year 3 Reading

0.0

01

.00

2.0

03

.00

4.0

05

.00

6D

ensi

ty

0 200 400 600 800100 300 500 700


Year 3 Reading

NAPLAN Band (%)1 2 3 4 5 6 Total

Pre-school3.3 9.0 13.6 19.8 28.6 25.8 100.0

No Pre-school*5.2 13.1 22.4 20.8 21.8 16.6 100.0


NAPLAN Score Distributions, Year 3 Spelling

0.0

01

.00

2.0

03

.00

4.0

05

.00

6D

ensi

ty

100 200 300 400 500 600 700 8000

preschool

no preschool

Year 3 Spelling

NAPLAN Band (%)

1 2 3 4 5 6 Total

Pre-school 2.7 7.9 13.6 26.9 27.0 21.9 100.0

No Pre-school *4.2 13.6 23.9 26.6 15.7 16.0 100.0


NAPLAN Score Distributions, Year 3 Writing

0.0

02

.00

4.0

06

.00

1.0

03

.00

5D

ensi

ty

0 200 400 600 800100 300 500 700


Year 3 Writing

NAPLAN Band (%)

1 2 3 4 5 6 Total

Pre-school 2.2 4.1 13.4 26.2 27.7 26.4 100.0

No Pre-school 3.2 6.8 19.0 30.6 21.7 18.7 100.0


NAPLAN Score Distributions, Year 3 Grammar

0.0

01.0

02.0

03.0

04.0

05.0

06D

ensi

ty

0 200 400 600 800100 300 500 700


Year 3 Grammar and Punctuation

NAPLAN Band (%)

1 2 3 4 5 6 Total

Pre-school 2.5 6.1 14.4 25.7 19.2 32.1 100.0

No Pre-school 6.1 7.4 22.2 25.5 17.8 20.9 100.0


Characteristics of Children who Attended Pre-school

Simple differences in test scores should not be regarded as causal effects of pre-school attendance.

They may reflect other characteristics that are correlated with both pre-school attendance and NAPLAN outcomes.

Children who did not attend pre-school were less advantaged than those who did attend.

More likely to be in low-income and lone parent households; children whose parents did not complete high school were less likely to attend pre-school.


Estimating the Effects of Pre-school Attendance

OLS estimates: Average effect of pre-school participation on NAPLAN scores.

Controlling for characteristics of :

– the child (gender, age, ATSI status, birth weight, health)

– the household (household income, household size, lone parent household, older and younger siblings, LOTE)

– the mother (education, age, employment status); and

– state and region of residence

To control for the innate ability of the child, models are re-estimated with the child’s score on the “Who Am I?” test, taken at the time of their 2004 interview.


The “Who Am I?” Test

A measure of cognitive development created by the Australian Council for Educational Research (ACER) in 1997.

Developed based on previous research about the use of copying and writing tasks for the assessment of children’s developmental level and school readiness.

Children are asked to write their name, copy and write letters, numbers, words and sentences with simple instructions and encouragement from their interviewer.

These abilities have been shown to be strongly associated with cognitive development and subsequent school achievement.

Provides a reliable measure of development which is valid across cultural groups and among children whose knowledge of English is limited.


OLS Estimates, Effect of Pre-school on NAPLAN Scores

Note: ***, ** and * represent statistical significance at the 1%, 5% and 10% levels respectively .


OLS Estimates: Characteristics of the Child

Boys score higher than girls in Numeracy.

Girls score higher than boys in Writing, Spelling, and Grammar.

Children of Aboriginal and Torres Strait Islander background have substantially lower scores for Spelling, Grammar and Numeracy.

Children who speak a second language have significantly higher Spelling scores.

Children who weighed less than 2.5kg at birth scored lower in Reading, Grammar and Numeracy.

Average test scores increase by 2 to 3 points for each additional month of a child’s age.


OLS Estimates: Household Characteristics

Children who had an older resident sibling had lower average test scores in Reading, Spelling and Grammar.

Children who had younger resident siblings had significantly higher scores for Grammar.

Across all five domains, there was a small but significant effect of household income, with average test scores increasing by approximately 1 point with every $100 of weekly household income.


OLS Estimates: Characteristics of the Mother

Mother’s education level is a very important factor.

Compared to children whose mother had not completed high school, average test scores of children whose mother had a degree-level qualification were significantly higher: Numeracy: 33 points Reading: 41 points Spelling: 22 points Writing: 30 points Grammar: 36 points

This may be due to innate ability, a better home learning environment or parents placing more value on education.


Propensity Score Matching

Children are divided into two groups:

The treatment group (who attended pre-school)

The control group (who did not attend pre-school)

Each child in the treatment group is matched as closely as possible with a child in the control group based on their observable characteristics.

Taking the difference in the mean NAPLAN scores between the set of students who attended pre-school and the observationally equivalent matched set of students who did not attend pre-school.

Propensity score matching can provide causal estimates, assuming that both the treatment assignment and the outcome of interest do not depend on unobservable characteristics.


Average Treatment Effects

Average effect of Treatment on Treated (ATT):

The benefit from pre-school attendance for those who attended.

Average effect of Treatment on Untreated (ATU):

How much higher the NAPLAN scores of children who did not go to pre-school might have been, if they had attended.

Treatment effects are estimated with and without the inclusion of the “Who Am I?” Score; and including variables measuring the home learning environment when the child was 4 years old:

How often the child was read to each week

How many children’s books in the child’s home

At home and out of home activities (e.g. music, sports)


Causal Estimates of Pre-school Participation

Without control

for ability With control

for ability

With controls for ability and home

learning environment

ATT ATU ATT ATU ATT ATU

Numeracy 18.2*** 19.3*** 15.2*** 16.6*** 12.4* 13.1***

Reading 20.9*** 24.9*** 18.0*** 21.1*** 16.1* 18.1***

Spelling 20.2*** 21.6*** 17.5*** 17.6*** 15.9** 17.0***

Writing 13.1** 15.8*** 10.1* 12.0*** 7.7 10.5*

Grammar 12.1 20.2*** 9.6 15.5*** 6.4 12.2**

Note: ***, ** and * represent statistical significance at the 1%, 5% and 10% levels respectively .


The Role of Pre-school Teacher Qualifications

In 2004, most children attended some type of pre-school or kindergarten program, but it is likely that there were substantial differences in the quality of pre-school programs.

No nationally agreed or consistent standards for staffing across the child care and pre-school sector.

Models are estimated to compare the effects of specific pre-school teacher qualifications:

Qualification of pre-school teacher %

Degree in Early Childhood Education or Child Care 47.5

Other Teaching Degree 9.8

Diploma in Early Childhood Education or Child Care 24.9

Certificate in Early Childhood Education or Child Care 10.2

Other (No relevant Child Care or Teaching Qualification) 7.5


Causal Estimates: Average Treatment Effect on Treated

Without Control for AbilityNumeracy Reading Spelling Writing Grammar

Early Childhood Education Degree 15.2** 19.4*** 17.5** 8.2 6.5Other Teaching Degree 18.4* 22.6* 19.2 19.5* 15.5Diploma 21.2*** 23.6*** 20.1** 9.7 15.0Certificate 12.6 17.4 18.1 7.81 2.4No relevant qualification 14.6 14.9 16.9* 10.2 12.3

With Control for AbilityNumeracy Reading Spelling Writing Grammar

Early Childhood Education Degree 12.5* 17.1** 15.0** 6.3 4.5Other Teaching Degree 15.5 19.3 16.3 16.6 11.9Diploma 16.5** 18.6** 15.4* 6.4 10.6Certificate 12.4 17.3 18.7 8.5 1.0No relevant qualification 14.2 13.9 17.5 8.8 10.8

Note: ***, ** and * represent statistical significance at the 1%, 5% and 10% levels respectively.


Conclusions

These are the first results for Australia that show a significant impact of pre-school attendance on later NAPLAN outcomes.

Pre-school is Important: For Reading, Spelling and Numeracy, causal ATT effects of 17 to 20

points. After controlling for ability, estimated effects are reduced by 2 to 4

points. These causal estimates are substantial, with pre-school amounting to

30-40% of the learning impact of one year of schooling, 3 years later.

Pre-school Teacher Qualifications and Specialisation are Important:

Children whose pre-school teacher had a diploma or degree in Early Childhood Education or Child Care gained the most.

These results contrast Early et. al (2007) who find no association between teacher qualification and outcomes in the pre-school year.


Policy Implications

The long-run causal impacts of pre-school attendance confirm the valueand importance of high quality pre-school programs for later cognitive outcomes.

For maximal program impact, pre-school teachers should have at least a diploma level qualification.

The COAG agreement ensuring all children have access to a pre-school program delivered by a degree qualified early childhood teacher is likely to have substantial long-term benefits.

The Productivity Commission Draft Report into Early Childhood Education and Care (2014) recommends that: “The Australian Government should continue to provide per child payments to the states and territories for universal access to a pre-school program of 15 hours per week for 40 weeks per year.”


Thank You

The complete paper is available at: http://www.melbourneinstitute.com/downloads/working_paper_series/wp2013n34.pdf

Documents

Diana Warren and John Haisken-DeNew ACER Research Conference 4 August 2013