1

Submission to the Productivity Commission draft report on the National Education Evidence Base

October 2016

2

Australian Research Alliance for Children and Youth (ARACY)

Submission to the Productivity Commission draft report on the

National Education Evidence Base

Overview

Learning is one of the important determinants of wellbeing, and quality education is the 4th

of the United Nations’ 17 Sustainable Development Goals. ARACY’s national framework (The

Nest)1 identifies six key dimensions for child and youth wellbeing: children are safe and

loved, have material needs met, are learning, healthy, participate in society and have a

positive sense of culture and identity. Each of these is a precondition for, and supports, the

other five. Thus, learning is a key factor in improving material basics, child safety, health,

democratic participation and a positive sense of identity. Similarly, each of these other

factors is a precondition for successful learning.

If we are to improve national wellbeing, broadly understood, Australia has to base its

education and learning systems2 on reliable evidence of what works, to ensure we apply our

resources to their best use and deliver outcomes. This inquiry is thus vitally important.

The draft report’s identification of the problem - that reforms to date have not produced

improvement in measures of student achievement - is a reasonable start to a problem

definition, albeit the way in which student achievement is characterised and resultant

measures identified in the report are somewhat limited in their scope. We suggest the

Commission take a broader approach to measurement of education outcomes that includes

other important dimensions of wellbeing which shape the learning process. ARACY

members with an interest in the social determinants of health and wellbeing are particularly

keen to draw attention to the key role education plays in producing a healthy society. We

note that greater collaboration between the fields of health and education would

significantly improve the evidence base for both.

1The Nest action agenda was launched in 2013. Information to support The Nest was gathered through

national consultations in 2012 involving more than 4,000 children and youth, parents, leading thinkers and researchers, child advocates, policy makers, and service planners and providers across Australia. These included a national survey with more than 3000 respondents, half aged 24 or younger, 500 face to face interviews with young people, and other consultations with experts in children and youth issues.

The Nest is also supported by evidence gathered from research (including reviews of effective interventions) and by data from the 2013 ARACY Report Card: The wellbeing of young Australians. It is the most comprehensive evidence-based plan for children and youth produced in Australia. 2 The term “learning system” we see as encompassing people, policies, practices, organisational arrangements,

governance, monitoring and evaluation, reporting and accountability.

3

No matter what way outcomes are interpreted, we strongly agree that resourcing and

accountability measures alone will not achieve gains in education outcomes (p.3) 3. Better use

of reliable evidence to improve practice is required.

ARACY supports the conclusions of the draft report that “monitoring outcomes, performance

benchmarking and competition between schools alone are insufficient to achieve gains in

education outcomes” and there is an important need to “identify, and then apply, the most

effective programs, policies and teaching practices” (p3). We do however note that the

practices that contribute to learning outcomes include more than classroom teaching

practices. They include for example early learning experience to give children a good

foundation prior to school (which the report itself recognises later), and student4, parent and

community engagement in learning processes.

Professionals and practitioners involved in education of children and young people would

benefit by learning from evidence-based research that informs structured continuous

professional learning. This would help improve the pedagogy, not just the curriculum and/or

syllabus of early years/foundation level teacher education. Pedagogy looks beyond teaching,

it embraces school and community ethos and how learning, teaching and development is

influenced by cultural, societal and political values we have for children. Developing such

improved pedagogy should be reinforced through a solid theoretical and practical basis that

draws on the best available evidence.

ARACY also agrees strongly that an evidence base is more than a national data repository. A

strategic approach to improving education outcomes through evidence has to include

improvement in our capacity to gather, apply, and continuously improve various forms of

relevant evidence. There is a pressing need for a two way flow, evidence to practice and vice

versa. Qualitative research is also important – it helps answer the “why?” questions about

success or failure of an education practice, complementing quantitative “what happened?”

evidence.

ARACY is pleased to see recognition of the importance of early years (“children’s physical,

cognitive, linguistic, social and emotional development and learning from birth to five years”:

p.37). These years are a key determinant of future outcomes, and continuing investment in

these years – especially to reduce the adverse effects of disadvantage – has a huge payoff for

governments and for Australia as a whole. Early identification of potential problems (through

3 References to page numbers in this submission are to the National Education Evidence Base draft report as

published online by the Productivity Commission.

4 Student engagement in learning is an important element of The Nest. The action agenda recommends a

number of strategies to promote student engagement (p.14). These strategies are also linked to and supported by strategies to promote the participation of young Australians.

4

screening or other means) and targeting of interventions aimed at overcoming disadvantage

are fundamental to a preventive approach to improve wellbeing.

A significant gap in the draft report is that it does not adequately address the last of the

inquiry’s Terms of Reference, to consider “potential barriers and challenges” to use of

evidence to improve outcomes. Australia needs thoroughgoing, persistent institutional

reform to overcome them.

The report demonstrates an optimistic but misplaced faith that gathering and disseminating

better evidence will be sufficient to change practice. For evidence to be taken seriously there

have to be receptive users and a willingness to change existing practices if evidence supports

the need. Both require cultural and institutional support; without that support, evidence that

challenges the status quo will be dismissed. This wider understanding is supported by much

of the previous analyses of the Commission in relation to the conditions under which major

reforms are possible. The observation that evidence alone is often insufficient to change

practice or policy is itself well supported and documented5.

An institution responsible for implementation of the evaluative research framework (draft

recommendations 7.2 and 8.1) could be ARACY. We are already active in this space, having

developed a national action plan and strategy for evidence (The Nest) that takes a holistic

approach to the wellbeing of children and youth. While we see the attraction of this function

being undertaken by an education specialist, the advantage of the ARACY approach is that it

recognises that wider aspects of wellbeing such as material needs, health, safety, and cultural

identity are also crucial determinants of learning outcomes. ARACY has more than sufficient

organisational capacity through its broad based membership to undertake this work. As

recognised in the draft report any such institution would require appropriate resourcing to

implement and administer the framework.

5 For a good example see Smith, K (2013) Beyond Evidence Based Policy in Public Health – the Interplay of

Ideas, Palgrave McMillan London

5

SUBMISSION

Introduction

The following sections of our submission address a range of issues raised by the draft report

that ARACY members have identified as important. They follow roughly the order of the

draft report, and thus are not necessarily presented in priority order.

Parent engagement

We note the observation of the draft report that

...there remain some significant gaps. Data are lacking, for example, on the nature of

parents’ engagement in their child’s education, and the culture of the home learning

environment. There is merit in collecting these data, but they do not have to be

collected for all students. The data considered to be most relevant could be collected

for a representative sample of students.” (p.14).

This has been recognised by the Federal government, and there is a large project underway,

coordinated by ARACY, on parent engagement. It represents a very successful model for

evidence creation and dissemination, including the establishment of a new but already highly

active Parent Engagement Network, and a body of information to support policy making. It

is a good case study of how to fill gaps in evidence – we have provided this case study at

Attachment A.

One source of existing data is two surveys undertaken by Dyslexia Support Groups, one in

Tasmania (2014-15) and one nationwide (2016) 6. These online surveys examined, amongst

other issues, parent engagement with schools. Both surveys found in the majority of cases

there was reluctance on the part of schools to engage with parents of children who had

specific learning difficulties. The majority of parents had to instigate and sustain connections

between themselves and schools.

6 On line survey Square Pegs Dyslexia Tasmania Support Group

6 On line survey Nationwide Australia http://www.outsidesquare.net/wp-

content/uploads/2015/04/Australian-Parent-Survey-on-the-Education-of-Students-with-Dyslexia-Final-

Report.pdf Forbes (2016)

6

Aboriginal and Torres Strait Islander education evidence

While the PC is right to refer to Australia as a developed nation (p.3), that description does

not apply to indigenous Australia. World expert on social determinants of health Sir Michael

Marmot has observed that “Australia ranks right up there, second or third on the Human

Development Index...[but]...Indigenous Australians, if you treated them as if they were a

separate country, would rank probably about 100th or below 100.”7

As The Nest action agenda describes: “Inequalities in outcomes between Aboriginal and

Torres Strait Islander and non-Indigenous children, young people and families in Australia

are marked and persistent, evidenced most notably in measures of poverty and deprivation,

early childhood vulnerability and educational attainment as well as in representation in child

protection, out-of-home care and youth justice”.

The extreme differences between the experiences of many Aboriginal and Torres Strait

Islander Australians and the non-indigenous population mean that the problem definition

and potential solutions for indigenous Australia are likely to differ in important ways to those

for Australia as a whole. There is a strong case for a separate chapter in the final report to

address the specific evidence gaps in education practice for Aboriginal and Torres Strait

Islander Australians. Findings from a number of important local-level initiatives could inform

this chapter.

The links between education and health

There is a wealth of evidence available on the health benefits of education and its impact

long term on a nation’s health status8. Education is also closely linked with the broadest

measures of social and psychological wellbeing9.

7 Interview, 18 May 2012, at http://www.abc.net.au/sundayprofile/stories/ See also his recent Boyer Lectures.

8 See Braveman P, Sadegh-Nobari T, and Egerter S (2011) Early Childhood Experiences and Health. Robert

Wood Johnson Foundation at http://www.rwjf.org/en/library/research/2011/03/early-childhood-experiences-and-health.html; Virginia Commonwealth University Center on Society and Health (2014) Education: It Matters More to Health than Ever Before, at http://www.rwjf.org/en/library/research/2014/01/education--it-matters-more-to-health-than-ever-before.html

9 See for example http://www.commissiononhealth.org/PDF/c270deb3-ba42-4fbd-baeb-

2cd65956f00e/Issue%20Brief%206%20Sept%2009%20-%20Education%20and%20Health.pdf

Francesconi, Marco and Heckman, James J., Symposium on Child Development and Parental Investment: Introduction. IZA Discussion Paper No. 9977. Available at SSRN: http://ssrn.com/abstract=2790014

Jones DE, Greenberg M, and Crowley M (2015) Early Social-Emotional Functioning and Public Health: The Relationship Between Kindergarten Social Competence and Future Wellness, American Journal of Public Health 105 (11): 2283-2290.

7

Education has profound health effects. In general, more schooling – not just health

education – equips a person to change behaviour in healthy ways. This effect is independent

of, but compounds, the well known effect of education on income and wealth (better

education leads to greater income which itself is linked to better health outcomes). Evidence

based investment in education will have health benefits for Australia as a whole.

While the draft report notes that “Data and evidence from sources outside education, such

as the health sector, can be useful in understanding the impact of external influences. For

example, information on family and socioeconomic background...” (p.58) the draft report

significantly underestimates the importance of the interactions between education and such

external influences. We do however commend the recognition in the draft report that

student wellbeing is both an influence on education outcomes and an outcome in its own

right (p.87), and its references to the Australian Child Wellbeing Project, an important

contributor to the data.

The Strengths and Difficulties Questionnaire (on p.88 the draft report notes that the AIHW

has recommended development of an indicator based on that questionnaire, and implicitly

endorses using it as an evidence base) is a mental health assessment tool – only one aspect

of wellbeing. It is clearly important within the health domain, but necessarily misses other

important aspects of wellbeing. While there is no agreed national aggregate measure of

children and young people’s social and emotional wellbeing, The Nest provides a framework

of indicators for measuring social and emotional wellbeing10 and a range of evidence-

informed strategies for improving the social and emotional wellbeing of young Australians.

We suggest that if an assessment tool were to be used to develop data, the Commission

might investigate whether there are more comprehensive methodologies available.

Wider evidence for education outcomes could also consider how well children are

developing qualities such as resilience, tenacity and gratitude. Holistic considerations need

to be embraced: education outcomes are broader than solely ‘achievement’ as measured

through NAPLAN scores or school league tables.

WW4K

We note the PC finding that

a central repository of trusted, high-quality evidence, including resources to support

practitioners, is needed.

10

This is measured, in The Nest, through measures of the ‘Loved and safe’ outcome, including: ‘positive supportive family environment’, ‘positive parenting practices’, ‘positive, trusted peer relationships and communication’, ‘community safety’, ‘children not placed in care’ and ‘youth not in detention’.

8

The US Institute of Education Sciences manages a repository of this type — the What

Works Clearinghouse. The Clearinghouse reviews research on policies, programs,

practices and products in education. High-quality evidence is summarised in

effectiveness ratings for different interventions and practice guides. (overview, p.17)

We draw your attention to a project currently underway with support from the Australian

Government and guidance from senior academic researchers, What Works For Kids (WW4K).

This will have a similar function to the US What Works Clearinghouse cited in the draft

report.

WW4K is an online database and discussion forum for researchers, practitioners and policy

makers working to improve child and youth wellbeing. Originally funded by the Australian

Government Department of Social Services, WW4K will provide the latest information about

the best evidence-based interventions for decision-makers to make informed investment

choices.

The project is led by leading researchers: Professor John Toumbourou, Chair in Health

Psychology, Faculty of Health, Deakin University and Associate Professor Sophie Havighurst,

Centre for Training and Research in Developmental Health, Department of Psychiatry,

University of Melbourne.

Evidence standards for each intervention are being classified using a Rapid Evidence

Assessment (REA) rating scheme adapted by ARACY from a REA developed by the Parenting

Research Centre and drawing on California’s Evidence-Based Clearinghouse for Child

Welfare. The ratings scale the quality of evidence from “well supported” through to

“emerging”, based on factors including the number of Randomised Control Trials (RCTs),

Quasi-Experimental analysis, and quantitative or qualitative analysis supporting the

intervention; all analyses should reveal positive outcomes. Additionally, sustainment of

positive effect is also incorporated into the REA.

During the initial development phase, more than 150 programs have been selected for

assessment. They will be published on the website once their ratings have been reviewed

(10 programs have already been published). Upon completion of the initial phase, it is

planned that individual programs will be able to request to add their programs to the

database.

Ethics

The draft report’s discussion on ethical approvals in Chapter 5 passes over a key issue –

introducing a new practice with accompanying research to determine the evidence (e.g. an

evaluation or RCT) requires approval by an ethics committee, an arduous process, whereas

introducing a new practice with no measurement or evaluation does not have to go through

an ethics committee. Indeed, the latter may attract applause for ticking the “innovation” box.

9

As noted in the report, even when the ethics approval hurdle is overcome there are multiple

other barriers to researchers obtaining the data required for analysis including privacy and

other legislative hurdles.

These factors bias decision making of education administrators towards introduction of new

interventions without evidence. We presume this is an unintended consequence of ethics

processes, not system design.

Moreover, system-wide level of difficulty in researchers obtaining and analysing information

increases the risk of interventions with no evidence base becoming entrenched practice and

remaining unexamined.

In light of this, we suggest the PC recommend that at a minimum new education practice

should be subject to a trial or pilot. These are still useful forms of evidence. Trialling a new

approach means that should it be found to be ineffective it can be retired gracefully before it

consumes too many resources or harms educational outcomes.

We disagree with the observation that “A key concern with randomised controlled trials lies

in the ethics of one group receiving a treatment that may advantage or disadvantage

them...although some students might be disadvantaged through trial participation, the

potential benefits from trial insights are significant...” (p.166). We rarely know in advance

what the impact of an intervention will be. In those rare cases where it is completely obvious

that an intervention will be beneficial, then it should be introduced without a trial. If there is

doubt, then the ethical question should be resolved in favour of a trial. Where a trial gathers

information on impact11, logically the only three possible outcomes are: benefit, no effect, or

disadvantage. Consider each of these.

A – Benefit. A new practice may have benefits. We should be certain of that before

rolling it out for the whole population. A trial gives us the evidence. Some

participants will gain that benefit early, while the control group misses out: which is

still a welfare gain, and better than not using evidence and so risking an intervention

that either has no benefits or has disadvantages. We should adopt the ethical course

used in medicine: if benefits are recognised early in the trial, and can be proven, then

the intervention can be applied to the whole population rather than wait until the

previously scheduled end date of the trial.

B – No effect. If an intervention has no effect, as demonstrated by a reliable trial,

then the public is saved the expense of introducing a reform of no value. The cost of

the trial is tiny compared with the cost of a useless programme; bear in mind that in

11

Sometimes the result of a trial may be that further work is still required to understand the impact – that is, outcomes are uncertain and the trial is inconclusive.

10

education a new practice will commonly be introduced across a large number of

schools either in a State/Territory or nationwide, at considerable cost to taxpayers or

fee paying parents.

C – Disadvantage. If an intervention causes harm, trialling it with a small group at

least confines that harm to a small number, rather than affecting all children. As with

the no effect case, the public is saved the expense of introducing a system-wide

reform that causes more harm than good, and also the further expense of having to

unwind that reform further down the track as evidence of its harm emerges. The

ethical approach to the length of the trial is opposite to the benefit case: if the trial

shows early proven results that the intervention is causing harm, it should be

discontinued.

The counterfactual is the introduction of a new approach (or “treatment” to use the language

of the draft report) without a trial. It is doubtful whether it is ethically superior to introduce a

new educational practice or approach without evidence of its effectiveness and run the risk,

however unintended, of entrenching something that might harm children’s learning.

Improving the use of evidence

We agree with the diagnosis in ch.6. For example, we agree that “enabling ongoing linkage

between Commonwealth and jurisdictional data collections remains a significant gap in

current linkage arrangements”. This continues to be a problem for researchers. We suggest

the PC not only identify the problem but work with the responsible agencies including AIHW

to develop a recommendation for solving the problem. Given the value and potential great

cost benefit of linking health and education data, the report should recommend that AIHW

should no longer have to wait ‘further resources from the Australian Government to gain

approval to procure and link data’ (page 94). There is also great merit in a universal identifier

that can be used to track children’s progress over time and into adulthood.

A further important area of interest to ARACY is creation of good evidence and then

enabling the use of (or in the Commission’s words, “mobilising”) such evidence.

We agree that better data, including linking existing data and giving researchers greater

access to that data, will help. In light of community mistrust of governments, ceding control

of access to the data to a trusted third party is a desirable mechanism. In this context,

making jurisdictions responsible for the systemic de-identification of data will be helpful, as

would a universal student identifier.

11

The draft report in some parts appears to draw heavily on the medical evidence literature. It

suggests for example that randomised control trials (RCT) are the “gold standard”12.

ARACY is strong proponent of RCTs as a way of gathering evidence in many circumstances,

but urges caution about application of medical parallels to education. RCTs should be done

where appropriate, but complemented by numerous other evidence gathering techniques.

Determining the causality of any given intervention requires careful research to compare

expected and actual results. Within a medical model paradigm it works well with proving a

single treatment delivering health benefit whilst excluding a placebo effect.

RCTs also have to exclude the very variables that need to be considered with educational

interventions. This important point is made on page 13: “Education outcomes are affected by

influences that the education system cannot directly manage, for example, a child’s gender

and health and the culture of their home learning environment. It is important to take these

external influences into account when evaluating the effects of education policies, programs

and practices on education outcomes.13

ARACY favours RCTs in many circumstances: for example, the ARACY, CCCH and WSU

randomised control trial of a sustained nurse home visiting program, Right@home, is a

leading example of a well conducted RCT. Nevertheless they are not always or necessarily

the highest standard of evidence. Randomisation is used to address selection bias. Double

blind settings in a randomised control trial address researcher bias. In education,

randomisation is particularly important where there is a strong advocacy group for a new

technique or organisational solution: were the advocates to choose the intervention and

comparator group there would be a high probability of selection bias.

However, RCTs may not be suitable in all cases. A non-random trial that compares outcomes

between an intervention group and a control group might be just as valid as an RCT in a case

where selection bias is not an issue. This could for example be the case with research

comparing outcomes from an intervention in a group of schools where, through an accident

of funding or local advocacy, an intervention has been trialled with results from comparable

schools where it has not been applied. ARACY strongly agrees that there would be value in

more RCTs in education; but would argue for use of other forms of reliable evidence as well.

12

“Gold standard” is itself a description derived from the medical evidence literature, which appears to have

strongly influenced this section of the draft report. We note that even in medicine it has long been recognized

that observational evidence can in some circumstances be more appropriate than a RCT.

13 A further problem is that “…most evaluative research focuses on short-term outcomes of downstream

interventions addressing clinical or behavioural risk factors” Pons-Vigues et al 2014

12

In addition, we note that governments are prone to miss opportunities for pilot testing to

ensure there are no unintended consequences associated with a policy. For example, the

recent State government decision to lower the school starting age to four in Tasmania would

have represented an ideal opportunity to trial the approach in a specified region rather than

taking an “all or nothing” gamble on its efficacy.

It is understandable that great store is placed in RCTs and meta-analyses as they have a

proven track record in the biomedical field. They are however less appropriate if applied as a

yardstick to inform the decisions of funders about educational programs and interventions

to continue, cease or modify, or to implement alternative approaches. In most medical

research based on RCTs the underlying principle for evidence is to inform what therapeutic

measures effectively lead to health benefits for the patient. The equivalent to the patient in

the education system is the individual student. If considering a whole population (i.e.

evidence for education systems rather than an individual student) the relevant branch of

medicine is Population or, more broadly, Public Health.

If medical research is to be used as a guide for evidence, an alternative approach the

Commission could consider is social epidemiology’s evidence framework: this is likely to

have closer parallels to education than clinical trials.

The individual as opposed to whole population approach is very different and can lead to

paradoxes such as the one espoused by Geoffrey Rose: namely that large numbers of people

must participate in a preventive strategy for direct benefit to relatively few14.

The draft report appears not to distinguish between approaches targeting individual

students and approaches that apply across all schools. It would helpful to divide the two to

take account of Rose’s paradox and also his theorem: “...a large number of people at small

risk may give rise to more cases of disease than a small number who are at high risk”

Validation and ensuring quality of evidence

Research needs to be validated – a key part of what ARACY does through its members.

Education research can be considered a subset of broader social research. Our experience

with such research is that validation to make it useful as evidence requires understanding of

both methodology (to ensure the results are reliable) and applicability (to ensure that small

sample, test or laboratory results are transferable into practice).

We are not sure the heading in ch. 7 “a different type of research is needed...” conveys the

right message. It reinforces the impression that all current education research is descriptive

14

Rose, G. 1981. ‘Strategy of prevention: lessons from cardiovascular disease’. British Medical Journal, 282, 1847-1851.

13

in nature. This is an over-generalisation. Many ARACY members with an interest in children

and youth outcomes are engaged in research to determine “what works best to improve

outcomes... [using] high quality, or rigorous, analysis of questions like this...and sophisticated

research methods”. Recent examples include the Australian Child Wellbeing Project,

recognised elsewhere in the draft report.

The draft report’s observation that research needs to be of high quality is self evident. The

important question however is what constitutes an appropriate measure of quality. Peer

review alone can often be insufficient – there is an extensive literature on the weaknesses of

peer review as a mechanism for quality assurance15. As noted previously, a RCT can often,

but may not always, be the best research methodology in terms of both validity of results

and cost effectiveness. A better test is whether the research methodology has been chosen

on the basis that it is the one best suited to the question to be considered.

The applicability of the research should also be part of the verification process for quality.

Through its broad membership ARACY makes sure research is not simply peer reviewed

academically, but is considered by users of that research. Practitioners in the field can and

generally will comment with insight on the usefulness of research proposals and research

results.

Evaluation, another important form of evidence, is useful if done well, but highly misleading

if done poorly. When it comes to improving outcomes the quality of available evidence is at

least as important as quantity. A good evaluation is relevant; ethical; independent, with no

bias or preconceptions; has a sound technical methodology; and delivers useful findings

focused on outcomes and impact.

We note the Commission’s support for “a central clearinghouse that efficiently and

effectively disseminates high quality evidence” (p174). We draw the Commission’s attention

to ARACY’s current role in doing exactly that, through our weekly e-bulletin, monthly

members’ newsletter, website, the dedicated What Works for Kids website currently in

construction, conferences, seminars, webinars, emails and other communications. We could,

with suitable resourcing, expand this role in collaboration with members with education

expertise.

Overcoming barriers to use of evidence

Section 7.2 on improving use of evidence is limited in scope. While there are impediments

to the use of evidence by policy makers, education providers and parents, these go well

15

See for example Smith, R (2006) ‘Peer review: a flawed process at the heart of science and journals’ Journal

of the Royal Society of Medicine London ; Wenneras and Wold (1997) Nepotism and Sexism in Peer Review

Nature Vol 387/22

14

beyond what the draft report identifies as capacity or capability to use data (p.176, re public

servants; p.178, re teachers) or interpret information (parents, p.179). We draw the

Commission’s attention to the extensive literature on social, cultural and institutional barriers

to use of evidence in medical practice (and some in education practice16). These studies

provide ample cases where evidence has been clear, unambiguous and totally ignored.

There is unfortunately less empirical evidence on barriers to use of evidence in Australia.

Based on advice from our members we could put forward a few possible candidates in

addition to the issues of capability as identified by the draft report. These include:

Evidence is often dismissed, even if of high quality, when it challenges existing

entrenched views on education held within education institutions or by high profile

individuals

The tendency for an education practice, once introduced, to remain in place

regardless of whether or not evidence supports it; path dependence17

Sometimes charismatic, high profile advocates can be more influential on whether or

not a practice is adopted than the actual evidence

Low levels of genuine communication between research, policy and practice

Australia’s federal system often leads to disparagement of evidence collected in one

State or Territory as irrelevant or inapplicable to another (otherwise known as the

“not invented here” syndrome)

Short term decision making favours quick rollouts of new initiatives at suitable times

in the political cycle even if evidence suggests a practice is not effective; meanwhile,

other practices that take longer to implement may be overlooked even if supported

by reliable evidence.

Further detailed inquiry would be required before we could reach conclusions on these

suggestions. We recommend the Commission investigate these and other possibilities. We

16

Jacob, B (2015) When Evidence is Not Enough: Findings from a Randomized Evaluation of Evidence-Based

Literacy Instruction (EBLI) NBER Working Paper No. 21643, Washington DC. See also Bostic, R Narrative and vehicle:

using evidence to inform policy (at http://www.whatcountsforamerica.org/portfolio/narrative-and-vehicle-using-evidence-to-

inform-policy/). We particularly note his view that “developing evidence is not a sufficient condition for implementing

evidence-based policy. More is needed”.

17 See for example Bradford, D and Wan W (2015) Reading Recovery: a sector-wide analysis, Centre for Education

Statistics and Evaluation, NSW Government, Sydney. This study examined, for the first time, the evidence for a

reading program and found it wanting: after it had been in place for more than 30 years. This case is a good

illustration of the value of establishing data and analysis units in education departments.

15

note the current work being done by Prof. Brian Head on use of evidence in public policy as

a good starting point18.

An evidence institution

On institutional arrangements, we again note that ARACY is already in place to encourage

the application of evidence to practice; the difference between ARACY and other bodies is

that we take a holistic view of children and youth needs. These encompass not only learning

but also safety, health, material needs, participation, and a positive sense of culture and

identity. Consideration of learning in isolation will miss these other key elements and their

interaction. To take a simple example: a child will be far less likely to achieve success in

learning if hungry, homeless, subject to repeated violence or in severe ill health.

We note the governance arrangements for a proposed evidence organisation as suggested

in the draft report omit some important aspects of good governance such as ethics and

legality; we suggest the Commission consider a broader approach to governance19.

If an evidence collecting and dissemination institution is established it should be flexible and

open to alternative forms of evidence rather than confined to predetermined approaches. A

culture in which evidence is collected and analysed in as frequent and varied a fashion as

possible will allow for serendipity: if a school, early learning centre, non-government

organisation or individual has developed an effective education practice, supported by a

range of reliable measures, it is important that others know about it.

Use of AEDC in evidence

A comment on page 79 refers to the expense of NAPLAN and AEDC and suggests this

expense could be reduced by sampling such as “high risk or disadvantaged groups”. In

relation to the AEDC this would render it no longer a census and would run counter to Rose’s

argument against the high risk approach as well as Marmot’s ‘proportionate universalism’

which could be equally applied to student populations: To reduce the steepness of the social

gradient in health, actions must be universal, but with a scale and intensity that is

proportionate to the level of disadvantage.

18

See for example Head, BW (2016) Toward more evidence-informed policymaking? Public Administration

Review, 76 (3): 472-484; Head BW, Ferguson M, Cherney A, and Boreham P (2014) Are policy-makers

interested in social research? Exploring the sources and uses of valued information among public servants in

Australia. Policy & Society, 33 (2): 89-101.

19 see for example Ramesh M, Saguin K, Howlett M, and Wu X (2016) Rethinking Governance Capacity As

Organizational and Systemic Resources, Lee Kuan Yew School of Public Policy Research Paper No. 16-12,

available at: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2802438; or Bartos S. (2004) Public Sector

Governance Australia, CCH, Sydney. We do appreciate that there is little experimental evidence for

governance practices, most of the literature is based on observational evidence.

16

Although the cost of the AEDC is not identified in the draft, it is likely not to have included

the return on investment from the community mobilisation and implementation of preschool

strategies precipitated and motivated by the provision of the results to local and regional

policy makers20. This needs to be considered in any review of whether the AEDC is “fit for

purpose”. Arguably its fitness for purpose is yet to be fully realised. It has already been found

to predict NAPLAN scores21; it is likely only a matter of time before the AEDC results will

predict entry to tertiary education, employment and longevity with many future

opportunities to inform policy and program interventions.

We have a concern with the observation on page 81 that “…there would be benefit in early

identification of children who might be vulnerable to poorer education outcomes, the

evidence suggests that assessment of all children would be difficult and would not yield

reliable information.” We observe that child and family health nurses and/or general

practitioners at present conduct periodic growth and developmental assessments in the

years prior to children commencing formal schooling. These are used to identify early signs

of speech, language, motor skill and other developmental delay, and facilitate early

intervention via referral to targeted services. This early engagement with preventative health

care is an essential feature of the Australian system and near universal.

Research suggests that by the time children start school, inequalities in learning and

development are already entrenched, and trajectories are difficult to shift. The draft report’s

recommendation on page 85 that all children be assessed in year 1 seems counter to the

observation on page 81. Screening in Year 1 would not be timely, as it would missed both

the most sensitive periods of early development and have no influence on AEDC results.

20

See Guhn M, Janus M, Hertzman C. The Early Development Instrument: Translating School Readiness Assessment Into Community Actions and Policy Planning. Early Educ Dev. 2007 Oct 11;18(3):369–74; Sayers M, Coutts M, Goldfeld S, Oberklaid F, Brinkman S, Silburn S. Building Better Communities for Children: Community Implementation and Evaluation of the Australian Early Development Index. Early Edu Dev. 2007 Oct 11;18(3):519–3; MCcormack JM, Verdon SE. Mapping speech pathology services to developmentally vulnerable and at-risk communities using the Australian Early Development Census. Int J Speech Lang Pathol. 2015 May 4;17(3):273–86; O’Connell M, Fox S, Hinz B, Cole H. Quality Early Education for All.

21 Brinkman S, Gregory T, Harris J, Hart B, Blackmore S, Janus M. Associations Between the Early

Development Instrument at Age 5, and Reading and Numeracy Skills at Ages 8, 10 and 12: a Prospective Linked Data Study. Child Indic Res. 2013 Apr 14;1–14.

17

Attachment A

Case Study - Building evidence in education - Parent Engagement

Parents’ attitudes, values and actions positively influence children’s education outcomes.

Partnerships between families and schools play a big role in supporting, promoting and encouraging

children’s learning and wellbeing.

At a practical level, parent engagement involves all the people in a child or young person’s life

working together to create a stimulating and supportive learning environment. This means there’s a

role for schools to work in partnership with families and to build their capacity for parent

engagement. Equally, it’s important for families to take up these opportunities in their everyday

interactions with their children, to promote reading, encourage children to pursue their passions

and connect learning at school, in the home and in the community.

Why is parent engagement important?

The classroom is not the only place that children learn. Parent engagement recognises that families

guide their children’s orientation towards learning, from birth.

This means that families have an important role to play in helping their children to become confident

and motivated learners, regardless of parents’ occupation, education or income.

Research shows that children’s learning and wellbeing improves when families and schools work

together. Parent engagement affects what children achieve, how they experience school, and

assists in the transition to school and into post-secondary education.

Parent engagement has also been shown to reduce the impact of socio-economic disadvantage on

educational outcomes.

About ARACY’s Parent Engagement Project

ARACY’s work has been commissioned by the Australian Government Department of Education and

Training over four years (2014 – 2018), with a specific focus on primary and secondary school age

children.

The project has a broad national focus and will also address the specific needs and priorities of

Aboriginal and Torres Strait Islander families and communities.

Through the project, ARACY will use the best available evidence to facilitate a shared understanding

of what parent engagement is and why it matters. It will build and share information about ‘what

works’ to strengthen parent engagement, and contribute to a shared approach to measurement to

enable the ongoing assessment of the impact and effectiveness of parent engagement policies and

strategies.

18

Work streams and priorities for the Parent Engagement Project

Develop a shared

understanding

Build the profile Data and

measurement

Build and share

Evidence

Conduct high

qualitative

research with families

and educators to

understand the beliefs,

attitudes and practices

around engagement in

learning and wellbeing

„.

Promote an evidence

informed approach to

parent engagement

between families,

schools, researchers

and policy-makers

„.

Develop practical

resources to help

schools, community

organisations and

others forge family-

school partnerships

based on trust and

two-way

communication about

parent engagement.

„. Establishing and

maintaining

relationships

with key stakeholders

„. Utilising National

Ambassadors to

highlight the important

role parents play

in supporting their

children’s learning

„. Develop Local

Champions to promote

evidence informed

approaches to parent

engagement within

their communities

„. Implement an

Australian

Parent Engagement

Network to facilitate

information sharing,

collaboration and

evidence-to-practice

nationally

„. Facilitate ongoing

collaboration,

partnership and

engagement to

promote and build

parent engagement in

Australia.

Support outcome

measurement in

research and practice

of parent engagement

through:

„. a shared national

definition of parent

engagement

„. the continuous

review of data

collections and

measurement of parent

engagement in both

research and evaluation

of programs

„. a framework to

support ongoing

measurement

and evaluation of

parent engagement

programs.

„. Critically review and

share Australian and

international research

and practical evidence

about ‘what works’

to improve parent

engagement practice

„. Contribute to

capacity building for

parents, teachers and

school leaders.

Develop a shared

The Australian Parent Engagement Network

The Parent Engagement Project aims to develop a shared understanding of parent engagement and

to disseminate information about ‘what works’ to strengthen its practice and encourage ongoing

measurement of its impact. As a key part of the project, ARACY is establishing the Australian Parent

Engagement Network, a virtual space for people passionate about engaging families in their

children’s learning and education to collaborate on and develop a better understanding of evidence.

19

Convened by a panel of experts working in the field of parent engagement, Network members

include researchers, educators, parents, policy makers, community organisations and others

interested in parent engagement in children’s learning.

The Network has four core functions – to share information, to advocate for parent engagement, to

build partnerships and to provide expert consultation to the Parent Engagement Project. Overall, the

goal of the Australian Parent Engagement Network is to drive better understanding and promotion

of parent engagement with the aim of embedding the concept and its practice in schools across

Australia.