19
A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah Buckley, Statistician York Trials Unit, University of York

A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

Embed Size (px)

Citation preview

Page 1: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

A randomised controlled trial to improve writing quality during the

transition between primary and secondary school

Natasha Mitchell, Research FellowHannah Buckley, Statistician

York Trials Unit, University of York

Page 2: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

Co-authors: Ainsworth, H., Heaps, C., Hewitt, C., Jefferson, L., Torgerson, C., & Torgerson, D.

Funded by: Education Endowment Foundation

Intervention delivery partner: Calderdale Excellence Partnership

Page 3: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

• Protocol available at:http://educationendowmentfoundation.org.uk/uploads/

pdf/Transitions_-_Calderdale.pdf

• Full report available at: http://educationendowmentfoundation.org.uk/uploads/

pdf/EEF_Evaluation_Report_-_Improving_Writing_Quality_-_May_2014_v2.pdf

Page 4: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

Overview

• EEF Transitions Round: • Evaluate 3 different writing interventions with

aim to improve children’s writing skills during transition from primary school to secondary school

• Individually Randomised (Discover)

• Split Plot Design (Exeter)

• Cluster Randomised (Calderdale)

Page 5: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

Background

• Many children leave primary school without achieving a Level 4 or above in writing (Dept of Education, 2013)

• The intervention was aimed particularly at children:– from disadvantaged backgrounds – not achieving a ‘secure Level 4’ in year 6

• Not reaching Level 4 at the end of KS2 linked to progress in secondary school (Dept of Education, 2014)

Page 6: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

• Using memorable experiences and the Self-Regulated Strategy Development (SRSD) programme.

• SRSD strategy developed in America

• Writing process model involves:• Plan• Draft• Edit • Revise

• Key aspects include self monitoring & goal setting to provide pupils with ownership over improving their writing.

Background

Page 7: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

What is the effectiveness of the Improving Writing Quality programme compared with “business as usual” on the writing skills of

participating children?

Research Question

Page 8: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

• Pragmatic cluster randomised controlled trial.

• Approximately 24 schools (feeding into 3 secondary schools) randomised on a 1:1 ratio to intervention and control groups

Design

Page 9: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

Intervention Group •The intervention was offered to all pupils in Year 6 in 2013.•Pupils predicted to achieve Level 3 or an insecure Level 4 in English (based on teacher assessment) continued to receive intervention in Year 7 (Autumn term).•Memorable experiences for pupils in Year 6.•Pupils used SRSD programme to embed these experiences into their writing.•Professional development for primary and secondary school teachers in key elements of the writing intervention SRSD programme (including discussion, modelling and planning).

Control Group•Business as Usual in 2013.•Primary schools trained in SRSD in 2014.

Design

Page 10: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

• Hosted a information meeting to explain intervention and evaluation, hosted by delivery partner and evaluators in February 2013.

• Recruited: 23 primary schools; 3 secondary schools.

• Randomised primary schools in March 2013.

• Post test conducted in December 2013 after transition to secondary school.

Design

Page 11: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

Outcome measure

•Progress in English (PiE) 11: Second Edition Long Form (LF) Test, GL Assessment

Primary outcome

•Combined score on extended writing tasks

Secondary outcome

•Combined score on reading tasks and combined score on the spelling and grammar tasks

Outcomes

Page 12: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

• Cluster trial

• Pupils from same primary school transitioning to different secondary schools

• Cross-classified multilevel model used to account for this

• Adjustment was made for baseline predicted KS2 writing level alongside other predictors of attainment (gender, FSM status, EAL status, month of birth)

• ITT

Analysis Methods

Page 13: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

• 209 pupils meeting inclusion criteria randomised to control and 223 to intervention

• Following transition 130 control and 154 intervention pupils

• Primary analysis conducted on 119 control and 142 intervention pupils

Results

Page 14: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

• A significant increase of 2.53 marks in writing score (p=0.002, ES=0.74) was found for struggling writers in the intervention group compared with the control group.

• Although this increase was larger when analysis was conducted on the FSM subgroup, there was no significant difference between FSM and non-FSM children.

• There was no evidence of a difference between the randomised groups in reading or spelling and grammar score.

Results

Page 15: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

Strengths•Study conducted to CONSORT standards.•Used blind marking of test papers.•Good relationship between CEP & schools therefore had high completion.•Analysis plan produced a priori.•Analysis accounted for clustering.•Followed principles of intention to treat.

Strengths & Limitations

Page 16: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

Limitations•During the trial there was ‘intervention development’.•Anglicising the intervention. •Refresher training to secondary schools incorporated ‘lessons learnt’ and adaptations.•Imperfect implementation in secondary schools.•Based in single geographical area.

Strengths & Limitations

Page 17: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

• This intervention appears to be highly effective at improving children’s writing skills despite being used for a relatively short period of time.

• No statistically significant difference found between FSM and non-FSM children.

Conclusions

Page 18: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

• Larger-scale effectiveness RCT of the intervention to confirm findings of this trial.

• Multiple geographical areas to be included.

• Developing an ‘Anglicised’ version for use in primary schools.

What next?

Page 19: A randomised controlled trial to improve writing quality during the transition between primary and secondary school Natasha Mitchell, Research Fellow Hannah

Questions?

Thank you