Ascend and IOE Partnership Evaluating impact: how do you know you are making a difference? Sue Hellman, David Godfrey and Sarah Seleznyov London Centre

Ascend and IOE PartnershipEvaluating impact: how do you know you are making a difference?

Sue Hellman, David Godfrey and Sarah SeleznyovLondon Centre for Leadership in LearningInstitute of Education www.ioe.ac.uk/lcll

• To deepen your understanding of impact framework and how to use this to demonstrate impact in relation to the aims of the Ascend project.

• To provide an opportunity for you to refine your impact frameworks having compared your impact frameworks to a sample framework

• To hear about responses to the lesson study so far

• To explore explicit links between lesson study, subject audits and impact framework

• Improve the quality of teachers’ subject specific knowledge

• Improve the quality of teaching

• Raise attainment and accelerate progress for pupils with FSM, underachieving and socially disadvantaged pupils in Maths, English and Science across Key stages 1-3

Impact Framework

Survey Responses

Lesson Study

Pupil Learning

Outcomes

Guskey’s levels

initial reactions

professional learning

organization support and change

pupil learning

outcomes

use of new

knowledge and skills

Input model: Focus on what we do,

what we offer

Impact model:

Focus on the difference we want to achieve

© Copyright LCLL, 2008

What is impact evaluation?

From what ………. to what?


How will you know?

Baseline picture• Current practice• Current data • Current attainment

measure

Impact picture• Changed practice• Changed data• Changed attainment

measure

‘…so that …’

Pupils’ learning and outcomes improve…

What will I be feeling / thinking?

What will I be saying?

What am I doing?

What will I be doing?

What am I feeling / thinking?

What am I saying?

© Copyright LCLL 2008

Baseline – Current practice Impact – Shift in practice

Evidence – How do you know?


Teacher Practice

Baseline Impact

I never sit with the lower ability children.

There will be problem solving activities for all children.

Lesson observations show that I never sit with a group for more than 10 minutes at a time because I move around the room responding to low ability children who have raised their hands for help.

I will work with a different group each day.

I set problem solving extension tasks for the most able pupils.

Planning scrutinies show that I work with a different ability group each day. Lesson observations show that I sit with a group for at least 20 minutes. Fewer lower ability children raise their hands and evidence that they use the problem strategies independently.

What will pupils be feeling / thinking?

What will pupils be saying?

What are pupils doing?

What will pupils be doing?

What are pupils feeling / thinking?

What are pupils

saying?

© Copyright LCLL 2008

Baseline – Pupils’ learning Impact – Shift in practice

Evidence – How do you know?


Pupils’ Learning

Baseline Impact

Most children struggle to stay engaged and on-task during lessons.

The target group will add more descriptive detail to their writing.

Lesson observations show that around 30% of the children drift into off-task talk after about 10 minutes of work time and around 4 or 5 children in any given lesson will get up and wander around the room. This means work outputs are too low for around 70% of the class.

Children will remain engaged and focused throughout the lesson.

The target group write mainly simple sentences.

Lesson observations will show that no children wander around the room and that 90% of the children remain on task and focused for the full 20 minutes of independent work time.

Working in pairs:

Looking at the sample Impact Frameworks

- How does the sample Framework compare with your own?

- What will you need to do to further refine and develop your own Impact Framework?

Working in pairs:

Take one person’s Impact Framework at a time.

You have 15 minutes to refine and develop the Framework in line with our discussions.

Impact Framework

Survey Responses

Lesson Study

Pupil Learning

Outcomes

How does this affect:• the focus of the lesson study?• the role of the observer?

Ascend and IOE Partnership

Feedback from lesson study first cycle and subject knowledge audits

David GodfreyLondon Centre for Leadership in LearningInstitute of Education www.ioe.ac.uk/lcll

Pupil learningoutcomes

• Surveys for Ascend project:

• Subject knowledge audits:

• All participants to complete for either English or Mathematics

• Post lesson-study survey:

• All participants to complete once as focus teacher and for each time as observers (non-focus teacher)



• More responses please!• Mathematics audit relies on NCETM categories and is about self-confidence• English audit tests participant knowledge but is also designed to highlight

areas where teacher may be more/less confident• Both are tools for self-evaluation and reflection on subject knowledge in

relation to areas you teach and key stage (particularly in relation to target students)

• Lesson study evaluation:

• Two versions:

• Focus teacher• Non-focus teacher (baseline and post-lesson evidence section omitted)

• Sections:

• Baseline data• Evidence of changed practice and pupil outcomes captured from lesson study

cycle• Key professional learning, challenges and future strategies • Participant reactions to the process


• Focus teacher reactions to the process:


• Non- focus teacher reactions to the process:


• Baseline data:• Use impact frameworks to fill this out – can copy and paste from

word document into survey• Student numbers: use teacher number plus student letter (e.g.

AL001a, AL001b)• Add key baseline data – combination of impact framework and first

lesson of cycle• Add observations from lesson study about target pupils especially

(soft data)


• Key professional learning, challenges and future strategies:

• Plenty of comments about insights into children’s learning and effectiveness of approaches

• Need to make specific comments that relate to target students• Some comments about the challenges of effective planning (making

time) and observation•

•To sum up…

Relate lesson study evaluation to details of impact frameworkFocus observations on target students (use identifier codes)Focus on strategies that directly or indirectly (e.g. through your own professional development needs) will lead to improved outcomes for target students

Ascend and IOE Partnership

Feedback from lesson study first cycle and subject knowledge audits

David GodfreyLondon Centre for Leadership in LearningInstitute of Education www.ioe.ac.uk/lcll

Impact Framework

Survey Responses

Lesson Study

Pupil Learning

Outcomes

Ascend and IOE PartnershipEvaluating impact: how do you know you are making a difference?

Sue HellmanLondon Centre for Leadership in LearningInstitute of Education www.ioe.ac.uk/lcll

[email protected]

Documents

Ascend and IOE Partnership Evaluating impact: how do you know you are making a difference? Sue Hellman, David Godfrey and Sarah Seleznyov London Centre