Predictive Modelling Round Table w w w .lo c a l.g o v .u ... · modelling approach help us achieve? Working in small teams to gain the advantage of all perspectives in the room Slides

Care and Health Improvement programmewww.local.gov.uk/chip Care and Health Improvement programme

Predictive Modelling Round Table

Output Pack17th June, 2019

Care and Health Improvement programmewww.local.gov.uk/chip

Section 1: Overview


Introduction

• On 17th June 2019, the LGA ran a workshop attended by 35 stakeholders

from across the country, who represented:

– Local Authority Commissioners

– Local Authority employees with specialist skills in a wide range of area including;

finance, data modelling and reporting and performance.

– Other local, regional and national stakeholders

• This workshop was facilitated by Doug Sheperdigian (Atlantic Customer

Solutions), who worked with LGA to design the structure and content

• This document, as with the workshop, is in 4 sections:

– Overview: Introduction and set up for the day

– Benefits: What would a predictive model help us achieve?

– Features: What are the components of a good predictive model?

– Next Steps: The pilot activity and final approach


Guide to work through this document

• The outputs from the workshop have been set out in this document in a way

that can be read and understood by anyone, with the following caveats:

– Outputs will make most sense to those who attended the workshop and / or have

some understanding of this work. If in doubt, check with LGA colleagues

• There are three types of slide in this document:

– Presentation given at the workshop – Slides with red headings

– Guidance and instruction at the workshop – Slides with blue headings

– Information produced by at or before the workshop – Slides with green headings

– Slides that provide a narrative for this document – Slides with black headings

• In each section, you will find the outputs as produced on the day.

– Posters with added post-it notes, comments and “likes” in the form of hearts.

– Printed text on feedback posters is the output from interviews with key

stakeholders conducted in advance of the workshop

– Each section includes a brief summary of the key outputs

– Hearts are colour coded to indicate a perspective


Workshop AgendaTime Topic Who Slide Pack Ref.

10.30 Registration and Coffee

10.45 Safety Briefing Brigid Day

10.50 Welcome Hazel Summers – Chair of Project Steering

Group

11.00 Ground rules Doug Sheperdigian – Facilitator

11.05 Who we are (Introductions) Facilitated session

11.15 Where are we now? Brigid Day and Rachel Carter Slides 8 – 11

11.25 Benefits – What should a good predictive

modelling approach help us achieve?

Working in small teams to gain the advantage of

all perspectives in the room

Slides 13 – 15

11.45 Report back Headlines from each team Slides 16 – 24

12.00 Features – what are the key components of

a good predictive modelling approach?

Working in small teams to gain the advantage of


Slides 25 – 27

12.30 Report Back Headlines from each team Slides 28 – 38

12.45 Individual inputs Capture views

13.00 Working Lunch Activities will continue!

13.30 Intro to Next steps Leon Goddard Slides 39 – 42

13.40 Next steps and pilot input Working in small teams to gain the advantage of


Slide 43

14.15 Report Back Headlines from each team Slides 44 – 53

14.45 Closing remarks and feedback Doug Sheperdigian Hazel Summers Slides 54 – 56

15.00 Finish

15.10 Market Analysis Philippa Lynch

15.30 Informal Networking (unstructured)


Start to the workshop

• There was a warm welcome from Hazel Summers, who described what she

hoped to get out of today’s session

• Doug Sheperdigian, our facilitator, then explained how he would run this

session in order to get the most value from it.

• Doug also asked for permission to manage everyone’s time and the group

were happy to agree to this, some by nodding their head, others did so

telepathically.

• Brigid Day and Rachel Carter then gave an update on the project to date …


Aims and benefits of overall project

To build on successful work progressed by some regions and councils to better

understand local care markets so councils can:

• Have access to comparable consistent market supply and demand

information within and across councils/regions

• Identify and manage better adult social care market risks

• Use tools to aid commissioners planning to meet demand and assist with

budget planning

• Improve commissioning plans and strategies


Some national contexts

Increasing focus on commissioning ……

• CQC area system review findings – more planned

• Continuing major funding and market supply and sustainability challenges

• Green Paper ?????!

• NHS interest/joint working on social care commissioning


Project Stages – proof of concept

1. Collection of dataset to create a market analysis tool on LG Inform Plus

enabling analysis and comparison within and between councils and

regions

2. Develop predictive modelling tools and support, to inform and enable

modelling of future markets at local, regional and national levels

3. Bespoke support to councils and regions to develop/refine their

commissioning plans & Market Position Statements

• 80 councils engaged

• Built on previous ADASS regional work

Co-designed data collection

82 councils submitted

Councils submit

data

Market Analysis Tools developed

Next steps

• Target June

• Show & Tell national findings to regions

First tools

available

• Are tools useful?

• Consider next steps

Review

• Forecast demand

• Understand options

• Decide markets needed?

Define aims

Models developed by councils and regions

Research

Workshop to plan coproduction approach

Options appraisal

report

Pilot phase across regions

Share progress

NEXT STEPS

Evaluation

Plan and agree future…

NCAS Conference

Support offers continue

Where we are now

Predictive

Modelling

Market

Analysis

Sept to Nov May to June July Nov- Jan

2018 → 2019 → 2020

Broad Predictive Model aims•Local focus - gold standard framework

for modelling with facility for local

‘deep dives’

•National focus - facilitating standard

tools to help councils & regions &

make national case for funding

Option 1

• No change

• No further action

Option 2

• Scale up one existing model

Option 3

• Combine best from council models

Option 4

• Develop a new model


Predictive Modelling – 3 key areas

Section 2:

Benefits

Section 3:

Features

Section 4:

Next Steps

Slides:

13 – 24

Slides:

39 – 53

Slides:

25 – 38


Section 2: Benefits

What should a good predictive

model help us to achieve?


Benefits Roundtable

Jupiter: Regionally

Mercury: Locally Venus: Nationally

Pluto: Market Shaping

and TransformationSaturn: Finance

Neptune: Market

Sustainability


What should a good predictive model

help us to achieve?

• Market Shaping and Transformation

– Achieving the right type, level and location of provision.

– Could seek gradual improvement or transformation

• Market Sustainability

– Matching provision with capacity and avoiding provider / market failure

• Finance

– Using financial modelling to make better projections and use of resources

• Locally

– Helping authorities, stakeholders and organisations in local areas

• Regionally

– Supporting cross boundary planning, commissioning and delivery

• Nationally

– Supporting the national policy agenda and financial support


Benefits Report Back


Pluto: Market Shaping


Neptune: Market Sustainability


Saturn: Finance


Mercury: Locally


Jupiter: Regionally


Venus: Nationally


Supports

opportunity for

regional

collaborative

commissioning

Useful for small

unitaries and

provide evidence

base for planning

at sub-regional

basis

Inform

regional SLI

programme

Future ability to

drill down into

granular to

identify best

practices/

service models/

outcomes

Accept the

differences for

different type of

care models e.g.

sti/ extra case may

require 10-15 years

Granularity

of data to

change

model/ trend

Provide key

intelligence to

inform STP/ICS

regional strategic

partnerships

Does this relate to

market oversight

and contingency

planning… Not

sure

Prompt

consistent

data

definitionsConsider

issues at the

boundaries of

the regions

e.g SW

London and

Surrey

Myth

busting

at a

local

level

Include sub-

regional as

markets are

often at that

level

Enables

“what if”

modelling

Encourage

s working

across LA

boundaries

Doesn’t need

to be 100%

correct, just

better than

now.

Inform

and

monitor

Regional gives

better

population

evidence and

see trends

Caution around

inappropriate

usage of

outputs

Do the

model/struct

ure once

not x no.

times

Cross boundary

approach

minimises cross

boundary impacts

where one LA

does something

affecting another

Sharing best

practice,

more easily

Support

working at

ICS/CCG

landscape

Core data

can be used

@

regional/sub

regional and

local level in

local

systems i.e.

P.A.M.M.S

Identify gaps

to meet short/

medium/long

term

Locally Regionally Nationally

This page contains ‘difficult to read post-its’ from the 3 previous slides

Locally, regionally and nationally – cont.


Benefits: Summary & Initial Analysis

• The workshop teams identified

and prioritised 105 Benefits*

categorised by:

– Locally – 45

– Regionally – 41

– Nationally – 19

• A database has been prepared

as a separate output, capturing

all benefits.

*Further work can be done to cluster and de-duplicate these

Top 5 Local Benefits

No. Scope Comments

L3 Local Better commissioning

L1 Local Better market position statements

L5 Local Long-term signalling to market

L4 Local Better understanding of alternative

L15 Local Commissioners understand their role in changing demand

Top 5 National Benefits

N3 National Able to support requirement for more funding with Treasury

N1 National (Robust) Evidence based dialogue with DHSC

N12 National Potential to consider all ages in the analysis

N2 National A means to align ADASS and LGA in national dialogues

N4 National Potential to analyse Self funders

Top 5 Regional Benefits

R3 Regional Able to benchmark

R5 Regional Improve ability to deliver 10 year plans

R4 Regional Support regional scorecards

R32 Regional Understand current position + trends and movements

R7 Regional Prompt consistent data definitions

*Further work can be done to cluster

and de-duplicate these


Section 3: Features

What are the components of a

good predictive modelling

approach?


Benefits Roundtable

Jupiter:

Communications

Mercury: Infrastructure Venus: Outputs

Pluto: Information Saturn: Technical

Neptune: Resources


What are the key components of a

good predictive modelling approach?

• Information

– What data, information and evidence must be included in the approach?

• Resources

– What type of resources (people, IT etc) are required and what are the costs?

• Technical

– What analysis, projections and programmes must support the approach?

• Infrastructure

– What needs to happen to ensure the right organisational structures, cultures

and approaches exist to make best use of this work?

• Communications

– What should we explain about this work, about how key stakeholders can be

involved in the development and about the approach and how it works?

• Outputs

– What must the model predict? What are the outputs


Features Report Back


Pluto: Information

What do first people then cost

First people data

People at front door + self funders

Standardsnational

Key question: how many people in system- Now- 2-3 yrs

Target audience, which customer

Risk safeguarding data

What is starting point? Whole population or social care population

Housing Data

Stress test End of

life

Length of stay reducing

HES data linked to death registration data

Self funder info helps care home risk

Inputs Technical Aspects Outputs

This page represents the post-its from the

previous page – transcribed for legibility

Pluto: Information cont.


Neptune: Resources

Governance e.g. across housing and ASC

Good leadership and conceptual maturity across all participansgroups

Wider applications and use of data beyond ASC

Issue of integrating data.

Actuals vs predictionse.g. impact in 5yrs will dictate monitoring

Has to be understandable approach

Frequency depends on user group/ service

What it does:Step to inform planning – not a panacea

Sophisticated vs solid base for a model

Web based tool vs excel

Extraction of data

Repeat users

Age of entry

Average time in provision

3 years*

Housing data

Performance data support- Local- national

Provider input

Children’s and transitions data

Example of information about informal carers.- What info needed which can influence

ICT- Local- national

Skill mix of staff team –“no one is an expert in everything”

Inputs Technical Aspects Outputs

This page represents the post-its from the

previous page – transcribed for legibility

Neptune: Resources cont.


Ability to bring

existing data

(from various

sources) into the

model, with an

agreed core

dataset.

Unique identifier

e.g. core

location I.D.

In all registered

managers to

keep up to date

Regular

(monthly)

data inputs

locally

Automated

collection

and delivery

with the

system

What if

planning.

“What if

scenarios”

Saturn: Technical


Self

funders

running out

of assets

Commercialisation

– Sell information

Mercury: Infrastructure


Jupiter: Communications


Elephant

Clarity of definitions of

data.

Keep it simple and

linked to recognised

categories

Who are the

audience for this –

internally –

leadership team

and members

Co produce with

providers + public

solutions/

transformations to

alter trajectory

Risk of providers

exiting the market

permanentlyHow to get national

implementation

- ongoing comms +

engagement

Share resource

available to

providers to

consider markets

too – open data

HOW:

Markets <-> predictive

Data <-> modelling

Yes but is the

proposed

model going

to do this?

Key, simple info about

expenditure to help

communicate with the

public.

Inputs Technical Outputs

Need to establish if this: -is only a forecasting tool or if it is also real time monitoring -incorporate provider info/links to care market data.

Jupiter: Communications – cont.

This page contains ‘difficult to read post-its’ from the Jupiter slide


Need to be able to

drill down to

understand “actual”

vs “forecast”

Cost

- Volume

- Complexity

- Market

prices

Workforce

- Volume

- Skill mix

- Supply

mix

Venus: Outputs


Features: Summary & Initial Analysis

• The workshop teams identified

and prioritised 168 Benefits*

categorised by:

– Inputs – 77

– Technical – 54

– Outputs – 37

• A database has been prepared

as a separate output, capturing

all benefits.

Top 5 Inputs

Top 5 Outputs and Key Features

Top 5 Technical Aspects

*Further work can be done to cluster

and de-duplicate these

No. Scope Comments

I1 Inputs

POPPI/PANSI historical volumes and costs (2 years)

deprivation

I2 Inputs Workforce avaliability

I4 Inputs Inflation estimate

I5 Inputs Integrated with Care IT systems (all of them)

I3 Inputs Land costs

T1 Technical

Transparent mathematics (so that we can explain) - "must not

be a dark art"

T2 Technical Local judgement applied

T3 Technical Statistical techniques (e.g. logistical regression)

T5 Technical Core set of data -consistent

T5 Technical AI/machine learning

O1 Outputs

Future net change in demand by service area (e.g. OP - res care) - new service users -leaving the

service -total service users -level of need (complexity) -"going beyond simple demographics"

O2 OutputsMonitoring actuals versus prediction on a [real time, monthly] basis

O3 Outputs 5 year projection of demand and costs

O4 Outputs Cost (budget prediction) by service area "going beyond simple inflation"

O5 Outputs Communicates to laypeople and non - analysts


Section 4: Next Steps

Pilot Activity and Final Approach


Next Steps – Pilot Activity

Build on current knowledge and information

Seek to support decisions and design around final approach

1. Purpose and objectives

– What are we trying to understand and learn from the pilot activity

2. Design and development

– What type of activity will be undertaken and what areas will it focus on?

3. When and Who

– When should the pilot happen? Who should be involved? How authorities will be

selected? What approaches should be piloted?


Next Steps – Final Approach

3 broad options that sit along a continuum

All options will be co-produced and based on learning and best practice

1. Single Model

– Complete and comprehensive model

– Expectation is that all authorities will use this model

2. Agreed Principles

– Key principles agreed by stakeholders

– Authorities to design their own approach, that follows these principles

3. Toolkit

– Include documents, specifications, guidance, hints and tips

– Authorities free to use what they want to deliver the approach they want


Individual Exercise – What point do you

see on the continuum?

What is the Best approach?

What is most practical and achievable?

What will Actually Happen?

One Model ToolkitAgreed Principles


Each Team has a Specific Assignment

Jupiter:

Option 1 –

Single Model

Mercury:

Option 2 – Agreed Principles

Final Approach – Please describe what you think

are the pros and cons of the different final

approaches – please state your preferred approach

Pilot – Please describe what you think any pilot

activity should look like; Use a flipchart to capture

your team’s views.

Venus:

Option 3 – Toolkit

Pluto: Purpose

and objectives

Neptune:

Design and

development

Saturn: When

and Who


Next Steps Report Back


Pluto: Purpose and objectives


Pluto: Purpose and objectives cont.


Neptune: Design and development


Saturn: When and who


Maximum benefit- National- Regional

Matching a national/ Regional NHS

Balancing whole population analysis vs those who use council funded services

Jupiter: Option 1 One Model


“One Model” Pros and Cons

Pros• Balancing whole population analysis vs

those who use council funded services

• Maximum benefit

- National

- Regional

• Matching a national/ Regional NHS

• Build on best practice

• Economies of scale (limited resources)

• Consistency at a national level

• Single narrative

• National benefits realised

• The only one that works

• Benchmark (value of)

• National evidence to influence policy development

Cons

• One size may not fit all

• but it does really!

• It might hurt a bit

• Input data is restricted to what is

available across all LAs


Mercury: Option 2 Agreed Principles


Venus: Option 3 Toolkit


Red=Regional

Yellow=National

Green= Technical, Finance

Blue=Commissioners

We asked individuals

to place a “like” in

answer to 3 questions:

• The majority chose the

“one model” approach.

• And, it was seen as

highly practical…

• However, the most likely

outcome attracted a wide

range of opinion.


Final Discussion and Next

Steps


We finished with a plenary discussion

• “Has opened discussion on scope”

• “Opened discussion for consensus and agreed way

forward”

• “More sizeable than initial thought”

• ”Narrow down focus and get focus agreed”

• “Last time real tension between local and national

purposes. Now it seems more aligned”

• “People want simple answer to complex questions”

• “Toolkit idea was “ok” but need best of both bits”

• “Need a consistent data set, clear on what we’re using it

for”

• “Not as simple as “one model”

• “Everyone’s at different points”

• “We’ve not separated starting points and assumptions,

weve looked at assumptions today but what’s the

starting point.?

• “We need to define what’s generating that percentage”

“What are the “process” parts of the model – what data

we want, what data are we putting it in, what are we

multiplying it 2by”

• The further up the food chain in the system, the more

inaccurate the data”

• “The starting point matters”

• “Potentially choose something that already exists rather

than trying to reinvent the wheel”

• “Use this as the starting point – test it – try it – analyse

it”

• “Consider what Alder have already done.”

• “Only at start of the process but thinking about

integration and health” What does this actually mean as

a bottom line to hospitals for beds etc…”

• “Considerations of different pressure scenarios included

in the model would help with this application link and

bridging that gap”

• “When we visited councils and regions we saw good

practices. Model is a puzzle, with different pieces, but

from different councils. Others had better bits than

others – call to effect etc. Principles more than

practicals – we already have enough to work with – don’t

need to start from scratch.”

• “What’s acceptable in terms of a model? – don’t want to

have to be polishing it forever”

• “Accurate usable meaningful working robust ease of use

– to develop credibility going forward”

• “Pilot has potential to fail quickly and allow us to move

on – pilot authorities may not be total representation of

all the potential participants”

• “Never going to be completely accurate but would be

happy with 80% good?? Way better than now!”

• “Find the right scale and ambition from the beginning

and not try to do too much too quickly”

• “Don’t run before you can walk but still aim high”

Care and Health Improvement programmewww.local.gov.uk/chip Care and Health Improvement programmewww.local.gov.uk/chip

Hazel brought us to a close, indicating

some proposed next steps

Pilots

• Identify at least one council per ADASS region to serve as a pilot

• Design and fund the pilot programme

• Deliver pilots (test different scope and complexity)

Communication

• Report back to working group from the Workshop

• Deliver a workshop at NCASC (November)

Governance

• Refresh the reference group and include predictive modelling as a key term

of reference.

Care and Health Improvement programmewww.local.gov.uk/chip Care and Health Improvement programme

Key Contacts:

Leon Goddard

Senior Adviser CHIP/LGA

Commissioning and Markets

07557 214985

[email protected]

Rachel Carter

Adviser CHIP/LGA

Commissioning and Markets

07827 083 942

[email protected]

For more about CHIP work on commissioning see

Commissioning and market shaping | Local

Government Association

mailto:[email protected]

mailto:[email protected]

https://www.local.gov.uk/our-support/our-improvement-offer/care-and-health-improvement/commissioning-and-market-shaping

Documents

Predictive Modelling Round Table w w w .lo c a l.g o v .u ... · modelling approach help us achieve? Working in small teams to gain the advantage of all perspectives in the room Slides