Evaluating health and social care interventions in a CCG

Jo Broadbent

Director of Integration & Innovation, NHS NEE CCG

Consultant in Public Health, Essex County Council

Overview

1. Why evaluate at a local service level?

2. Case study 1 – Evaluation of Stroke Pathway Improvements

3. Case study 2 – MDT Case Management Evaluation (“Virtual Wards”)

4. A note on Information Governance

5. Challenges and Pitfalls

6. Conclusion

Why evaluate at a local service level?• “Integration Fever”

– Lack of evidence of impact of integrated health & social care interventions on quality, demand and cost

• “Innovation Fever” / Reality Check– How robust and how generalisable are ‘best practice’ results?

• Economic challenges & Value for Money– Stewardship of limited public finances– Informing monitoring of ongoing cost / effectiveness

• Managing expectations– Increasing gap between public & professional expectation vs.

effectiveness & affordability– Scepticism & making the case for change: “Can it work here?”

• Pragmatism – Short-term funding and Pilot-itis

CASE STUDY 1 – EVALUATION OF STROKE PATHWAY

IMPROVEMENTS IN NE ESSEX

HYPOTHESIS – IMPROVED CLINICAL STROKE CARE MAKES SAVINGS TO

SOCIAL CARE

Overview of Stroke in NE Essex

• c. 600 acute stroke admissions p.a. in NEE; average admission cost £5,032 (Essex-wide cost; SUS)

• c. 17% stroke mortality rate across Essex (SUS)• c. 170 high risk TIAs treated p.a. in NEE (SUS)• No. known stroke / TIA survivors = 6,625 (2.0% of population; QOF

2011/12)• 33-bed hyper/acute stroke unit with 10 rehab beds in Kate Grant

Unit, Clacton Hospital• c. 55% of patients have Early Support Discharge (ESD / rehab at

home) in their own home• Post-stroke social care costs an average of £18,458 p.a. (Essex

County Council)

Milestones in Pathway Development

2007/8 Developing acute care & introduction of thrombolysis2008/9 AF screening introduced at Flu Clinics2009/10 Primary Care Local Enhanced Service introduced – to increase diagnosis

and improve management of cardiovascular conditions

March 2010 “As is” pathway mapping workshop with clinicians, patients & carersMay 2010 Patient Listening Exercise

Essex Cardiac & Stroke Network Peer Review of RehabPathway redesign workshop with clinicians, patient reps, voluntary sector & commissioners

September 2010 ESD Procurement started

March 2011 GP TIA awareness training

2011/12 Multi-provider Joint Care Planning CQUIN

October 2011 ESD Service admits first patients

December 2011 Decommissioning of rehab beds in acute sector

February 2012 Process to switch grant-funded Voluntary Sector Patient & Carer Support to commissioned basis starts

Befriending for post-stroke patients funded

2012/13 Midlands and East Stroke Review

May 2012 Extended access to ESD access to >50% by lowering clinical thresholds

January 2013 Commissioned Life After Stroke voluntary sector service replaces grant-funded services

Stroke Spells in NE Essex (SUS)

Linking Health & Social Care Data

SAFE HAVEN

Data Manipulation and Cleaning

Specific social care records were attributed to a stroke using the following rules:

•A social care package that was active at the date of discharge – length of stay (i.e. date of admission) was classed as a “pre-stroke package”. This was so that we could estimate potential increases in packages attributable to the stroke.•A re-ablement package that was started within 5 days of discharge or whilst the patient was still in hospital was attributed to the stroke.•Any other package that was stared within 5 days of discharge or whilst the patient was still in hospital was attributed to the stroke.•Any other package that was stared within 7 weeks of discharge was attributed to the stroke.

Impact - Quality of Patient Care

  2009 2012

Patients diagnosed with AF 5,409 6,423

Spending 90% of stay on stroke unit 61% 79%

High risk TIAs treated in 24 hours 25% 96%

Stroke HSMR 108 <80

Patients supported by Early Supported Discharge

0 60%

Impact - Pathway Efficiency

  Oct 2010-Sept 2011

(n=629)

Oct 2011–Sept 2012

(n=510)No. Rehab Beds in System 20 10Average Annual Cost of Social Care Package

 - £18,458

% patients receiving re-ablement 4.3% 1.0%% patients receiving new or amended post-stroke domiciliary care package

1.9% 0.8%

% patients newly entering Residential or Nursing Care

2.7% 0.2%

What was the impact of increasing ESD in NEE?

• A 75% reduction in need for social care packages post-stroke in NEE following the introduction of ESD

• ESD-attributable reduction in social care demand estimated at 57%

  Oct 10 – Sept 11 Oct 11 – Sept 12 DifferenceNo. Stroke patients 629 510 119 / -18.9%% Stroke Patients receiving ESD

0% 47% +47%

Baseline No. / % (95% CI) Patients receiving Social Care Package

56 / 8.9% (6.9-11.4%) 14 / 2.7% (1.6-4.6%) -42 / -75%

Upper Estimate No. / % Patients receiving Social Care Package

93 / 14.8% (12.2-17.8%) 23 / 4.6% (3.0-6.7%) -70 / -75%

       Reduction in Packages Due to -

    No. /% of the observed reductions -

Reduced Incidence of Stroke   8 / 18.9%

Reduced Need / Supply     10 / 23.8%ESD     24 / 57.3%

So What?• A 57% reduction in post-stroke social care demand, due

to ESD, could save:• £172k - £572k pa in NE Essex• £977k - £3,243k pa across Essex• Essex County Council agreed to use s256 Sustainability

funding for dedicated social workers for each acute stroke unit & stroke pathway in Essex, to facilitate timely discharge and support for domiciliary ESD

• A case for using the Better Care Fund to fund improved stroke care?

CASE STUDY 2 – MULTI-DISCIPLINARY CASE

MANAGEMENT EVALUATION

HYPOTHESIS - COMMUNITY “VIRTUAL WARDS” IMPROVE CARE QUALITY BUT REDUCE DEMAND ON HEALTH & SOCIAL CARE SERVICES

What is a Community “Virtual Ward”?Multi-agency, integrated assessment and case management

Offered to –

Patients with multiple, complex needs including long term health conditions

Staffed by –

A team or ‘ward’ of nurses working closely with a patient's GP and other health, social care and voluntary sector staff.

Core team: GP, community matron, social worker, ward clerk

Wider support: social care, voluntary sector, district council, COTE consultant, Specialist Community Nurses

Pilot Virtual Wards were first established in the Clacton area in January 2011, covering 9 GP practices.

Evaluation period = 12 months

Who is Admitted to the Community Virtual Ward?

Aims of the Virtual Ward ProjectDoH LTC QIPP workstream aimed to: “reduce unscheduled hospital admissions by 20%, reduce length of stay by 25% and maximise the number of people controlling their own health through the use of supported care planning.”

NE Essex aims:“to enhance care for people with multiple long term conditions and/or those at high risk of acute admission in NE Essex, through –•building on the current clinical case management role of Community Nurses;•using risk prediction software to increasingly focus activity on preventing exacerbation and avoidable acute admissions;•admitting high risk patients to “Virtual Wards” run by Clinical Case Managers;•establishing multi-disciplinary teams (MDT) across health and social care, which link to Local Authority and voluntary sector services, and have Case Manager at centre;•Case Managers directly drawing support for patients from MDT, Local Authorities and Voluntary Sector.”

Evaluation Objectives(after Maxwell)

Effectiveness

Service usage and LTC risk factor management. This will address –•Quality•Adding value•Reduction in unscheduled admissions

Cost-Effectiveness

Additional costs incurred by the Virtual Wards will be collated. These will be compared with costs/savings from change in service usage and impact of risk factor control changes. This will address –•Sustainability•Delivering savings (QIPP)

Acceptability / Experience

A before and after survey of the experience of patients, family/carers, practice staff and virtual ward staff will be undertaken. This will address –•Quality•Adding value•Effectiveness of joint teams•Empowerment of patients•Sustainability

Appropriateness•For NE Stakeholders•For commissioners to decide if and how to expand the service

Impact – Improved Patient Experience

“If I didn’t have this help to look after me … I’d have to go in a home. Which I don’t want to do because you get so lonely.”

“Excellent idea having Community Matrons. Community Matron is able to pull thing together and liaise with everyone.”

“Myself and family feel this is an excellent service and since being on the ward have not been in hospital”

“When Community Matron got involved everything went smoothly”

“Very happy with Community Matron who is kind and explains things to me”

Impact – High Quality of LTC Care

• 72% of GPs and 100% of Community Matrons report that virtual wards have improved LTC patient care

• High quality of care found by clinical audit (individual level QOF-based vs wider population)

Fewer AVOIDABLE ambulatory care sensitive (ACS) hospital admissions – through better Community-based care

Impact - Reducing Avoidable Admissions

19% reduction in ACS admission rate = 20 fewer admissions / 1000 patients pa. = saving of c. £58,000 pa. per 1,000 patients case managed

40% significant reduction in ACS bed days = 400 bed days / 1000 patients pa

Impact - Reducing Avoidable Admissions (System-Level)

Rate of increase in all NEL admissions for all >65s was three-fold lower at GP practice level in practices supported by a VW than in all other GP practices in Tendring:

All NEL admission rate –

VW Practices = 4.73% increase vs non-VW = 14.11%

ACS Admission rate –

VW Practices = 20.66% increase vs non-VW = 28.42%

Addressing Unmet Social Care Needs / Early Social Care Intervention

•20% of virtual ward patients have been found to have unmet social care needs and are now receiving small preventive packages of care

– eg next day initiation of domiciliary care (eg personal care, meals and medication support), home and furniture adaptations, and benefits assessment and advice.

•45% of contacts involved some form of carers’ provision (either advice or services)

Reducing Social Care Package Cost •Published evidence suggests that net residential care cost avoidance of £44,722 p.a. were made (not evaluated directly)•Pre-existing packages of care were reduced on average by £10 per person per week (= £16k p.a. in the pilot phase)•NET social care savings of around £6k should be made in 2013/14, increasing to £127k in 2014/15 IF numbers of patients increase

Impact - Impact on Social Care

Impact – Increased Integration of Services

85% of GPs report improved communication with Community Health and Social Care teams

•42% of GPs report that the single greatest impact on patient wellbeing is from having a social worker embedded in the team

Close working with the voluntary sector is a key feature of NE Essex virtual wards

•Voluntary Sector organisations commissioned to provide a range of support essential to health and wellbeing include:

– Mental Health First Aid training for staff – MIND– Befriending & Outreach - Clacton Family Support, Age UK, Tendring Specialist

Stroke Services– A ‘Message in a Bottle’ for all patients - The Lions Club– Voluntary Services Directories - CVSs – Support literature – St Helena Hospice & Epilepsy Action

So What?• Multi-disciplinary care planning delivers a good patient

experience, but has limited impact on service demand

• “Everyone Counts” and the Better Care Fund are pushing health and social care into integrated commissioning and delivery – on the premise that this will release efficiencies. Lack of evidence of ability to realise large scale efficiencies may provide a reality check to scale of savings achievable?

Information Governance

• These evaluations were carried out pre-April 2013, when Regulations on access to Patient Identifiable Data changed.

HOW COULD THE EVALUATIONS HAVE BEEN

IMPROVED?

Challenges and Pitfalls1. INFORMATION

GOVERNANCE

2. Limitations in routinely collected data (content / completeness / timeframe / costing)

3. Identifying meaningful controls / comparators

4. Selling the benefits of evaluation to delivery teams

5. Producing a balanced review of quality & cost

5. Linking datasets (high % mismatch)

6. Attribution of cause and effect

7. Over-interpretation of limited data

8. Wide confidence intervals

9. Managing expectations of decision-makers

10. Inconclusive results make decisions difficult

CONCLUSION

Evaluation in a service environment will not be perfect and will not go to plan.

The “science” of public health evaluation in a service context is to deliver something sufficiently

robust for the evaluation purpose.

The “art” of public health evaluation is understanding the needs of decision-makers, and

knowing when ‘good enough’ is good enough!