BazianBetter health at lower cost

30 June 2011

Overview

2

Introduction to Bazian Evidology and cost vs quality Examples of Bazian work Search and appraisal example – aetiology of ADHD Behind the Headlines Brief introduction to NHS Evidence Hubs

Overview

3

Introduction to Bazian Evidology and cost vs quality Examples of Bazian work Search and appraisal example – aetiology of ADHD Behind the Headlines Brief introduction to NHS Evidence Hubs

Bazian

Started by two clinicians from the BMJ Clinical Evidence - Anna Donald and Vivek Muthu

Employ doctors, epidemiologists, librarians and scientists, and train intensively in evidence-based methods, communication, and implementation

Interpret evidence to make it useful for policy, not just a dry summary of the literature

4

Our Purpose: To clarify the cost-quality relationship in healthcare....

Working with clients we use the following skills to answer questions and apply answers

Demand and capacity modelling

Health Technology Assessments

Data Analysis / Literature reviews

Clinical process flows

Economic analysis

6

What is the clinical effect of doing X? Who in this population benefits/risk from X? What is the cost of achieving X? What is the national and local average for X? Is there any national guidance? What is a realistic target for this population? How is X achieved? What we should (not) do? How much do we need? What it will cost? What it should deliver? How do we monitor performance?

Typical questions we answer

?

With a dedicated team of experienced clinicians, information specialists and health analysts

7

Project management Methods advisersSpecialist clinical advisers

Ev·i·do·l·o·gy n.A medical specialty that enables medical research to be incorporated into clinical practice.

[Latin videre to discern, comprehend; evideri to appear plainly]

Overview

9

Introduction to Bazian

Evidology and cost vs quality Examples of Bazian work Search and appraisal example – aetiology of ADHD Behind the Headlines Brief introduction to NHS Evidence Hubs

What is happening in healthcare?

10

Variable outputs

Expenditure does not relate to quality

11Source: The Commonwealth Fund: Mirror, Mirror on the Wall

Rising healthcare costs

12

The obligatory joke

13

Examining the cost quality relationship

UK – Dept of Health is prioritising its workload NICE QIPP

Dartmouth Atlas Cost/Quality US budget proposal

Savings to fund healthcare will come from among other things:“create ‘quality incentive payments’ for hospitals” ($12.1 billion), “drive down hospital readmission rates for Medicare patients” ($8.43 billion).

“Comparative effectiveness research $1.1billion)

14

MythQuality in healthcare is proportional to cost

What works, what doesn’t & what we don’t know

Based on 2400+ systematic reviews of treatmentsSource: Clinical Evidence, Volume 14, December 200515

Questions need to be answered

CostHow can we reduce spend and tackle budget deficits without

compromising the clinical effectiveness of treatments?

Evidence for treatmentHow should evidence on healthcare effectiveness guide

commissioning decisions?

Budget AllocationsHow might commissioners focus resources on treatments which

work best and divert them away from those that do not?

Evidology uses all medical research to inform clinical and management decisions

16

What evidence can do…

Evidence enables insurers, purchasers and providers of healthcare to:

identify, adopt, monitor and develop better and more efficient health services and care processes for the populations they serve

address the real causes of medical cost escalation

disinvest from poorer and less efficient health services and care processes

To permit us, the consumers of healthcare, to:

enjoy longer, healthier, happier lives

afford the healthcare that we desire

17

18

Savings example #1: knee arthroscopy

Each year over 18,000 knee arthroscopies are performed on the NHS

Many of these procedures are done on patients with osteoarthritis for

whom evidence suggests joint injections, or no intervention at all,

would have been just as effective and a lot cheaper:

Procedure Cost per patientKnee arthroscopy £1000-£2,500Joint injections < £100No intervention £0

Source Evidence-Based Healthcare & Public Health (2005) 9, pp 192-196, Author: Bazian Ltd

Savings example #2: Smoking cessation

Before EB:Lots of GP’s opinions whether it was worth

advising people to quit smoking as routine practice, or whether it made no difference, other than to alienate them..

The evidence:Brief advice to encourage people to quit smoking

delivered by a physician, compared with no advice: AR 5.9% with brief advice v 4.1% with no advice OR 1.69, 95% CI 1.45 to 1.98

Combining nicotine replacement therapy with advice: AR 16.8% with advice + NRT v 10.2% with

advice + placebo OR 1.74, 95% CI 1.64 to 1.86

After EB:GP practices in the UK now adopt smoking

cessation advice as a routine policy, rather than as an optional, if-you-feel-like-it event.

19

© Bazian Ltd, page 20

Why is evidology not used more?

What do you meanmedicine isn’t based

on‘evidence’?

© Bazian Ltd, page 21

Systematic reviews are hard work…

E.g. Does the MMR vaccine cause autism? >3600 studies. 36 relevant and reliable to this question.

1940s….15 studies1950s….30 studies1960s….150 studies1970s...500 studies1980s…900 studies1990s…1000 studies2000s….1000 + studies

The long and painful process of putting evidence into practice is worth it because…

Essential to escape the healthcare cost escalation

There really are ‘better’ and ‘worse’ treatments and care configurations and we need to be able to distinguish between them

It is the only way to reveal previously opaque cost-quality relationships

Essential for maintaining competitiveness as healthcare becomes increasingly commoditised

22

Why don't commissioners do this already?

Asking commissioners to ‘find their own evidence’ is a bit like asking GPs to do their own MRI scans. They simply do not have the years of training, experience, equipment, or time to do so.

They need a responsive and dedicated evidence service – a bit like a specialist radiology service – which provides evidence answers in real time.

23

Overview

24

Introduction to Bazian Evidology and cost vs quality

Examples of Bazian work Search and appraisal example – aetiology of ADHD Behind the Headlines Brief introduction to NHS Evidence Hubs

A few of our clients

25

26

DeliverablesProvision of documents including training, guidance for PCTs, evidence synthesis reports and draft scopesSpecialist support for Appeals

Example #1CSAS: ensuring commissioners engage with NICE

To help commissioners engage in NICE TAs so that population needs and opportunity costs are represented.

We are the clinical and analytical engine behind the CSAS reports for NICE.

What will this new NICE TA mean for my PCT?How can I influence the NICE process?

27

“Analysis revealed that: For an average sized local health economy, for £1.4million, 20 women can be treated with Drug X as an alternative to Drug Y...

At the end of 1 year, about 3 extra people will be alive who otherwise would not have been

After 2 years there will be no overall survival benefit for the women treated with combination therapy ....”

28

Example #2EMSCG: improving commissioning of RRT

To help commissioners purchase renal replacement therapy services across the East Midlands, including long-term planning for the dialysis service.

Objectives1.to map the geographical variation in need across the East Midlands (age/sex and ethnicity)2.to report on capacity and provision of dialysis services by provider and compare this to need3.to qualitatively describe any pressures within the system.

Service capacity planning (EMSCG)

Local need, demand and capacity model

Interactive pathway simulation identified future service efficiencies, and gained buy-in from specialist clinicians

31

Map of demand density vs local capacity to plan unit location

33

Some more single-slide examples of work we’ve done…

34

DeliverableAn “Evidence Brief” which includes;

•Policy background and description of local demographics and issues from existing reports•Evidence summary and policy suggestions•Description of the technology•Evidence tables•Methodology

What are the psychological outcomes following gender reassignment surgery for gender dysphoria?We carried out independent evidence-based analysis of important positive and negative outcomes, including psychological effects, suicide rates and quality of life.

35

DeliverablesBazian summarised the findings of the review, describing implications with regard to the issues raised. Including how to improve and update the eligibility criteria used by the NHS in England.

Presentation of findings to NSC or expert group

We carried out a guideline comparison and survey of international current practice to ensure fair access to the drugs used to treat these disorders are in line with international practice.

How do clinicians use Ultra Orphan Drugs for Lysosomal Storage Disorders?

36

DeliverableA literature reviewAn economic reportAn interactive economic model in spreadsheet (.xls) format suitable for downloading by users from a website.

What is the effect of Functional Electrical Stimulation (FES) for drop foot of the central neurological origin?

To form the basis of CEP’s agreement with supplier for delivery of those products within appropriate constraints

37

DeliverableA literature reviewAn interactive economic model in spreadsheet (.xls) format .

Can we have a guideline for…?

Revised and updated the original guideline for the recognition and assessment of acute pain in children

Overview

38

Introduction to Bazian Evidology and cost vs quality Examples of Bazian work

Search and appraisal example – aetiology of ADHD

Behind the Headlines Brief introduction to NHS Evidence Hubs

ADHD aetiology plan

39

Search strategy

Must balance sensitivity and specificity

To refine list of putative risk factors we carried out a brief scoping in PubMed

An information specialist reviewed the results and extracted a list of potential risk factors. This list was augmented by adding risk factors suggested by the reference group. Key search terms covering these risk factors were included in the search strategy to increase likelihood of identifying relevant studies

Sources: MEDLINE, EMBASE, CINAHL and PsycINFO Hand-searched the reference lists of relevant papers The reference group and “key informants” identified by the reference group

were contacted to supply personal reference lists and information on ongoing, unpublished or in-press research

Grey literature sources such as Google Scholar, TRIP and OAIster.

40

ADHD aetiology search (Medline)

41

1 Attention Deficit Disorder with Hyperactivity/ (13393) 2 (attenti$ adj3 (deficit$ or disorder$ or hyperactiv$ or hyper?activ$ or adhd or addh or ad??hd or "add")).mp. [mp=title, original title, abstract, name of substance word,

subject heading word] (18981) 3 ((hyperkin$ or hyper?kin$) adj3 (deficit$ or disorder$ or hkd)).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (496) 4 or/1-3 (19291) 5 (cohort$ or longitudinal or follow?up or randomi$ or randoml$ or RCT or prospective or retrospective or case?control or cross?section$ or observational or epidemiolog$ or

case?series or linkage$ or epigenetic$).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (1573617) 6 ((genome$ or gene$ or chromosome$) adj5 (wide or analys$ or map$ or associa$ or environ$)).mp. [mp=title, original title, abstract, name of substance word, subject

heading word] (332066) 7 ((candidate or suscept$) adj5 (gene$ or loci$)).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (46254) 8 between group$.tw. (43128) 9 ((twin$ or sibling or adopti$) adj3 (stud$ or analys$ or pair$)).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (13716) 10 or/5-9 (1891632) 11 (risk$ or aetiol$ or etiol$ or caus$ or associat$ or linkage$ or heritab$).mp. (3734422) 12 (dopamine$ or serotonin$ or hla-dr$ or adrenergic or catechol O-Methyltransferase or cannabinoid$ or polymorphi$ or genom$ or thyroid).mp. [mp=title, original title,

abstract, name of substance word, subject heading word] (877609) 13 ((maternal or paternal or parent$ or famil$ or mother$ or father$ or prenatal or pregnan$) adj3 (smok$ or tobacco or depression or stress or restrictiveness or substance

or alcohol$ or coffee or social or class or education or psychiatri$ or disorder$)).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (50975) 14 exp family/ (182571) 15 (lead or mercury or manganese or polychlorinated biphenyl or hexachlorobenzene or pollutant$ or PBB or PCB or DDE or DDT or pesticide$ or organophosphate$ or

pyrethroid$ or zinc or iron or magnesium or omega-3 or sugar$ or "e numbers" or e-numbers or colouring$ or toxin$ or additive$ or diet$ or malnutrition$ or malnour$ or undernutrition$ or undernour$).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (1253538)

16 exp pesticides/ or exp coloring agents/ or exp flavoring agents/ or exp sweetening agents/ or Environmental Exposure/ (422004) 17 ((pregnan$ or delivery or birth) adj3 (weight or preterm or complication$ or BMI or season)).mp. [mp=title, original title, abstract, name of substance word, subject

heading word] (164841) 18 exp Pregnancy Complications/ or exp Obstetric Labor Complications/ (276386) 19 (breast-feeding or breast-fed).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (24937) 20 exp Infant Nutritional Physiological Phenomena/ (35245) 21 (head adj3 (trauma$ or injur$)).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (22597) 22 exp Craniocerebral Trauma/ (95881) 23 (television or video or computer).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (385594) 24 exp "Play and Playthings"/ (6132) 25 (childhood adj3 depriv$).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (102) 26 sleep depriv$.mp. [mp=title, original title, abstract, name of substance word, subject heading word] (6647) 27 or/11-26 (6092705) 28 4 and 10 and 27 (3536) 29 (animal$ not human$).sh. (3286408) 30 (letter or editorial).pt. (910772) 31 28 not (29 or 30) (3446)

Condition

Aetiology & causes

Study type

Search assessment form checklist

42

We use RefMan to manage references

43

Note that searches vary according to client or project need

44

Appraisal and inclusion/exclusion Three stages:

1. First pass appraisal at the level of title and abstract, appraised simply on whether the study was

relevant to the question. No decision was made on study quality. The appraisals were carried out by an information specialist and a health research analyst. Tagged keywords in Reference Manager were used to provide an audit trail of accepted and rejected articles. The full text versions of articles selected for further appraisal were then acquired

2. Second stage appraisal by a health research analyst at the level of abstract only and applying the full

inclusion and exclusion criteria detailed below. Those that did not provide enough information at the level of the abstract were obtained at full text, for third stage appraisal

3. Third stage appraisal1. by a health research analyst at the level of full text and applying the full

inclusion and exclusion criteria detailed below

45

Quality appraisal: Scoring

46

Data is extracted into data tables etc…

47

Overview

48

Introduction to Bazian Evidology and cost vs quality Examples of Bazian work Search and appraisal example – aetiology of ADHD

Behind the Headlines Brief introduction to NHS Evidence Hubs

18 April 202349

Summary of the news stories, the research and the take home message

Where did the story come from?

What kind of scientific study was it?

What were the results of the study?

What interpretations did the researchers draw from these results?

What does the NHS Knowledge Service make of this study?

Links to headlines, science, and further reading

18 April 202350

Content

How do we critically appraise the research?

Ask a number of key questions:

1. Has the research come from a reliable source?2. What question was the research trying to answer?3. How well designed was the study for answering the question?4. How well was the study conducted?5. What are the results?6. What do the results mean for the average person?7. How is the study reported in the news?

18 April 202351

1. Has the research come from a reliable source? Has the research been fully published?

12% of a random sample of 200 press releases from 20 US academic medical centres in 2005 promoted unpublished research from scientific meetings *

5% of these included a caution about the unpublished nature of the research *

Has the publication been peer reviewed?

How was the research funded? 68% reported sources of funding *

Are there any potential conflicts of interest? 4% noted potential conflicts of interest *

*Woloshin et al. Ann Int Med 2009;150(9):613-618

18 April 202352

2. What question was the research trying to answer? Does a treatment work?

Acupuncture for back pain Blood pressure drugs for all over 55s

Does a treatment cause harm? Aspirin and brain bleed

Can an intervention prevent a disease? Aspirin and cancer

Does a diagnostic test work? Test for malignant prostate cancer

What causes a disease/condition? Nicotine gum and cancer Autism genetics

How does a treatment work/an exposure cause a disease? Cancer magnet protein

How common is a disease/condition/behaviour? CAM use among cancer patients

18 April 202353

3. How well designed was the study for answering the question? Some study designs are more appropriate than others for

answering certain types of questions

Was the study in humans? among potential therapeutic drugs reported in highly cited

animal trials in top scientific journals 1980 to 2000†: 45% were not followed up by a published human RCT (by 2006) 18% were found not to be effective in a published human RCT 37% were found to be effective in a published human RCT 11% (8 interventions) were approved for use in patients

43.5% of 2005 US academic medical press releases were of animal or laboratory studies; 10% of these included a caution about extrapolating results to humans*

about 20% of BtH studies in 2008 were animal or laboratory studies

†Hackam & Redelmeier JAMA 2006; 296(14): 1731-1732 *Woloshin et al. 2009

18 April 202354

Hierarchy of study designs

18 April 202355

Systematic reviews

US academic medical centres press releases 2005*

Behind the Headlines 2008

10%6%

16%

22%

8%

13%

4%

1%

27% controlled observational studies

3% uncontrolled intervention studies

*Woloshin et al. 2009

3%

4. How well was the study conducted? How many participants were there? What type of participants were included? How were they selected? Was an appropriate control used? Was the study prospective or retrospective? Was the outcome a clinical outcome (e.g. death, heart

attack) or a proxy outcome (e.g. blood pressure)? How was the outcome measured?

Validated scale? Researchers and/or participants blinded to treatment?

Particularly important for subjective outcomes (e.g. pain)

How many people dropped out? Was the method of analysis appropriate?

18 April 202356

5. What are the results?

How are the results presented? Benefits and harms Absolute and relative figures Contrast the impact of:

“Wonder drug reduces deaths from horrible disease from 0.1% to 0.05%”

“Wonder drug prevents 5 deaths for every 10,000 people treated” “Wonder drug halves deaths from horrible disease”

Are the results statistically significant? Which results are statistically significant?

Primary outcome(s) Secondary outcome(s) Only one comparison out of many comparisons Between group comparison or within group comparison versus

baseline Are the results clinically significant? Are authors’ conclusions justified?

18 April 202357

6. What do the results mean for the average person?

Who do the results apply to? What outcome do they apply to? Should people change their behaviour because of this

study?

18 April 202358

7. How is the study reported in the news? Does the news give a reasonable summary or is it over-

interpreted? Does the headline answer a different question to that asked

by the study? Do the photos/illustrations accurately represent the study? Does the body text clearly and accurately report:

study source potential conflicts of interest question asked whether study was in humans what outcomes were assessed study design size of study results (absolute figures as well as relative figures) limitations to the study reasonable implications of study findings?

18 April 202359

Summary

There is a hierarchy of study designs: some designs are more appropriate for certain questions than others

All studies have limitations, these may be related to: how appropriate the study design is for answering the question

asked how the study was conducted how the results are interpreted how far the results can be extrapolated

It’s important to know what these limitations are when determining what conclusions can be drawn from the study

18 April 202360

April 18, 202361

Type of question and its appropriate study designType of question Appropriate study design

Exploration of hypotheses Qualitative research

Treatment RCT or crossover RCT

Harms RCT ideally, or observational

Prognosis Cohort

Aetiology Cohort

Diagnosis Cross-sectional

Prevalence Cross-sectional

Incidence Cohort

April 18, 202362

Size isn’t everything

Error

Study size

Source: Rothman, 2002

Systematic error (bias)

Random error (chance)

Overview

63

Introduction to Bazian Evidology and cost vs quality Examples of Bazian work Search and appraisal example – aetiology of ADHD Behind the Headlines

Brief introduction to NHS Evidence Hubs

18 April 202364

NHS Evidence

Created following the Darzi report, ‘High Quality Care for All’ (2008).

The report recognised that NHS staff wanted to work with the best quality information available, but could find it difficult to access

NHS Evidence search portal and service was launched on 30 April 2009, operating within NICE

“NHS Evidence aims to be the service that health and social care staff rely on for up-to-date, relevant, high quality evidence-based local and national information and knowledge to inform decision-making resulting in better quality patient care.”

Doesn’t happen overnight!

18 April 202365

Specialist collections, all change

Previously, over 30 “specialist collections”, outsourced to a range of organisations within the NHS and wider health community. The specialist collections also produced Annual Evidence Updates on key conditions, and supported engagement with and use of evidence within their specialist community

In April 2011, NHS Evidence was launch with a redesigned evidence service

Core to the delivery of this improved service was the redesign of the existing Specialist Collection service into a new specialist information management service which provided by two Evidence Hubs: one within NICE (Manchester), and one at Bazian

The Hubs are invisible to the user. It’s not a replacement of the Specialist Collections but rather a completely new model of content provision

Other, smaller Hubs provide content on screening, public health, medicines etc

18 April 202366

18 April 202367

Contract Host Location

Number of

collections

managed

Total FTE excl

Clinical Leads

Infections Health Protection Agency

London 1 1.6

Kidney and male urogenital diseases

Health Care Libraries, Oxford University Library Services

Oxford 1 1

Later life & Mental health

Minervation Oxford 2 2.2

Learning disabilities

Minervation Oxford 1 1.4

Musculoskeletal & Orthopaedics

Arthritis Research Campaign

Chesterfield 2 2

Oral health Cardiff University Cardiff 1 1.1

Skin disorders University of Nottingham

Nottingham 1 1

Supportive and palliative care

University of Sheffield Sheffield 1 1.2

Surgery, theatres, perioperative and critical care

Royal College of Surgeons of England

London 1 2

Women's health

Nuffield Department of Obstetrics and

Gynaecology, John Radcliffe Hospital

Oxford 1 2

Contract Host Location

Number of

collections

managed

Total FTE excl Clinical Leads

CancerOxford Radcliffe Hospitals NHS

TrustOxford 1 2.4

Cardiovascular including Stroke and Vascular

University of Surrey

Guilford 3 5.2

Respiratory

Sheffield Children's

Hospital/University of Sheffield

Sheffield 1 1

Commissioning & Health management & I&I

King's Fund London 3 3.8

Complementary and alternative medicine

Royal London Homoeopathic

Hospital London 1 1.2

Emergency care & Nutrition and metabolic (inc Diabetes)

Warwick Medical School, University

of WarwickWarwick 2 2.7

ENT and audiologyOxford Radcliffe Hospitals NHS

TrustOxford 1 1

Ethnicity and health

De Montfort University

Leicester 1 0.9

Eyes and vision

Moorfields Eye Hospital

Foundation NHS Trust

London 1 1.3

Gastroenterology and liver diseases & Neurological conditions

Royal Free Hospital Medical

Library, Royal Free Hampstead

NHS Trust

London 2 1.4

Genetic conditions

Birmingham Women's

Healthcare NHS Trust (National

Genetics Education and Development

Centre)

Birmingham 1 1

18 April 202368

  Bazian Evidence Hub NICE Evidence HubSubject Area: Organ system or clinical speciality based

Anaesthetics Cancers Cardiovascular system

disorders Child Health Complementary and

alternative therapies Commissioning Critical care Diabetes Diagnostics Ear, nose and throat Fertility and childbirth Eyes and vision Gastrointestinal disorders Genetics Gynaecological disorders Health management Hepatic disorders Later life Learning disabilities Mental health and illness Neonates and neonatal

care Nervous system disorders Oral and dental health Perioperative care Renal and urogenital

disorders Sexual health Surgery

Allergies Endocrine disorders Emergency and urgent

care Equality and Diversity Palliative and End of

Life Care Haematological and

immunological disorders Infections Medically unexplained

symptoms Metabolic disorders Musculo-skeletal

disorders Orthopaedics Respiratory disorders Skin conditions Supportive care Trauma

Total 27 15

*

*

*

18 April 202369

“Key product and activities”

Identifying and adding value to evidence resources FAST ingested resources (inc. core and non-core sources) Orphan sources (bibliographic) Orphan sources (other)

Providing an evidence notification service Eyes on Evidence and “Important new evidence”

Contributing to producing Evidence Updates About n=30 a year

Identifying Research Recommendations for UK DUETs Through Evidence update process

Supporting engagement with Specialist Communities Health Management and Commissioning Expertise

Working with The Kings Fund

Thank you for listening!

70

Introduction to Bazian Evidology and cost vs quality Examples of Bazian work Search and appraisal example – aetiology of ADHD Behind the Headlines Brief introduction to NHS Evidence Hubs