EBM Revisited: a brief refresher

Introduction and Description of the ADEPT Course

Andrew Booth & Marishona Ortega

Why ADEPT?• librarians need to be able to support

Evidence Based Healthcare

• users need to be able to do their own focused searching

• only a small proportion (about 1%) of the medical literature is clinically relevant

The Ingredients of EBM

• Focused questions• Hierarchy of evidence• Filters• Problem based learning

– Scenarios– Anatomy of the question

– Shared expertise (SEE-ONE, DO-ONE, TEACH-ONE)

Focusing the question

• Helping requester to break information need down into EBM anatomy clarifies the originating question.

• “OR” within categories and “AND” between categories.

Understanding study design

• Adds to your ability to recognise “good studies”and “bad studies”.

• Helps you to identify the question being answered by an article.

Using methodological filters

• One line, sensitive and specific versions

• rely on publication types or occurrence of textwords to identify methodologically superior studies

• are also used to compile pre-filtered database products e.g.. Cochrane Library (clinical trials), DARE (reviews) & NHS EED (economic evaluations)

Topics

Types of question - primary studies

• Diagnosis

• Etiology

• Prognosis

• Therapy

Types of article - secondary studies

• Reviews• Economic Evaluations• Guidelines

[and now Qualitative]

The Process• Clinical Scenario

• Focusing the question

• Translating into MeSH terms

• Conducting the literature search

• Applying the methodological filters

• Filtering for quality studies

• Specialist resources

ResourcesFor each study:• User Guides from

JAMA• ACP Journal Club or

EBM MEDLINE Guides

• CASP filters page http://www.phru.org.uk/ ~casp/filters.htm

About ADEPT:• Becoming ADEPT:

Project Overview - HLR Dec 1997.

• Becoming ADEPT: delivering distance learning on EBM to Librarians - HLR Sept 1998.

Principles of Study Design: a librarian’s perspective

Andrew Booth

Scenario

You are a postgraduate medical librarian in Kleptovale NHS Trust. You have bid for a Nicksnix Library security system. The postgraduate Dean wants some evidence that it is effective. There is no published evidence so you will need to conduct an original study. What are you going to do?

Lower forms of evidence

• Anecdote (I once knew a librarian who……)

• Case study

• Case series

Answer. A Case-control study• You call up the Regional Library Unit.

Details of 10 libraries with book losses of >5% (cases) and of 10 libraries with losses of <1% (controls).

• Phone all 20 libraries and ask “Do you use a library security system?” Yes/No

You compile a 2 by 2 table.

Yes >5% No >5% Total

YesSystem

4 6 10

NoSystem

6 4 10

Total 10 10 20

But........• Controls might be underreporting their

losses to make system look good.

• People with security systems may be more security conscious (using other methods)

• Bias (flaw in research method) or confounding (some other factor not taken into account).

Case-control studies [Retrospective]

Advantages• Quick and cheap• It’s the only method

for rare events• Requires fewer

subjects than other designs

Disadvantages• Relies on

recall/records• Open to confounding• Selection of controls

is difficult• Potential bias in recall

or selection.

Answer: A Cohort Study

Convince 110 members of HLG to take part in a prospective cohort study. Over a two year period you record who uses the system [Exposed] and who doesn’t [Non-Exposed]. At end of two years you calculate rate of heavy losses within both groups.

Another 2 by 2 table

No. No. of heavylosses

Nicksnix 30 3 1/ 10

Non-Nicksnix

80 12 1.5/ 10

But.....• OK so you may have eliminated recall and

reporting bias through baseline data collection, but

• Libraries with security systems may be in richer areas and less prone to theft anyway.

Cohort studies

Advantages• Subjects can be

matched• Eligibility criteria can

be standardised• Easier to run than an

RCT.

Disadvantages• Blinding is difficult• Randomisation is not

present• Requires a large

sample and longer follow up.

Answer: An RCT• You go for a sponsored Randomised

Controlled Trial. You recruit another group of HLG members. You randomly assign half with a free NicksNix system and the others must go without. After a year you have the following results.

Yet another 2 by 2 table

No. No of losses

Nicksnix 100 2 2%

Non-Nicksnix

100 2 2%

Finally....

After three years and thirty minutes and three separate studies the Postgraduate Dean believes he has the right answer:-

No difference, No Nicksnix,

No comment!!!

Randomised Controlled Trials

Advantages• Unbiased distribution

of confounders• Blinding more

possible• Randomisation

facilitates statistical analysis

Disadvantages• Expensive in time and

money• Volunteer bias• Ethical problems,

sometimes• May create artificial

behaviour

Postscript

Practically, you wouldn’t go to all that effort. Maybe use a cross-over design where one Library (n-of-1 study) or more went through periods without and then with a system. You would need to make sure that there was a washout period. Also you would have to guard against a placebo effect.

Hierarchy of effectiveness• Systematic Review

• Randomised Controlled Trial

• Cohort study [Prospective]

• Case-control [Retrospective] (May be contemporary or historical)

• Case series

• Individual case study

There are other hierarchies

E.g. Hierarchy of patient experience• Qualitative research study• Interviews/Surveys• Case study• Anecdote

• C.p. Signal to noise ratio (Edwards AG, Russell LT & Stott NCH (1998) Signal versus noise in the evidence base for medicine: an alternative to hierarchies of evidence? Family Practice 15 (4): 319-322)

Constructing a Question

Marishona Ortega

EBP Process

• Define problem

• Find evidence

• Appraise evidence

• Apply results of appraisal

• Evaluate change

• Redefine problem

Formulating a question using PICO

• Population/Patient/Condition

– Who is the question about?

• Intervention/Exposure

– What is being done to or happening to the patient/population?

• Comparison(s)

– What could be done instead of the intervention?

• Outcome(s)

– How does the intervention affect the patient/population?

Another way of looking at PICO

Focused Questions - the benefits

• Guides search process

• Prompts for considering synonyms

and alternatives

• Saves time - increases relevance

Translating your question into a strategy

Your department is exploring the idea of introducing near-patient testing for Group A ß-Streptococcal throat infection. You have been asked by your manager to gather the evidence for this procedure with particular reference to sensitivity, specificity and cost.

PICO Applied:

• Patient/Condition– Group A ß -Streptococcal throat infection

• Intervention– Near-patient testing or rapid test

• Comparison– Laboratory testing

• Outcome– Sensitivity, specificity, cost

Now your turn…

As background to your service you are compiling information identifying the risks of infection from HIV, Hepatitis B & C and CMV following homologous blood transfusion.

How would you structure your search?

Now your turn - contd

• Patient/Population– Persons receiving blood transfusion

• Intervention– Homologous blood transfusion

• Comparison– Another form of blood transfusion i.e. autologous

• Outcome– Not becoming infected with HIV, Hepatitis B&C or

CMV

EBP Process

• Define problem

• Find evidence

• Appraise evidence

• Apply results of appraisal

• Evaluate change

• Redefine problem

Where do I start?

• Develop your question• Decide on the most appropriate

source(s)• Divide your search into

‘concepts’• Think of alternative terms for

each concept• Search each concept separately• Combine concepts using

Boolean logic

Therapy Studies: Design, Terminology and

Filters

Learning Objectives

• Describe the main characteristics of a rigorous therapy study.

• Recognise some of the terminology associated with studies of therapy

• Itemise sub-headings associated with treatment• Avoid common pitfalls associate with searching

for therapy studies• Utilise the methodological filters associated with

therapy literature searches

What is a therapy study?

• Treatment or management of a condition– Objective of treatment– Choice of treatment– Target of treatment

• Literature-based evidence can assist with all three aspects, but most particularly “Choice of treatment”

Therapy Studies

• A search for therapy studies focuses on the decision between treatment choices

• Therefore the evidence will be comparative

• The preferred design for a therapy study is a RANDOMISED CONTROLLED TRIAL (RCT)

Searching for therapy studies

Methodological filters 1

1 randomized controlled trial.pt.

2 controlled clinical trial.pt.

3 randomized controlled trials/

4 random allocation/

5 double blind method/

6 single blind method/

7 or/1-6

8 clinical trial.pt.

9 exp clinical trials/

10 (clin$ adj25 trial$).ti,ab.

11 ((singl$ or doubl$ or trebl$ or tripl$) adj25 (blind$ or mask$)).ti,ab.

12 placebos/

13 placebos.ti,ab.

14 random.ti,ab. 15 research design/ 16 or/8-15 17 comparative study/ 18 exp evaluation studies/ 19 follow up studies/ 20 prospective studies/ 21 (control$ or prospectiv$ or

volunteer$).ti,ab. 22 or/17-21 23 7 or 16 or 22 24 animal/ 25 human/ 26 24 and 25 27 24 not 26 28 23 not 27

Methodological filters 2

Reinhard Wentz:1 explode clinical trials2 explode research design3 randomized controlled trial.pt.4 clinical trial.pt.5 (single or double or treble or triple).tw.6 (mask$ or blind$).tw.7 5 and 68 placebos/ or placebo$.tw.9 1 or 2 or 3 or 4 or 7 or 8

Methodological filters 3

Mid-range strategy from Health Information Research Unit, McMaster:

1 randomized controlled trial.pt.2 or dt.fs.3 or tu.fs.4 or random$.tw.5 or/1-4

And last but not least...

The one-line filter:

clinical trial.pt

• Retrieves 97% of rigorous therapy studies

• Only includes 3% of irrelevant studies

So how does this work in practice?A 17-year-old female was travelling home one night when

the car her friend was driving careered off the road. Fortunately, no-one was seriously injured but she presented to A&E with localised low back pain and was diagnosed with lumbar strain.

For some time the Senior Physio has been considering the treatment for low back pain and wonders if acupuncture might be a more effective and comfortable alternative. She asks you to run a search to find the relevant evidence.

What’s your Question?

• Patient/Condition– 17 year old female with low back

pain/lumbar strain

• Intervention– Acupuncture

• Comparison– Existing treatment

• Outcome– Pain relief

What’s your Strategy?

• Usually start with PATIENT/CONDITION and then the INTERVENTION

• Thus:1. low back pain2. acupuncture3. 1 and 24. limit 3 to clinical trial.pt

Practical Session

Go to:http://gateway.ovid.com

Username: trainshePassword: train123

If you need help - just ask!

Feedback time:

How did you get on?!

Filtering for quality studies

Andrew Booth

1. A Comparison

• Drug A versus Drug B• Drug versus other

(alternative) therapy• Drug versus placebo• Service versus

“standard care”

2. Randomisation

• To ensure both groups are similar at the beginning except for the play of chance

3. Blinding

• Subjects are blinded• Those delivering

treatment are blinded• Those doing analysis

are blinded

• Contrast: Concealment of allocation

4. Withdrawals and drop outs

• Are all those who entered into study accounted for at its conclusion

• Was an intention to treat analysis used?

5. Follow up

• Were all participants followed up and was data collected in the same way?

6. Sample size

• Were there enough participants to minimise the play of chance?

Preview of Distance Learning Module

Andrew Booth

Each module

1. Reading material

2. Worked exercise

3. Exercise with deadline date

4. Group (Anonymised) Feedback

Modules

• Reviews• Diagnosis• (A)etiology/Prognosis• Guidelines/Economic

Evaluations• Qualitative Research

Contact Us:

Marishona Ortega• M.Ortega@sheffield.ac.uk• 0114 222 0797Andrew Booth• A.Booth@sheffield.ac.uk • 0114 222 0705Or leave a message with Tricia Campsell• 0114 222 0703