Critical AppraisalCritical Appraisal
Steve HaighSteve Haigh
Senior Medicines Information and Senior Medicines Information and FFormularyormulary
Sherwood Forest Hospitals, Nottinghamshire.Sherwood Forest Hospitals, Nottinghamshire.
a critical reading.
This lecture and other resources at This lecture and other resources at
Version 19.6.2015Version 19.6.2015
The second one is the meant definition, but the first one is possibly better. Which seems harsh, but when you realise you are up against people trying to mislead you, it becomes more appropriate. Other useful words are skeptic (like the second definition) and cynic (the first definition).
Identify key components of clinical trial design and apply theseIdentify key components of clinical trial design and apply these to a critical to a critical
appraisal of the literatureappraisal of the literature
Be able to work out measures of effectivenessBe able to work out measures of effectiveness
In this In this noninferioritynoninferiority
trial, we randomly assigned 18,113 patients who had atrial fibrtrial, we randomly assigned 18,113 patients who had atrial fibrillationillation
and a risk of stroke to receive, in a blinded fashion, fixed dosand a risk of stroke to receive, in a blinded fashion, fixed doses of dabigatran es of dabigatran --
110 mg or 150 mg 110 mg or 150 mg
twice daily twice daily --
or, in an unblinded fashion, adjustedor, in an unblinded fashion, adjusted--dose warfarin. The median duration of the followdose warfarin. The median duration of the follow--up up
period was 2.0 years. The primary outcome was stroke or systemicperiod was 2.0 years. The primary outcome was stroke or systemic
Rates of the primary outcome were 1.69% per year in the warfarinRates of the primary outcome were 1.69% per year in the warfarin
group, as comparedgroup, as compared
with 1.53% per year in the group that received 110 mg of dabigatwith 1.53% per year in the group that received 110 mg of dabigatran (relative risk withran (relative risk with
dabigatran, 0.91; 95% confidence interval [CI], 0.74 to 1.11; P
If we use warfarin in 1000 people, If we use warfarin in 1000 people, 15 will come 15 will come
a cropper but if we use dabigatran only 10 will.a cropper but if we use dabigatran only 10 will.
So this makes a difference for 1 in every 172So this makes a difference for 1 in every 172 patients.patients.
For exampleFor example……
Smiley face chartSmiley face chart
Cates plot: Cates plot: www.nntonline.net/visualrxwww.nntonline.net/visualrx//
Or even better distil it down to something even a patient can understand.
But before you get excited about dabigatran we need to check the quality of the data. We are not the Daily Mail who trot out any old nonsense without fact checking.
Systematic examination of evidence to assess its validity and reSystematic examination of evidence to assess its validity and relevancelevance
What is Critical Appraisal?What is Critical Appraisal?
‘‘The conscientious, explicit and judicious use of current The conscientious, explicit and judicious use of current
best evidence in making decisions about the care of best evidence in making decisions about the care of
individual patientsindividual patients’’
DL, Richardson WS, Rosenberg W et al. EvidenceDL, Richardson WS, Rosenberg W et al. Evidence--based medicine: how to practice and teach based medicine: how to practice and teach
EBM. Churchill Livingstone, London. 1997EBM. Churchill Livingstone, London. 1997
What is Evidence Based Medicine?What is Evidence Based Medicine?
Critical appraisal is part of the overall master plan - Evidence Based Medicine. Which is about using the best available evidence to make decisions about patients. To do this, you need to be able to analyse the information that is provided.
Cf science based medicine – where the whole of scientific knowledge is taken into account. Eg trial data has to be weighed against the laws of physics.
Just because something is published doesnJust because something is published doesn’’t make it validt make it valid
If the data is valid within the confines of the trial protocol, If the data is valid within the confines of the trial protocol, is it applicable to is it applicable to
your patients?your patients?
If the data is applicable to your patients, is it in a readilyIf the data is applicable to your patients, is it in a readily--understandable understandable
form for your audience?form for your audience?
Why do Critical Appraisal?Why do Critical Appraisal?
CASP tools very usefulCASP tools very useful
Different tools for different types of trialsDifferent tools for different types of trials
What isWhat is……? Series from Bandolier? Series from Bandolier
BMJ publishing BMJ publishing –– How to Read a Paper How to Read a Paper (Trisha (Trisha GreenhalghGreenhalgh))
Bandoliers little bookBandoliers little book
How to critically appraise the evidenceHow to critically appraise the evidence
The tools available help to give you a system to appraise trials and make sure you don’t miss anything. You can have a look at various tools after the course, but we’ll run through most of the key aspects to consider in this session.
We’re going to run through a few aspects of trial design, and thinking about what we’d ideally want in a trial (if you know what you’d ideally like then you can compare what you have with it and see whether its good enough)
Another barrier to understanding and appraising a trial is the use of specialist terminology. The use of ‘jargon’ isn’t bad in itself, as it allows very precise meanings to be communicated, but it can appear intimidating if you don’t know what it’s about. A lot of this session will be about working out what the terminology means, so you can pull the relevant information out
Randomised Controlled Trial (RCT)Randomised Controlled Trial (RCT)
Case studies / case seriesCase studies / case series
Prospective / retrospectiveProspective / retrospective
What do the above types of study actually mean? What type do yoWhat do the above types of study actually mean? What type do you think is u think is
ideal? Is this true / feasible for all situations?ideal? Is this true / feasible for all situations?
Types of studyTypes of study
““In my experienceIn my experience””
--Dr Mark Dr Mark CrislipCrislip ((QuackcastQuackcast podcast)podcast)
What about anecdotes?What about anecdotes?
What are the three most dangerous words in medicineWhat are the three most dangerous words in medicine……??
Going to discuss the things you want to think about when critically appraising a RCT, but there are other types of study
Meta-analysis – combines information from a number of trials
RCT – usually considered gold standard
Cohort – follow the same cohort of patients through time until the event happens
Case control – event already happened, find similar people without event and compare histories
Cross-over – every participant gets both trial and comparator treatments – act as own control
Case studies – ‘we did this in a patient and this happened’
Headache stack graphHeadache stack graph
Patients with headache
This trial shed some light on the problem of anecdotes. Discerning the signal from the noise.
From the Book Meaning, Medicine and the 'Placebo Effect' by Daniel Moerman. Referencing: Analgesic effects of branding in treatment of headaches. A BRANTHWAITE, P COOPER. BMJ VOL 282 16 May 1981.
Overwhelming observable evidence?Overwhelming observable evidence?
The actual highest level of evidence!
Everyone can observe it for themselves and it’s indisputable.
Doesn’t apply to many aspects of using drugs as the effects are often too subtle.
Applies to things like “Stemming the blood flow from a severed artery is a good idea” and “Does propofol make you sleep”.
But not “is propofol better than sevoflurane”