Embed Size (px)
Citation preview
116 (27%)185 (43%)
49 (11%)
How to critically appraise a systematic review
Igho J. Onakpoya MD MScUniversity of Oxford
Centre for Evidence-Based MedicineNuffield Department of Primary Care Health Sciences
Learning outcomes
Principles of critical appraisal of SRs
How to undertake critical appraisal of SRs
How to appraise published SRs
Assessing the relevance of published SRs
2 sessions Part I - Teaching (90 mins)
Part II - Class activity (90 mins)
1 short break in session 1 (5 mins)
15 minute break between sessions 1 and 2
Opening scenarioMs X is a 35 year old banker with five children. She lost her dad who was hypertensive three years ago due to a heart attack. She is worried about her risk of having high blood pressure but does not want to take any conventional medicines because “they have lots of side effects”. Her close friend has told her to take green tea but she is not sure whether “it works”. She therefore wants your opinion on whether this a good idea.
Get on the internet
Let’s see PubMed Clinical Queries!
Decide on which review/reviews meet your need .
Actually, we have done ours too!
What is a systematic review?
Systematic identification, appraisal and synthesis of the evidence from studies addressing the same research question.
Why systematic reviews?In
crea
sing
stre
ngth
of e
vide
nce
Traditional (narrative) reviews
Don’t accept the conclusions of every
systematic review hook, line and sinker!
How to read a systematic review
Start with the title!
Next step: Useful pneumonic
Q: Was there a clear research question?
F: Did they find all relevant studies?
A: Was study quality assessed?
S: Were the results adequately summarized?
Step 1: Is there a clear and focussed research Question?
Be sure of what you are looking for.
Clear Question?
Participants
Intervention
Comparator
Outcome
Based on the paper Onakpoya et al.
Q: What is the PICO?
P: Normotensive or hypertensive adults I: Green tea C: Placebo/identical controlsO: Blood pressure
Step 2: How did they Find the studies?
1. Which databases were searched, how many?
2. Was the search period specified?
3. Was grey literature searched?
4. Did they contact study authors?
5. Did they search bibliography?
F: How were studies included?
Independent determination of study eligibility
How many were eligible?
How many were excluded?
Were the reasons for exclusion reported?
Is there a list of excluded studies?
Resolution of disagreements
Flow charts could be helpful
QF
Step 3: Did they Assess study quality?
Many quality assessment checklists available:
Cochrane
Jadad
CONSORT
STROBE
GRACE
Quality rating can be subjective
Onakpoya et al. The effect of cactus pear (Opuntia ficus-indica) on body weight and cardiovascular risk factors: a systematic review and meta-analysis of randomized clinical trials. Nutrition. 2015 May;31(5):640-6
Cochrane Assessing the risk of bias
Step 4: Did they Summarize the evidence?
1. Did they extract results data from included studies?
2. Did they statistically combine the data?
3. Did it make sense to combine the data?
Statistical data combination
http://www.cochrane.org/
Basics of meta-analysis
Statistically combines data from individual studies: forest plot
Uses fixed or random effects model Can test for variations in trial designs and participants:
heterogeneity
Can test for publication bias: funnel plot
What is a meta-analysis?
• A way to calculate an average
• Estimates an ‘average’ or ‘common’ effect
• Improves the precision of an estimate by using all available data
What is a meta-analysis?
Optional part of a systematic review
Systematic reviews
Meta-analyses
At the bottom there’sa horizontal line. This is the scale measuringthe treatment effect.
The vertical line in themiddle is where thetreatment and control have the same effect – there is no differencebetween the two
For each study there is an id
The data foreach trial are here, divided into the experimental and control groups
This is the % weightgiven to thisstudy in the pooled analysis
The label above the graph tells you what statistic has been used
The data shown in the graph are also given numerically
The pooled analysis is given a diamond shapewhere the widest bit in the middle is located at the calculated best guess (point estimate), and the horizontal width is the confidence interval
Note on interpretation
If the confidence interval crosses the line of no effect, this is equivalent to saying that we have found no statistically significant difference in the effects of the two interventions
trials
Confidence interval
Line of no effect
Overall effect
Effect of green tea supplementation on systolic blood pressure (mmHg).
Did the authors interpret the results correctly?
Beware of missing data!
Appendix 1a: Funnel plot of mean differences plotted against sample size. The vertical line represents the mean difference for all trials.
Based on the review findings what advice would you give Ms X?
And why?
Questions to ponder on
Do the results appear valid and reliable?
How widely applicable are the study findings?
Are the results relevant to my local population?
Do the results apply to this patient/individual?
Useful pneumonic
Q: Was there a clear research question?
F: Did they find all relevant studies?
A: Was study quality assessed?
S: Were the results adequately summarized?
Part II: Class activity
Critically appraise the systematic review: 45 minutes
2 groups will look at chlorogenic acids
2 groups will look at steviol glycosides
Nominate someone to present for your group
We re-group at 4:30 pm
5 minute presentation summary of your review
Class activity
Any Questions?
Useful resources
Cochrane Handbook of Systematic Reviews of
Interventions [updated March 2011].
http://handbook.cochrane.org/
Gough D, Oliver S, Thomas J. An Introduction to
Systematic Reviews. SAGE Publications Ltd. 2012
PRISMA. Transparent Reporting of Systematic Reviews
and Meta-Analyses. http://
www.prisma-statement.org/index.htm
