Embed Size (px)
Citation preview
Statistics as a methodology for calculating clinical uncertainty
What kind of studies are relevant to practice?
Institute for Preventive Medicine and Public HealthInstitute for Environmental HealthCenter for Evidence Based Medicine
Cochrane PortugalFaculty of Medicine, University of Lisbon
Lisbon Academic Medical CenterPORTUGAL
António Vaz Carneiro, MD, PhD, FACP, FESC
Structure of the talk
• COI• Who am I and what I do for a living• The Center for Evidence Based Medicine at the University of Lisbon School of Medicine (CEMBE)
• Causality in medicine and clinical science• Evidence Based Medicine• Examples of clinical studies relevant to practice.
António Vaz Carneiro, MD, PhD, FACP, FESC
• I am a doctor• Specialist in Internal Medicine, Kidney Diseases and
Clinical Pharmacology (Portugal and USA)• I am a full Professor of clinical medicine at the
University of Lisbon School of Medicine• I am the Head of
• Institute for Preventive Medicine and Public Health• Institute for Environmental Health• Center for Evidence Based Medicine• Cochrane Portugal.
The Center for Evidence Based Medicine (CEMBE)at the
University of Lisbon School of Medicine
Origins of CEMBE
• Created in 1999• Structural unity of ULSM• Major objectives:
• Promote scientific knowledge through EBM• Clinical research• Teaching (pre, post, CME and advanced)• Knowledge translation• Scientific consulting.
Teaching department
• Pre, post, continuous and advanced medical education (Master and PhDs)
• Guideline elaboration and dissemination• Best Evidence Medical Education• Several committees at Association of Medical Education in Europe
• AVC was the Head of the Medical School’s Department of Medical Education.
Medical education expertise
• Curricula development• Faculty development• Post‐graduate training• Continuous medical education• CPD in health professions• Health information technologies• Medical decision support systems• Medical editing and design.
Research department
• Medical education• BEME review (“OSCE for pre‐med students”)
• Medicine and health• Secondary research (systematic reviews/meta‐analysis)• Observacional studies (prevalence and incidence)• Cost and burden of diseases studies• HOR and HSR• Methodological papers• Guideline science (AGREE, ADAPTE, etc.)
• Evidence based management• Evidence based policy.
CEMBE
CATS UNIFESP, Brasil Cochrane
Portugal
OMA Angola
DynaMed, USA
MORE, MacMaster, Canada
GuidelineInternationalNetwork
European Society of Cardiology‐ CPGC
NHS EvidenceNICE, UK
HSR Europe
ICHOMUSA, UK
Harvard, USA
GIN
IberoGPC
CEMBE’s International Network
10‐05‐2019 CEMBE 10
Head: A. Vaz Carneiro, MD, PhD, FACP, FESC
A short view on the concept of causality in medicine
Question for clinical medicine (therapy)
• What is the probability that what works in a group of patients will work in an individual?
• What is the probability that what harms some patients may harm an individual?
Do the results support a causal relationship between intervention and outcome?
• If an association is found, then it may be due mostly to one (or more) of the following mechanisms:• Observation bias• Operation of confounding factors• Chance• True causality.
The reliable assessment of a beneficial effect of treatment on a pre‐determined outcome requires that
studies control for:
• Bias (using the same methods in all the subjects of the study, under the same circumstances, by the same researchers, together with blinding techniques)
• Diminish the influence of potential confounding factors (for example using proper randomization with or without stratification for known factors) and
• Reduce chance (by applying rigorous statistical analysis).
The relevance of causal relationships
Field of knowledge GoalsTherapy (clinical care) • Improvement in patient’s conditionHealth services (management) • Improvement in health of communityHealth policy (government) • Resource allocationMedical education (teaching) • Improvement in learning and practice
General definition of cause
• A factor is a cause of an event if its operation increases the frequency of the event
• A causal principle for a situation lays out all the factors that operate to bring about the outcome in question in that situation, and shows how this combine to produce it.
Types of causal relationships
NECESSARY • The outcome occurs only if the causal factor has operatedSUFFICIENT • The operation of the causal factor always results in the
outcomeBOTH • The causal factor and the outcome have a fixed
relationship; neither occurs without the otherNEITHER • The operation of the causal factor increases the frequency
of the outcome, but the outcome does not always result,and the outcome can occur without the operation of the causal factor
A scheme for assessment of causation(Elwood M, 2007)
A. DESCRIPTION OF THE EVIDENCE
What was the exposure or intervention?
What was the outcome?
What was the study design?
What was the study population?
What was the main result?
B. INTERNAL VALIDITY: CONSIDERATION OF NON‐CAUSAL EXPLANATIONS
Are the results likely to be affected by observation bias?
Are the results likely to be affected by confounding?
Are the results likely to be affected by chance variation?
C. INTERNAL VALIDITY: CONSIDERATION OF POSITIVE FEATURES OF CAUSATION
Is there a correct time relationship?
Is the relationship strong?
Is there a dose‐response relationship?
Are the results consistent within the study?
Is there any specificity within the study?
D. EXTERNAL VALIDITY: GENERALIZATION OF RESULTS
Can the study results be applied to the eligible population?
Can the study results be applied to the source population?
Can the study results be applied to the relevant populations?
E. COMPARISON OF THE RESULTS WITH OTHER EVIDENCE
Are the results consistent with other evidence, particularly evidence from studies of similar or more powerful study design?
Does the total evidence suggest any specificity?
Are the results plausible in terms of biological/behavioral mechanism?
If a major effect is shown, is it coherent with the distribution of the exposure and the outcome?
Evidence‐Based Medicine
A definition of EBM
Evidence‐Based Medicine is the integration of:
• The best scientific evidence with• Clinical experience of the individual doctor with
• Patient’s preferences.
Sackett DL 1997
Steps in EBM
1. Formulating the clinical question2. Localizing and selecting the evidence with which
to answer that question3. Critically appraising the evidence for its validity,
impact and reliability, and applicability4. Integrating the findings of critical appraisal with
clinical judgment5. Applying the evidence in the clinical context to
induce change in practice.
Two Fundamental Principles of EBM
• First: EBM posits a hierarchy of evidence to guide clinical decision making
• Second: evidence alone is never sufficient to make a clinical decision.
Decision makers must always trade off the benefits and risks, inconvenience, and costs associated with alternative management strategies and, in doing so, consider their patients' values and preferences.
In contrast to the traditional paradigm of medical practice, EBM:
• Places lower value on unsystematic clinical experience and pathophysiologic rationale
• Stresses the examination of evidence from clinical research
• Suggests that interpreting the results of clinical research requires a formal set of rules
• Places a lower value on authority than the traditional medical paradigm.
4 perspectives on clinical knowledge
• There is simply too much information to be used successfully
• Good information is hard to locate and difficult to differentiate between the good and the bad
• The information sources are defective or even corrupt
• Clinical information is (mostly) reliable and there are ways to differentiate the good from the bad (separating the wheat from the chaff).
The question: the PICO format
• P (patients)• I (intervention)• C (comparison)• O (outcome)
• In patients with lone atrial fibrillation, which is the most efficacious treatment ‐ anticoagulation vs. anti‐platelet drugs – in preventing stroke?
Finding the evidence
• Primary DBs: Medline, Embase, Cinahl, PsychInfo, ERIC, AIDSline, etc.
• Secondary DBs: Cochrane Library, Clinical Evidence, revistas EBM, GIN, IberoGPC
• Softwares EBM: UpToDate, DynaMed, BMJ BestPractice, eMedicine
• Other resources: ARHQ, SIGN, NZGGG, etc.
Search engines for primary studies
• PubMed (http://www.ncbi.nlm.nih.gov/entrez• Ovid (http://gateway‐di.ovid.com/) • Google (http://www.google.pt/)• Google Scholar (http://scholar.google.com/) • SumSearch (http://sumsearch.uthscsa.edu/) • TRIP (http://www.tripdatabase.com)
2x2 table to calculate measures of association in clinical studies
Outcome + Outcome ‐ Total
Exposed a b a+b
Non‐exposed c d c+d
Total a+c b+d a+b+c+d
• Case‐control: Odds Ratio (OR)• Cohort: Relative Risk (RR)• Cross‐sectional: Prevalence• Clinical trials: Absolute Risk Reduction (ARR), Relative Risk
Reduction (RRR) e Number Needed to Treat (NNT)
The evidence‐based pratice cycle
Common clinical questions
Clinical questions
• If the test is abnormal, how likely is the patient to have the disease?
• If I prescribe this medicine, how likely is it that the patient gets better?
• If the patient has this risk factor, how likely is he to have the disease in the future?
If the test is abnormal, how likely is the patient to have the disease?
Result of the gold‐standard TotalDisease present
(test +)Disease absent
(test ‐)
Result of the new test(under study)
Test + True +a
False +b
a + b
Test ‐ False –c
True –d
c + d
Total a + c b + d a + b + c + d
Diagnostic characteristics of a test
• Sensibility = a/a+c• Specificity = d/b+d• Positive predictive value = a/a+b• Negative predictive value = d/c+d• Accuracy = (a+d) / (a+b+c+d)• Positive likelihood ratio = a/(a+c) / b/(b+d)• Negative likelihood ratio = c/(a+c) / d/(b+d)
Example(Ann Int Med 2003;138:787‐794)
• It is important to diagnose (deep vein thrombosis) DVT
• The authors compared• the USC (compression ultrasonography ‐ GS) with
• the D‐dimer test (DD)
• In 254 patients• 55 patients with DVT had a DD +• 1 with DVT had a DD ‐• 198 patients without DVT had DD +• 302 patients without DVT had a DD ‐.
TVP (USC and 3 month followup) Total
Disease present(test +)
Disease absent(test ‐)
D‐dimer results Test + 55 198 253
Test ‐ 1 302 303
Total 56 500 556
• Sensibility = a/a+c = 55/56 = 0,98 = 98%• Specificity = d/b+d = 302/ 500 = 0,60 = 60%• Positive predictive value = a/a+b = 55/253 = 0,22 = 22%• Negative predictive value = d/c+d = 302/303 = 0,9967 = 100%• Accuracy = (a+d) / (a+b+c+d) = 357/556 = 0,64 = 64%• Positive likelihood ratio = a/(a+c) / b/(b+d) = 0,98/0,4 = 2,45• Negative likelihood ratio = c/(a+c) / d/(b+d) = 0,017/0,6 = 0,03
Interpretation
• Only 22 out of 100 d‐dimer + patients had DVT• Almost no patient with negative d‐dimer had DVT• 64 out of 100 patients were correctly classified using the d‐dimer test
• A patient with an abnormal d‐dimer result has 2.5 more chances of having DVT than one with normal d‐dimer.
If I prescribe this medicine, how likely is it that the patient gets
better?
Table 2 x 2 and its (dichotomic) results derived from clinical trials
OutcomeYes No
Exposure Yes a b a + bNo c d c + d
a + c b + d
• ABSOLUTE RISK (AR) = [a / (a + b)] and [c / (c + d)]• ABSOLUTE RISK REDUCTION (ARR) = [c / (c + d)] ‐ [a / (a + b)]• NUMBER NEEDED TO TREAT (NNT) = (1 / RRA) x 100• RELATIVE RISK (RR) = [a / (a + b)] / [c / (c + d)]• RELATIVE RISK REDUCTION (RRR) = [c / (c + d)] ‐ [a / (a + b)] / [c / (c + d)]
Bisoprolol(EER)
Placebo(CER)
RRR ARR NNT
Global mortality 12% 17% 29% 5% 20
CIBIS II TRIAL(Lancet 1999;353:9‐13)
RRR = (CER‐EER)/CER = (17‐12)/17 = 0,29ARR = CER‐EER = 17‐12 = 5NNT = 1/ARR = 1/5 x 100 = 20
If the patient has this risk factor, how likely is he to have the disease
in the future?
Measures of association in case control studies
Outcome present Outcome absent Total
Exposed a b a+b
Not exposed c d c+d
Total a+c b+d a+b+c+d
• An odd is a relation between the probability of occurrence of a given event and that of non‐occurrence of that event
• Odds: a / b for cases and c / d for controls• ODDS RATIO (OR): (a / b) / (c / d) or ad / bc• If the OR is positive this means that the probability of a result
in an exposed is X times greater than the unexposed.
Risk of lung cancer with smoking (10 years)
LC + LC ‐ Total
Smokers 120 10500 10620
Non smokers 4 11000 11004
Total 124 21500 21624
LUNG CANCER RISK (LC) WITH SMOKING• Of 10,620 smokers, 120 have LC• Of 11,004 non‐smokers, 4 have LC
Measures of association in case‐control studies
LC + LC ‐ Total
Smokers 120 10500 10620
Non smokers 4 11000 11004
Total 124 21500 21624
• Odds: a/b for cases and c/d for controls• Odds Cases: 120/10500 = 0.01143• Odds controls: 4/11000 = 0.00036• ODDS RATIO (OR): odds cases / odds controls = 0.01143 / 0.00036 = 31• The OR tells us that the probability of a smoker having LC is 31 times
higher than the non‐smoker.
Modern clinical practice is uncertain and risky, based on scientific evidence of variable quality, as well as in poorly reproducible clinical experiences.
António Vaz Carneiro, MD, PhD, FACP, FESC
António Vaz Carneiro, MD, PhD, FACP, FESC
Institute of Preventive Medicine and Public HealthInstitute for Environmental HealthCenter for Evidence Based Medicine
Cochrane PortugalFaculty of Medicine University of Lisbon
Lisbon Academic Medical CenterPORTUGAL
Telefs: +351 217 985 135 / 217 940 [email protected]
www.cembe.org
