A Simple Method A Simple Method for Evaluating the for Evaluating the Clinical LiteratureClinical Literature

““PP-ICONSPP-ICONS” ” approachapproach

Based on Robert J. Flaherty - Family Practice Based on Robert J. Flaherty - Family Practice Management – 5/2004Management – 5/2004

We need information that is both We need information that is both valid and relevant to our patientsvalid and relevant to our patients

Unfortunately, a great deal of Unfortunately, a great deal of research reported in journal articles research reported in journal articles is poorly done, poorly analyzed or is poorly done, poorly analyzed or bothboth

Several organizations can help us. Several organizations can help us. They develop clinical questions and They develop clinical questions and then review journal articles to then review journal articles to identify the best available evidenceidentify the best available evidence

Clinical EvidenceClinical Evidence Up-To-DateUp-To-Date Cochrane LibraryCochrane Library

The most relevant studies will involve The most relevant studies will involve outcomes that matter to patients (e.g., outcomes that matter to patients (e.g., morbidity, mortality and cost) versus morbidity, mortality and cost) versus outcomes that matter to physiologists outcomes that matter to physiologists (e.g., BP, BS or Chol levels)(e.g., BP, BS or Chol levels)

We have long assumed that improving the We have long assumed that improving the physiologic parameters of a disease will physiologic parameters of a disease will result in a better disease outcome, but result in a better disease outcome, but that is not necessarily truethat is not necessarily true classic classic example – Swan-Ganzexample – Swan-Ganz

Start with a Clinical Start with a Clinical VignetteVignette

Short case descriptionShort case description

What made you look up this journal What made you look up this journal article?article?

Not a Case conferenceNot a Case conference

Which article to present?Which article to present?

Stick to the “Big five”Stick to the “Big five” Check if there is an editorialCheck if there is an editorial RCT is the "gold standard" in RCT is the "gold standard" in

medical researchmedical research Case reports, cohort studies and Case reports, cohort studies and

other research methods simply are other research methods simply are not good enough to use for making not good enough to use for making patient care decisionspatient care decisions

the Big Fivethe Big Five

Impact factor - NEJM tops the Impact factor - NEJM tops the listlist

What is PP-ICONS?What is PP-ICONS?

When you find an article you need to When you find an article you need to determine whether it is valid and determine whether it is valid and relevantrelevant

Simple but effective way to identify a Simple but effective way to identify a valid or relevant article within a valid or relevant article within a couple of minutescouple of minutes

"PP-ICONS""PP-ICONS"

ProblemProblem Patient or populationPatient or population InterventionIntervention ComparisonComparison OutcomeOutcome Number of subjectsNumber of subjects StatisticsStatistics

ProblemProblem

Problem refers to the clinical Problem refers to the clinical condition that was studied. condition that was studied.

From the abstract, it should be clear From the abstract, it should be clear that the researchers studied the that the researchers studied the same problem you are interested insame problem you are interested in

If the problem studied were not If the problem studied were not sufficiently similar to your clinical sufficiently similar to your clinical problem, the results would not be problem, the results would not be relevant.relevant.

Patient or populationPatient or population

Is the study group similar to your Is the study group similar to your patient or practice? Are they primary patient or practice? Are they primary care patients are they patients who care patients are they patients who have been referred to a tertiary care have been referred to a tertiary care center? Are they of a similar age and center? Are they of a similar age and gender? gender?

If the patients in the study are not If the patients in the study are not similar to your patient, for example if similar to your patient, for example if they are sicker, older, a different they are sicker, older, a different gender or more clinically complicated, gender or more clinically complicated, the results might not be relevant.the results might not be relevant.

InterventionIntervention

The intervention could be a The intervention could be a diagnostic test or a treatmentdiagnostic test or a treatment

Make sure the intervention is the Make sure the intervention is the same as what you are looking forsame as what you are looking for

ComparisonComparison

The comparison is what the The comparison is what the intervention is tested againstintervention is tested against

It could be a different diagnostic test It could be a different diagnostic test or another therapyor another therapy

It could even be placebo or no It could even be placebo or no treatmenttreatment

Make sure the comparison fits your Make sure the comparison fits your questionquestion

OutcomeOutcome

The outcome is particularly importantThe outcome is particularly important Many outcomes are "disease-oriented Many outcomes are "disease-oriented

outcomes," which are based on "disease-outcomes," which are based on "disease-oriented evidence" (DOEs)oriented evidence" (DOEs)

DOEs usually reflect changes in physiologic DOEs usually reflect changes in physiologic parameters, e.g. BP, BS, Chol., etcparameters, e.g. BP, BS, Chol., etc

We have long assumed that improving the We have long assumed that improving the physiologic parameters of a disease will physiologic parameters of a disease will result in a better disease outcome, but that result in a better disease outcome, but that is not necessarily trueis not necessarily true

DOEs look at the kinds of outcomes that DOEs look at the kinds of outcomes that physiologists care aboutphysiologists care about

More relevant are outcomes that patients More relevant are outcomes that patients care about, often called "patient-oriented care about, often called "patient-oriented outcomes.“ They look at outcomes such as outcomes.“ They look at outcomes such as morbidity, mortality and cost.morbidity, mortality and cost.

Thus, when looking at a journal article, Thus, when looking at a journal article, DOEs are interesting but of questionable DOEs are interesting but of questionable relevance, whereas POEMs are very relevance, whereas POEMs are very interesting and very relevantinteresting and very relevant

NumberNumber

The number of subjects is crucial to whether The number of subjects is crucial to whether accurate statistics can be generated from the accurate statistics can be generated from the datadata

Too few patients in a research study may not be Too few patients in a research study may not be enough to show that a difference actually exists enough to show that a difference actually exists between the intervention and comparison between the intervention and comparison groups (known as the "power" of a study)groups (known as the "power" of a study)

Many studies are published with less than 100 Many studies are published with less than 100 subjects, which is usually inadequate to provide subjects, which is usually inadequate to provide reliable statistics. A good rule of thumb is 400 reliable statistics. A good rule of thumb is 400 subjectssubjects

StatisticsStatistics

The statistics you are interested in The statistics you are interested in are few in number and easy to are few in number and easy to understandunderstand

Since statistics are frequently Since statistics are frequently misused in journal articles, it is misused in journal articles, it is worth a few minutes to learn which worth a few minutes to learn which to believe and which to ignoreto believe and which to ignore

Some important statistics Some important statistics The Good, the Bad and the The Good, the Bad and the

UglyUgly

The GoodThe Good

Absolute risk reduction (ARR)Absolute risk reduction (ARR) The difference between the control group's The difference between the control group's

event rate (CER) and the experimental group's event rate (CER) and the experimental group's event rate (EER)event rate (EER)

Number needed to treat (NNT)Number needed to treat (NNT) Number of patients that must be treated to Number of patients that must be treated to

prevent one adverse outcome or for one prevent one adverse outcome or for one patient to benefitpatient to benefit

NNT is the inverse of the ARR; NNT = 1/ARR.NNT is the inverse of the ARR; NNT = 1/ARR.

The BadThe Bad Relative risk reduction (RRR)Relative risk reduction (RRR) Percent reduction in events in the treated Percent reduction in events in the treated

group compared to the control group event group compared to the control group event raterate

RRR is not a good way to compare outcomesRRR is not a good way to compare outcomes It amplifies small differences and makes It amplifies small differences and makes

insignificant findings appear significantinsignificant findings appear significant Nevertheless, the RRR is very popular and will Nevertheless, the RRR is very popular and will

be reported in nearly every journal article, be reported in nearly every journal article, perhaps because it makes weak results look perhaps because it makes weak results look goodgood

Ignore the RRR. It will mislead you.Ignore the RRR. It will mislead you.

ExampleExample

Tx reduces the risk of a bad eventTx reduces the risk of a bad event Beta-blockers to prevent deaths in high-risk Beta-blockers to prevent deaths in high-risk

patients with recent MI patients with recent MI

Relative risk reduction (RRR): Relative risk reduction (RRR): CER-CER-EER/CER EER/CER (.66 -. 50)/.66 = .24 or 24 percent (.66 -. 50)/.66 = .24 or 24 percent

Absolute risk reduction (ARR): CER-EER Absolute risk reduction (ARR): CER-EER (.66 - .50) = .16 or 16 percent (.66 - .50) = .16 or 16 percent

Number needed to treat (NNT): 1/ARR Number needed to treat (NNT): 1/ARR 1/.16 1/.16 = 6= 6

The Good againThe Good again

Absolute risk reductionAbsolute risk reduction AAR is the difference in the outcome AAR is the difference in the outcome

event rate between the control group event rate between the control group and the experimentally treated groupand the experimentally treated group

Unlike the RRR, the ARR does not Unlike the RRR, the ARR does not amplify small differences but shows the amplify small differences but shows the true difference between the true difference between the experimental and control interventionsexperimental and control interventions

Number needed to treat Number needed to treat (NNT)(NNT)

Single most clinically useful statistic Single most clinically useful statistic NNT is the number of patients who NNT is the number of patients who

must be treated to prevent one adverse must be treated to prevent one adverse outcomeoutcome

To think about it another way, the NNT To think about it another way, the NNT is the number of patients who must be is the number of patients who must be treated for one patient to benefittreated for one patient to benefit

The rest who were treated obtained no The rest who were treated obtained no benefit, although they still suffered the benefit, although they still suffered the risks and costs of treatmentrisks and costs of treatment

What is a reasonable What is a reasonable NNT?NNT?

In a perfect world, a treatment would In a perfect world, a treatment would have an NNT of 1, meaning that every have an NNT of 1, meaning that every patient would benefit from the patient would benefit from the treatmenttreatment

Real life is not so kind Real life is not so kind Clearly, an NNT of 1 is great and an Clearly, an NNT of 1 is great and an

NNT of 1,000 is terribleNNT of 1,000 is terrible NNT < 10 is good and NNT < 5 is NNT < 10 is good and NNT < 5 is

excellentexcellent

Note that NNTs for preventive Note that NNTs for preventive interventions will usually be higher interventions will usually be higher than NNTs for treatment than NNTs for treatment interventionsinterventions

NNT for prevention < 20 is goodNNT for prevention < 20 is good

NNT examplesNNT examples Triple antibiotic therapy to eradicate H. Triple antibiotic therapy to eradicate H.

pylori pylori 1.1 1.1 Statins for secondary prevention of Statins for secondary prevention of

adverse CVS outcomes adverse CVS outcomes 11 11 Statins for primary prevention of adverse Statins for primary prevention of adverse

CVS outcomes CVS outcomes 35 35 Finasteride to prevent one operation for Finasteride to prevent one operation for

BPH BPH 39 39 Misoprostol to prevent any GI Misoprostol to prevent any GI

complication in NSAIDs users complication in NSAIDs users 166166

Take Home MessageTake Home Message

Clinical VignetteClinical Vignette QuestionQuestion SearchSearch PP-ICONSPP-ICONS DiscussionDiscussion

EndEnd

Click to go back to SVCH website – Click to go back to SVCH website – svch.blogspot.comsvch.blogspot.com