Critical Appraisal.1.2.3,4

8/13/2019 Critical Appraisal.1.2.3,4

1/42

Critical Appraisal

On the step 3 EBM process, after we found the resources about the evidence,

we have to do the critical appraisal that evidence for its validity (closeness to the

truth), impact (size of the effect), and applicability (usefulness in our clinical

practice).

he !ritically appraisin" that evidence to studies of the he #tility of

diagnostic, he effect oftherapy, Prognosisof disease and Causative(etiolo"y of

disorders).

(1). Critical Appraisal for Diagnosis and Screening.

$e have to answer three %uestions about dia"nostic tests&

'. s this evidence about the accuracy of a dia"nostic test valid

*. +oes this (valid) evidence show that this test can accurately distin"uish

patients who do and do not have a specific disorder

3. ow can apply this valid, accurate dia"nostic test to a specific patient

he post-test probabilities after positive (upper curve) and ne"ative (lower

curve) test results for the ran"e of possible pre-test probabilities of disease. he

first %uestion, on validity, ass if we can believe the information in the "raph. he

second %uestion, on importance, ass if the results show clinically worthwhile

shifts in uncertainty (the further apart the post-test curves, the lar"er this shift).

he third %uestion means we need to understand how the test results mi"ht

chan"e our dia"nostic uncertainty on application, not only for the patients in the

study but, more importantly, for a particular patient.


2/42

IS THIS EVIDENCE ABOT THE ACC!AC" O# A DIA$NOSTIC TEST

VA%ID&

avin" found a possibly useful article about a dia"nostic test, how can we

%uicly critically appraise it for its pro/imity to the truth he %uestions listed in

able 3.* are for individual reports, but we can also apply them to the

interpretation of a systematic review (overview) of several different studies of the

same dia"nostic test for the same tar"et disorder.

Is t'is eidence ao*t a diagnostic test alid&

'. +eas*re,ent-was the reference (0"old1) standard measured independently,

i.e. blind to our tar"et test

*. !epresentatie- was the dia"nostic test evaluated in an appropriate

spectrum of patients (lie those in whom we would use it in practice)

3. Ascertain,ent-was the reference standard ascertained re"ardless of the

dia"nostic test result

(2ourth %uestion to be considered for clusters of tests of clinical prediction rules&

was the cluster of tests validated in a second, independent "roup of patients)

1. Measurement: Was there an independent, blind comparison with a

reference (gold) standard?

he patients in the study should have under"one both the dia"nostic test in

%uestion (say an item of the history or physical e/amination, a blood test, etc.)and the reference or 0"old1) standard (an autopsy or biopsy or other confirmatory

0proof1 that they do or do not have the tar"et disorder).

ometimes investi"ators have a difficult time comin" up with clear-cut reference

standards (e.". for psychiatric disorders), and we4ll want to "ive careful


3/42

consideration to their ar"uments 5ustifyin" the selection of their reference

standard. Moreover, we caution you a"ainst the uncritical acceptance of

reference standards, even when they are based on 0e/pert1 interpretations of

biopsies6 for e/ample, in an Evidence Base Medicine note, 7enneth 2lemin"

reported that the de"ree of a"reement over and above chance in readin" breast,

sin and liver biopsies is less than 89:; he results of one test should not be

nown to the fol who are applyin" and interpretin" the other. (2or e/ample, the

decision to complete the biopsy should not be dependent on the result of the

dia"nostic test under study, and the patholo"ist interpretin" the biopsy that

comprises the reference standard for the tar"et disorder should be 0blind1 to the

result of the blood test that comprises the dia"nostic test under study.) n this

way, investi"ators avoid the conscious and unconscious bias that mi"ht

otherwise cause the reference standard to be 0over-interpreted1 when the

dia"nostic test is positive and 0under-interpreted1 when it is ne"ative.

. !epresentati"e: Was the diagnostic test e"aluated in an appropriate

spectrum of patients (li#e those in whom we would use it in practice)?

+id the report include patients with all the common presentations of the tar"et

disorder (includin" those with its early manifestations), and patients with other

commonly confused dia"noses tudies that confine themselves to florid cases

vs. asymptomatic volunteers (a dia"nostic 0case-control1 study) are useful only

as a first crude chec of the test, because when the dia"nosis is obvious to the

eye we don4t need any dia"nostic test. he really useful articles will be set in the

dia"nostic dilemmas we face, and include patients with mild as well as severe

symptoms, early as well as late cases of the tar"et disorder, an amon" both

treated and untreated individuals.

$. %scertainment: Was the reference standard ascertained regardless of

the diagnostic test result?


4/42

$hen patients have a ne"ative dia"nostic test result, investi"ators are tempted

to fore"o the reference standard, and when the latter is invasive or risy (e.".

an"io"raphy) it may be wron" to carry it out on patients with ne"ative test results.

o overcome this, many investi"ators now employ a reference standard for

provin" that a patient does not have the tar"et disorder6 this re%uires that the

patient doesn4t suffer any adverse health outcome durin" a lon" follow-up despite

the absence of any definitive treatment (for e/ample, convincin" evidence that a

patient with clinically suspected deep vein thrombosis did not have this disorder

would include no ill-effect durin" a prolon"ed follow-up despite the absence of

antithrombotic therapy).

f the report we4re readin" fails one or more of these three test, we4ll need toconsider whether it has a fatal flaw that renders its conclusions invalid. f so, it4s

bac to more searchin" (either now or later6 if we4re already used up our time for

this wee, perhaps we can interest a collea"ue or trainee in tain" this on as an

0educational prescription1 . On the other hand, if the report passes this initial

scrutiny and we decide that we can believe its results, and we haven4t already

carried out the second critical appraisal step of decidin" whether these results

are important, the we can proceed to the ne/t section.

DOES THIS VA%ID EVIDENCE DE+ONST!ATE AN I+O!TANT ABI%IT" O#

THIS TEST TO ACC!ATE%" DISTIN$ISH ATIENTS /HO DO AND

DON0T HAVE A SECI#IC DISO!DE!&

Sensitiit2 specificit2 and li3eli'ood ratios

n decidin" whether the evidence about a dia"nostic test is important, we will

focus on the accuracy of the test in distin"uishin" patients with and without the

tar"et disorder. $e4ll consider the ability of a valid test to chan"e our minds from

what we thou"ht before the test (we4ll call that the 0pre-test1 probability of some

tar"et disorder) to what we thin afterwards (we4ll call that the 0post-test1


5/42

probability of the tar"et disorder). +ia"nostic tests that produce bi" chan"es

from pre-test to post-test probabilities are important and liely to be useful to us

in our practice.

+ata from& Luyatt L, O/man I+, ali M, et al. Len ntern Med '>>*6 @&'N8-83

Hrevalence A (ac)?(abcd) A =9>?*8@> A 3' :

Hositive predictive value A a?(ab) A @3'?'99' A @3:


6/42

Je"ative predictive value A d?(cd) A '899?'8@= A >8:

ensitivity A a?(ac) A @3'?=9> A >9:

pecificity A d?(bd) A '899?'@@9 A =8:

D


7/42

Is you can see from able 3.3, our patient4s result (C9 mmol?D) places her in the

top row of the table, either in cell 0a1 or in cell 0b1. Qou mi"ht note from able 3.3

that >9: of patients with iron deficiency have serum ferritin in the same ran"e as

our patient a?(ab)P6 that property, the proportion of patients with the tar"et

disorder. n dia"nostic studies, we sometimes see a description of the proportion

of patients who don4t have the tar"et disorder and who have ne"ative (or normal)

test results6 this is called the 0specificity1. 9:?'8: A C) to be seen in someone with iron deficiency anemia as insomeone without the condition6 that ratio is called the 0lielihood ratio1 for a

positive test result (D


8/42

you did it ri"ht if you wind up with an answer for post-test probability that is

identical to its e%uivalent, the positive predictive value)

(6). Critical appraisal for rognostic

$hether posed by our patients, collea"ues or ourselves, we fre%uently need

to consider %uestions about pro"nosis. 2or e/ample, a patient newly dia"nosed

with Ilzheimer4s dementia mi"ht as, 0$hat4s "oin" to happen to me1 Is

clinicians, we mi"ht consider& hould we screen for microalbuminuria in a patient

with diabetes mellitus hould a patient with microalbuminuria be treated with an

an"iotensin-convertin" enzyme inhibitor and will this chan"e his pro"nosis


9/42

baseline data), althou"h trial participants may not be entirely representative of

the population with a disorder. !aseGcontrol studies (in which investi"ators

retrospectively assess pro"nostic factors by determinin" the e/posures of cases

who have already suffered the outcome of interest and controls who have not)

are particularly useful when the outcome is rare or the re%uired follow-up is lon".

Irrivin" at a dia"nosis or pro"nosis rarely relies on a sin"le si"n, symptom, or

laboratory test. Occasionally, when completin" our literature search, we find

articles describin" tools that %uantify the contributions that the clinical e/am,

laboratory, and radiolo"ical investi"ations mae in establishin" a dia"nosis or a

pro"nosis for a patient. hese tools that combine dia"nostic and pro"nostic

information are called 0clinical prediction "uides1 (!HLs).

$e have to answer three %uestions about Hro"nosis tests&

'. s this evidence about the Hro"nosis test "alid

*. s this valid evidence about pro"nosis important

3. !an we appl& this valid, important evidence about pro"nosis to our

patient

ER+EJ!E IBO# E H


10/42

1. Was a defined, representati"e sample of patients assembled at a

common point in the course of their disease?

deally, the pro"nosis study we find would include the entire population of

patients who ever lived who developed the disease, studied from the instant it

developed. #nfortunately, this is impossible and we4ll have to determine how

close the report we4ve found is to the ideal with respect to how the tar"et disorder

was defined and how participants were assembled. f the study sample fully

reflects the spectrum of illness we find in our own practice, we are reassured.

owever, considerin" our clinical scenario, if the study that we found included

only patients from a specialty cardiolo"y practice, we mi"ht not be satisfied that

the sample is representative of the patients that we4re interested in. he study

should also describe the standardized criteria that were used to dia"nose the

tar"et disorder.

But from what point in the disease should patients be followed f

investi"ators be"in tracin" outcomes only after some patients have already

finished their course with the disease, then the outcomes for these patients mi"htnever be counted. ome would have recovered %uicly, while others mi"ht have

died %uicly. o, to avoid missin" outcomes by 0startin" the cloc1 too late, we

loo to see that study patients were included at a uniformly early time in the

disease, ideally when it first becomes clinically manifest6 this is called an

0inception cohort1. I study that assembled patients at any defined, common point

in their disease may provide useful information if we want information only about

that sta"e of the disease. owever, if observations were made at different points

in the course of disease for various people in the cohort, the relative timin" of

outcome events would be difficult to interpret. 2or e/ample, it would be difficult to

interpret the results from a study desi"ned to determine the pro"nosis of patients

with rheumatoid arthritis that included patients with newly dia"nosed disease as

well as those who had had the disease for '9 years or more. deally, we4d lie to


11/42

find a study in which participants are all at a similar sta"e in the course of the

same disease.

Information about the study type and sampling method is usually found in the abstract andmethods sections of the article. The study that we found is an inception cohort includingpatients with asymptomatic mitral valve prolapse.

. Was the follow+up of the stud& patients sufficientl& long and complete?

deally, every patient in the cohort would be followed until they fully recover or

develop one of the disease outcomes. f follow-up is short, it may be that too few

patients develop the outcome of interest and therefore we wouldn4t have enou"h

information to help us when advisin" our patients, and in this case we4d better

loo for other evidence. n contrast, if after years of follow-up, only a few adverse

events have occurred, this "ood pro"nostic result is very useful in reassurin" our

patients about their future.

he more patients who are unavailable for follow-up, the less accurate the

estimate of the ris of the outcome will be. he reasons for their loss are crucial.

ome losses to follow-up are both unavoidable and mostly unrelated to

pro"nosis (e.". movin" away to a different 5ob) and these are not a cause for

worry, especially if their numbers are small. But other losses mi"ht arise because

patients die or are too ill to continue follow-up (or lose their independence and

move in with family), and the failure to document and report their outcomes will

reduce the validity of any conclusion the report draws about their pro"nosis.

hort of findin" a report that ept trac of every patient, how can we 5ud"e

whether follow-up is 0sufficiently complete1 here is no sin"le answer for allstudies, but we offer some su""estions to help. In analysis showin" that the

baseline demo"raphics of the patients who were lost to follow-up are similar to

those followed up provides some reassurance that certain types of participants

were not selectively lost, but such an analysis is limited by those characteristics

that were measured at baseline. nvesti"ators cannot control for unmeasured


12/42

traits that may be important pro"nostically, and that may have been more or less

prevalent in the lost participants than in the followed-up participants. $e su""est

considerin" the simple 08 and *91 rule& fewer than 8: loss probably leads to little

bias, "reater than *9: loss seriously threatens validity, and in-between amounts

cause intermediate amounts of trouble. $hile this may be easy to remember, it

may over-simplify clinical situations in which the outcomes are infre%uent.

Ilternatively, we could consider the 0best1 and 0worst1 case scenarios in an

approach that we4ll call a 0sensitivity analysis1. ma"ine a study of pro"nosis

wherein '99 patients enter the study, N die and 'C are lost to follow-up. I 0crude1

case-fatality rate would count the N deaths amon" the =N with full follow-up,

calculated as N?=NAN.=:. But what about the 'C who are lost ome or all of

them mi"ht have died too. n a 0worst case1 scenario, all would have died, "ivin"

a case-fatality rate of (N nown'C lost)A*9 out of (=N followed'C lost) A '99, or

*9?'99 (i.e. *9:), which is four times the ori"inal rate that we calculated; Jote

that, for the 0worst case1 scenario, we4ve added the lost patients to both the

numerator and the denominator of the outcome rate. On the other hand, in the

0best case1 scenario, none of the lost 'C would have died, yieldin" a case-fatality

rate of N out of (=N followed'C lost), or N?'99 (i.e. N:). Jote that, for the 0best

case1 scenario, we4ve added the missin" cases to 5ust the denominator. $hilethis 0best case1 of N: may not differ much from the observed N.=:, the 0worst

case1 of *9: does differ meanin"fully, and we4d probably 5ud"e that this study4s

follow-up was not sufficiently complete and threatens the validity of the study. By

usin" this simple sensitivity analysis, we can see what effect losses to follow-up

mi"ht have on study results, which can help us 5ud"e whether the follow-up was

sufficient to yield valid results.

In the study that we found, follow-up was 97% at a median of 5. years, so no threat tovalidity here!

$. Were obecti"e outcome criteria applied in a blind fashion?


13/42

+iseases affect patients in many important ways6 some are easy to spot and

some are more subtle. n "eneral, outcomes at both e/tremesTdeath or full

recoveryTare relatively easy to detect with validity, but assi"nin" a cause of

death is often sub5ective (as anyone who has completed a death certificate

nows;). n between these e/tremes are a wide ran"e of outcomes that can be

more difficult to detect or confirm, and where investi"ators will have to use

5ud"ment in decidin" how to count them (e.". readiness for return to wor, or the

intensity of residual pain). o minimize the effects of bias in measurin" these

outcomes, investi"ators should have established specific criteria to define each

important outcome and then used them throu"hout patient follow-up. $e4d want

to satisfy ourselves that they are sufficiently ob5ective for confirmin" the

outcomes we4re interested in. he occurrence of death is ob5ective, but 5ud"in"

the underlyin" cause of death is very prone to error (especially when it4s based

on death certificates), and can be biased unless ob5ective criteria are applied to

carefully "athered clinical information. But even with ob5ective criteria, some bias

mi"ht creep in if the investi"ators 5ud"in" the outcomes also now about the

patients4 prior characteristics. Blindin" is crucial if any 5ud"ment is re%uired to

assess the outcome because unblinded investi"ators may search more

a""ressively for outcomes in people with the characteristic(s) felt to be ofpro"nostic importance than in other individuals. n valid studies, investi"ators

main" 5ud"ments about clinical outcomes are ept 0blind1 to these patients4

clinical characteristics and pro"nostic factors.

In the mitral valve prolapse study that we found, the outcomes included total mortality andcause of death. "ause of death was ascertained by hospital notes, death reports, deathcertificates, and autopsy records, or by contacting the patients# physicians. It is not clear ifthe outcomes assessors were blinded.

-. f subgroups with different prognoses are identified, was there

adustment for important prognostic factors and "alidation in an

independent test set patients?


14/42

Hro"nostic factors are demo"raphic (e.". a"e, "ender), disease-specific (e.".

mitral valve prolapse with mitral re"ur"itation), or comorbid (e.". hypertension)

variables that are associated with the outcome of interest. Hro"nostic factors

need not be causalTand in fact they are often notTbut they must be stron"ly

associated with the development of an outcome to predict its occurrence. 2or

e/ample, althou"h mild hyponatremia does not cause death, serum sodium is an

important pro"nostic marer in con"estive heart failure (individuals with

con"estive heart failure and hyponatremia have hi"her mortality rates than heart

failure patients with normal serum sodium).*


15/42

that one of these methods has been applied before we tentatively accept the

conclusion about a different pro"nosis for the sub"roup of interest.

$e say 0tentatively1 because the statistics of determinin" sub"roup

pro"noses are all about prediction, not e/planation. hey are indifferent to

whether the pro"nostic factor is physiolo"ically lo"ical or a biolo"ically

nonsensical and random, non-causal %uir in the data (whether the patient lives

on the north or the south side of the street, or was born under a certain

astrolo"ical si"n). 2or this reason, the first time a pro"nostic factor is identified,

there is no "uarantee that it really does predict sub"roups of patients with

different pro"nosesTit could be the result of a chance difference in its

distribution between patients with different pro"noses. ndeed, if investi"atorswere to search for multiple potential pro"nostic factors in the same data set, a

few would be liely to emer"e on the basis of chance alone. he initial patient

"roup in which pro"nostic factors are found is called a 0trainin" set1 or 0derivation

set1. Because of the ris of spurious, chance nomination of pro"nostic factors, we

should loo to see whether the predictive power of such factors has been

confirmed in subse%uent, independent "roups of patients, termed 0test sets1 or

0validation sets1. o see if this was done, we4d loo for a statement in the study4s

methods section describin" a pre-study intention to e/amine this specific "roup

of pro"nostic factors, based on their appearance in a trainin" set or previous

study. f a second, independent study validates the predictive power of

pro"nostic factors, we have a very useful !HL of the sort that we met earlier in

this section and which were discussed fully in !hapter 3, but this time predictin"

our patient4s outcome after he or she is dia"nosed.

In our study, after ad$usting for age, se, and comorbid conditions, moderate-to-severe mitralregurgitation and e$ection fraction &5'% were found to be independent predictors ofcardiovascular mortality. The authors also identified several prognostic factors thatindependently predicted cardiovascular morbidity and mortality. The significance of theseprognostic factors was not confirmed in a validation set.


16/42

f the evidence about pro"nosis appears valid after considerin" the above

"uides, we can turn to e/aminin" its importance and applicability. But if we

answered 0no1 to the %uestions above, we4d be better off searchin" for other

evidence.

IS THIS VA%ID EVIDENCE ABOT !O$NOSIS I+O!TANT&

1. ow li#el& are the outcomes o"er time?

. ow precise are the prognostic estimates?

1. ow li#el& are the outcomes o"er time?

Once we4re satisfied that an article4s conclusions are valid, we can e/amine it

further to see how liely each outcome is over time. ypically, results from

pro"nosis studies are reported in one of three ways& as a percenta"e of survival

at a particular point in time (such as '-year or 8-year survival rates)6 as median

survival (the len"th of follow-up by which 89: of study patients have died)6 or as

survival curves that depict, at each point in time, the proportion (e/pressed as a

percenta"e) of the ori"inal study sample who have JO yet had a specified

outcome. n pro"nosis studies we often find results presented as 7aplanGMeier

curves, which are a type of survival curve.

2i"ure N.'shows four survival curves, each leadin" to a different conclusion. n

panel I of this fi"ure, virtually no patients have had events by the end of thestudy, which could mean that either pro"nosis is very "ood for this tar"et disorder

(in which case the study is very useful to us) or the study is too short (in which

case this study isn4t very helpful). n panels B, ! and +, the proportion of patients

survivin" to ' year (*9:) is the same in all three "raphs. Ind we could tell our

patients that their chance of survivin" for a year are *9:. owever, the median
http://127.0.0.1:1818/chaps/chap4-1.htm#reffig4-1http://127.0.0.1:1818/chaps/chap4-1.htm#reffig4-1


17/42

survival (point at which half will have diedTshown by the dashed line) is very

different& 3 months for panel B, vs. > months for the disorder in panel !. he

survival pattern is a steady, uniform decline only in panel +, and the median

survival here is appro/imately @.8 months. hese e/amples hi"hli"ht the

importance of considerin" median survival and survival curves in order to fully

inform our patient about pro"nosis.

. ow precise are the prognostic estimates?

Is we pointed out earlier, investi"ators study pro"nosis in a sample of

diseased patients, not the whole population of everyone who has ever had the

disease. Hurely by the play of chance, then, a study repeated '99 times amon"

different "roups of patients (even with identical entry characteristics) is bound to


18/42

"enerate different estimates of pro"nosis. n decidin" whether a "iven set of

pro"nostic results is important, we need some means of 5ud"in" 5ust how much

its results could vary by chance alone. he 0confidence interval1 provides the

ran"e of values that are liely to include the true estimate, and %uantifies the

uncertainty in measurement. By convention, the 0>8: confidence interval1 (>8:

!) is used6 this represents the ran"e of values within which we can be >8: sure

that the population value lies. he narrower the confidence interval, the more

assured we can feel about the result. Jote that, if survival over time is the

outcome of interest, earlier follow-up periods usually include results from more

patients than in later periods, so that survival curves are more precise (that is,

they provide narrower confidence intervals) earlier in the follow-up.

(rom our study, we found that, at a median follow-up of 5. years, mortality was )).5%.*oderate-to-severe mitral regurgitation +haard ratio )., 95% "I ).' to .'/ and e$ectionfraction &5'% +haard ratio 0., 95% "I ).'5 to ./ were independent predictors ofcardiovascular mortality.

CAN /E A%" THIS VA%ID2 I+O!TANT EVIDENCE ABOT !O$NOSIS

TO O! ATIENT&

1. s our patient so different from those in the stud& that its results cannot

appl&?

his "uide ass us to compare our patients with those in the article, usin"

descriptions of the study sample4s demo"raphic and clinical characteristics.

nevitably, some differences will turn up, so how similar is similar enou"h $e

recommend framin" the %uestion the other way& Ire the study patients so

different from ours that we should not use the results at all in main" predictions

for our patients 2or most differences, the answer to this %uestion is 0no1, and

thus we can use the study results to inform our pro"nostic conclusions.

. Will this e"idence ma#e a clinicall& important impact on our conclusions

about what to offer or tell our patient?


19/42

Evidence re"ardin" a person4s pro"nosis is clearly useful for decidin" whether or

not to initiate therapy, for monitorin" therapy that has been initiated, and for

decidin" which dia"nostic tests to order. f, for e/ample, the study su""ests an

e/cellent pro"nosis for patients with a particular tar"et disorder who didn4t

receive treatment, our discussions with patients would reflect these facts and

would focus on whether any treatment should be started. f, on the other hand,

the evidence su""ests that the pro"nosis is "loomy without treatment (and if

there are treatments that can mae a meanin"ful difference), then our

conversations with patients would reflect these facts and more liely lead us to

treatment. Even when the pro"nostic evidence does not lead to a treat?don4t treat

decision, valid evidence can be useful in providin" patients and families with the

information they want about what the future is liely to hold for them and their

illness.

1eturning to our patient, transthoracic echocardiography revealed mild mitral regurgitationand an e$ection fraction of 235%. 4iven her age +&5' years/ and the absence of the otherprognostic factors identified in the study that we found, we can reassure our patient that sheis considered low ris6 for cardiovascular morbidity and mortality, and that her outcome +withrespect to mitral valve prolapse/ will be similar to that of the general population.

Ifter we4ve "one to the trouble of retrievin" and appraisin" an article, sometimes

it4s nice to eep a copy of our appraisal. $e4ve provided on the accompanyin"

!+ an abbreviated version of our appraisal that you can download to your H+I.

REFERENCES

). 8vierinos (, 4ersh :, *elton , et al. ;atural history of asymptomatic mitral valve

prolapse in the community. "irculation 0''0< )'3= )55>3).0. *ettauer :, 1ouleau ?, :ichet @, et al. Aodium and water ecretion abnormalities incongestive heart failure= determinant factors and clinical implications. 8nn Intern *ed)93< )'5= )3)>7.

. 4uyatt 4, 1ennie @, eds. Bsers# 4uides to the *edical ?iterature. 8 *anual forCvidence-:ased "linical Dractice. "hicago= 8*8 Dress, 0''0.

. Aac6ett @?, Eaynes 1:, 4uyatt 4E, Tugwell D. "linical Cpidemiology. 8 :asic Aciencefor "linical *edicine, 0nd edn. :oston= ?ittle, :rown, )99).


20/42

(7). Critical appraisal for ca*satie st*d (Har,)8 etiolog

$e cannot pic up a newspaper or 0surf the net1 without bein" bombarded with

concerns about potentially harmful interventions. hese concerns lead to the posin" of%uestions by both us and our patients, such as& +oes livin" close to hydroelectric

power lines increase the ris of cancer, +o statins cause cancer, +oes e/posure to

aluminum cause Ilzheimer4s dementia, +o elevated homocysteine levels cause

coronary artery disease Ind, we must mae 5ud"ments about whether these medical

interventions and environmental a"ents could be harmful to our patients.

o mae these 5ud"ments, we need to be able to evaluate evidence about causation

for its validity, importance, and direct relevance to our patients. Issessin" the validity

of the evidence is crucial if we are to avoid drawin" the false-positive conclusion that

an a"ent does cause an adverse event when, in truth, it does not, or the false-ne"ative

conclusion that an a"ent does not cause an adverse event when, in truth, it does.

!linical disa"reements about whether a "iven patient has had an adverse dru"

reaction are not uncommon, and 5ust because an adverse event occurred durin"

treatment, it does not inevitably follow that the adverse event occurred because of that

treatment.

he "uides in able C.'can help us to appraise the validity of an article about a

putative harmful a"ent. $e4ll consider the followin" clinical scenario to illustrate our

discussion.
http://127.0.0.1:1818/chaps/chap6-1.htm#reftab6-1http://127.0.0.1:1818/chaps/chap6-1.htm#reftab6-1


21/42

Table 6.1Is this evidence about harm validF

). Gere there clearly defined groups of patients similar in all important ways other thaneposure to the treatment of other causeF

0. Gere treatmentsHeposures and clinical outcomes measured in the same ways in bothgroupsF +Gays the assessment of outcomes either ob$ective or blinded to eposureF/

. Gas the follow-up of the study patients sufficiently long for the outcome to occur/ andcompleteF

. @o the results of the harm study fulfill some of the diagnostic tests for causationF

Is it clear that the eposure preceded the onset of the outcomeF

Is there a close-response gradientF

Is there any positive evidence from a dechallenge-rechallange studyF

Is the association consistent from study to studyF

@oes the association ma6e biological senseF

C%INICA% SCENA!IO+urin" a routine health maintenance visit, we see a N8-year-old woman whocomplains of ur"e incontinence which has "otten pro"ressively worse over the last *years, si"nificantly impactin" on her %uality of life. er past history is remarable forthree pre"nancies (she had ur"e incontinence associated with each of these), andforceps were used durin" two of the births. he taes lorazepam occasionally at ni"htfor difficulty sleepin". he is on no other medications and has no smoin" history.er caffeine intae consists of @89mD (*8 oz) of coffee per day. er clinicale/amination is unremarable e/cept for visible urine leaa"e when she is ased to

cou"h while in the lithotomy position. er post-void residual urine volume is *9mDand a urinalysis is unremarable. he recently read in the paper that caffeine cancause urinary incontinence and wants to now if this is true or if there are otherfactors that could be contributin" to her problem.


22/42

n response to this scenario, we posed the %uestion& n a woman with ur"e

incontinence, does caffeine consumption cause urinary incontinence #sin" the

!linical Uueries function of HubMed we were able to identify an article that mi"ht help

us to answer this %uestion.'

T"ES O# !EO!TS ON HA!+8ETIO%O$"

Is we discovered in !hapter 8, ideally the best evidence that we can find about the

effects of therapy (and putative harmful a"ents) comes from systematic reviews.

ndividual randomized trials are seldom lar"e enou"h to detect rare adverse eventswith precisionTemphasizin" the need to search for a systematic review. I systematic

review that combines all relevant randomized trials or cohort studies mi"ht provide us

with sufficiently lar"e numbers of patients to detect even rare adverse events. $hen

assessin" the validity of such a systematic review, we need to consider the "uides in

able C.'as well as those in able 8.>. #nfortunately, such reviews aren4t common,

thus the discussion in this chapter will focus on randomized trials, cohort, caseGcontrol

studies, and cross-sectional studies.

A!E THE !ES%TS O# THIS HA!+8ETIO%O$" STD" VA%ID&

1. Were there clearl& defined groups of patients, similar in all important wa&s

other than e/posure to the treatment or other cause?

deally, our search would yield a systematic review or a randomized trial in which

women had been allocated, by a system analo"ous to tossin" a coin, to caffeine

(amount of caffeine could be specified) consumption (the top row in able C.*, whose

total is a b), or some comparison intervention (decaffeinated bevera"es) or placebo

(the bottom row in able C.*, whose total is c d) and then followed over time for the

development of urinary incontinence.


23/42

for baseline differences in other putative causal a"ents between the "roups), so we4d

liely consider any statistically si"nificant increase in ur"e incontinence (ad5usted for

any important baseline differences) in the intervention "roup to be valid.


24/42

the ris of urinary incontinence. ncreasin" a"e is associated with urinary incontinence

and older women may be more liely to be usin" estro"en. herefore a"e could be a

confounder when comparin" the ris of urinary incontinence in women usin" and not

usin" estro"en. nvesti"ators must document the characteristics of both cohorts of

patients and either demonstrate their comparability or ad5ust for the confounders they

identify (usin" special statistical techni%ues such as multivariable analysis). Of course,

ad5ustments can only be made for confounders that are already nownand have been

measured, so we have to be careful when interpretin" cohort studies.W

f the outcome of interest is rare or taes a lon" time to develop (e.". the development

of cancer or asbestosis), even lar"e cohort studies may not be feasible and we willhave to loo for alternatives, such as caseGcontrol studies. n this study desi"n, people

with the outcome of interest (a c) are identified as 0cases1 and those without it (b d)

are selected as 0controls16 the proportion of each "roup who were e/posed to the

putative a"entTi.e. a?(a c) or b?(b d)Tis assessed retrospectively. here is even

more potential for confoundin" with caseGcontrol studies than with cohort studies

because confounders that are transient or lead to early death won4t even be able to be

measured. 2or e/ample, if patients are selected from hospital sources, the relationship

between outcome and e/posure will be distorted if patients who are e/posed are more

liely to be admitted to the hospital than are the une/posed. his was illustrated nicely

in a systematic review that looed at the association between vasectomy and prostate

cancer& the relative ris of prostate cancer followin" vasectomy was elevated in

hospital-based studies but not in population-based ones.

nappropriate selection of control participants can lead to false associations, and

0control1 participants should have the same opportunity for e/posure to the putative

a"ent as the 0case1 participants. 2or e/ample, if we found a caseGcontrol study

evaluatin" the association between benzodiazepines and urinary incontinence that

assembled women with urinary incontinence as cases but e/cluded patients with a

history of falls or an/iety disorder from the control "roup (who mi"ht be at increased

ris of e/posure to benzodiazepines), we4d be ri"ht to wonder whether an observed
http://top.banner.openfn%28%27ch06fn.htm/#FN2');http://top.banner.openfn%28%27ch06fn.htm/#FN2');


25/42

association was spurious.

$e can see that this study desi"n easily lends itself to the e/ploration of possible

relationships between many e/posures and the outcome of interest (especially with thecommon usa"e and availability of administrative databases). herefore we must bear

in mind that, if a lar"e number of potential associations are e/plored, a statistically

si"nificant findin" could be based on chance alone.

$hen we4re searchin" for an answer to an etiolo"y %uestion, the articles that we most

commonly find describe cross-sectional studies6 unfortunately, such studies are

susceptible to even more bias than caseGcontrol studies. n a cross-sectional study,

the authors would loo at a "roup of women with ur"e incontinence (a c) and a "roup

without (b d) and assess caffeine consumption (0a1 or 0b1) in both. E/posure and

outcomes are measured at the same time, and this hi"hli"hts one of the ma5or

problems with this study type& which came first Ilso, as with cohort and caseGcontrol

studies, ad5ustment must be made for confounders.

2inally, we may only find case reports of one patient (or a case series of a few

patients) who developed an adverse event while receivin" the suspected treatment

(cell 0a1 in able C.*). f the outcome is unusual and dramatic enou"h (phocomelia in

children born to women who too thalidomide), such case reports and case series may

be enou"h to answer our %uestion. But, because these studies lac comparison

"roups, they are usually only sufficient for hypothesis "eneration and thus hi"hli"ht the

need for other studies.

$e usually find information about the study type and how participants were selected in

the abstract and methods sections of the article. nformation describin" participants isoften found in the results section.

The article that we found describes a case>control study that established a group ofwomen with detrusor instability and a group without, and assessed caffeine consumption

in both groups. @etrusor instability was diagnosed if the women had symptoms of urgeincontinence and had evidence of bladder contraction and lea6age on urodynamic testing.

"ontrol participants had stress incontinence< ideally, we#d want the control group to


26/42

include age-matched continent women.

. Were treatments0e/posures and clinical outcomes measured in the same

wa&s in both groups? (Was the assessment of outcomes either obecti"e or

blinded to e/posure?)

$e should place "reater confidence in studies in which treatment e/posures and

clinical outcomes were measured in the same way in both "roups. Moreover, we would

prefer that the outcomes assessors were blind to the e/posure in cohort studies and to

the outcome and study hypothesis in caseGcontrol studies. !onsider a report

describin" a cohort study looin" at the association between coffee consumption and

urinary incontinence. $e4d be concerned if the investi"ator searched more

a""ressively for urinary incontinence in women who were nown to have heavy

caffeine consumption. ndeed, when the outcomes assessors aren4t blinded to the

e/posure, they may search harder for the disease in the e/posed "roup and identify

disease that mi"ht otherwise have been unnoticed. Jow consider a caseGcontrol study

evaluatin" the same potential association6 if the investi"ator is not blind to the outcome

or study hypothesis, they mi"ht loo harder for a history of heavy caffeine consumption

in women whom they now to have urinary incontinence. imilarly, women with urinary

incontinence mi"ht have considered their situation more carefully and may have

"reater ability or incentive to recall possible e/posure. hus, we4d feel more assured

about the study if the report described that the patients (and their interviewers;) were

blinded to the study hypothesis.

his discussion raises another, finer, point re"ardin" the source of the information

about the outcome or e/posure of interest. ometimes in articles we find that theinvesti"ators use medical records to see information about the e/posure or outcome

retrospectively, and, as clinicians who complete these records (and often have to use

them at a later date to dictate a dischar"e summary;), we have to as ourselves if we

consider this method sufficiently accurate. !onsider, for e/ample, the impact on a

study4s results if the lielihood of the data bein" recorded differs between the "roups.


27/42

imilarly, information from some administrative databases mi"ht not be as accurate as

that collected prospectively (althou"h, for certain types of information such as dru"

usa"e, a dru" claims database will provide more accurate information than patient or

physician recall;).

Information about measurement of the eposure or outcome is usually included in themethods and results sections. In the study that we found, participants in both the control

and the case groups were as6ed to complete a -hour voiding diary before urodynamictesting. This diary consisted of a structured inta6e and output diary to record the type

and amount of fluid. They were as6ed to record their inta6e of coffee, tea, cola, andcocoa using a measuring cup. The reproducibility of this diary was assessed in a random

sample of women after an interval of ) wee6. It does not state if the patients were awareof the study hypothesis. *edical charts were reviewed to obtain information about

potential confounders, including parity and smo6ing history.

$. Was the follow+up of the stud& patients sufficientl& long (for the outcome to

occur) and complete?

deally, we4d lie to see that no patients were lost to follow-up because these patients

may have had outcomes that would affect the conclusions of the study (in a cohort

study looin" at the association between urinary incontinence and caffeine

consumption, ima"ine the impact on its results if a lar"e number of women in the

caffeine cohort left the study6 we wouldn4t now if it was because they developed ur"e

incontinence and left the study to see treatment, or because they became frustrated

with the study). Is mentioned in !hapter 8, evidence-based 5ournals of secondary

publication such asACP Journal Cluband Evidence Based Medicineuse a *9: loss

to follow-up as an e/clusion criterion because it would be rare for a study to suffer

such a loss and not to have its results affected. Ind, we4d lie to see that the patients

were followed for an appropriate period of time. 2or e/ample, if we found a study of the

association between Ilzheimer4s dementia and aluminum e/posure that followed

patients for only a few wees, we wouldn4t be able to distin"uish a true- from a false-

ne"ative association.

-. *o the results of the harm stud& satisf& some of the diagnostic tests for


28/42

causation?

nvesti"ators may identify an association between an e/posure and outcome, but is the

e/posure causative here are a few 0dia"nostic tests for causation1 that can help us

with this concern, includin" the followin"&

s it clear that the e/posure preceded the onset of the outcome?

$e4d want to mae sure that the e/posure (e.". caffeine) occurred prior to the

development of the adverse outcome (e.". urinary incontinence).

s there a doseresponse gradient?

he demonstration of an increasin" ris (or severity) of the adverse event with

increasin" e/posure (increased dose and?or duration) to the putative causal a"ent

stren"thens the association. 2or e/ample, in a recent caseGcontrol study looin" at the

association between homocysteine and ischemic heart disease, for each 8Xmmol?D

increase in serum homocysteine level, there was a correspondin" increase in the risof ischemic heart disease.*n the caseGcontrol study that we found looin" at caffeine

consumption and urinary incontinence, people with "reater caffeine consumption

(FN99m"?day) had a hi"her ris of urinary incontinence than those with lower caffeine

consumption.

s there an& positi"e e"idence from a dechallengerechallenge stud&?

$e4d lie to see that the adverse outcome decreases or disappears when thetreatment is withdrawn and worsens or reappears when it is reintroduced.


29/42

s the association consistent from stud& to stud&?

f we were able to find multiple studies, or, better yet, a systematic review of the

%uestion, we could determine whether the association between e/posure and theadverse event is consistent from study to study. 2or e/ample, a recent systematic

review looed at the association between serum homocysteine levels and ischemic

heart disease and retrieved 39 relevant studies.3nterestin"ly, the authors noted that

the retrospective studies that used population controls found a stron" association, but

that the prospective studies observed a less impressive association. #nfortunately, we

weren4t able to find additional recent studies looin" at caffeine consumption and

urinary incontinence.

*oes the association ma#e biological sense?

f the association between e/posure and outcome maes biolo"ical sense (in terms of

pathophysiolo"y, etc.), a causal interpretation becomes more plausible. he authors of

our study theorize that caffeine has an e/citatory effect on detrusor smooth muscle,

possibly throu"h the release of intracellular calcium. (7eep in mind that this theory is

based on data from animal models;)

A!E THE VA%ID !ES%TS O# THIS HA!+ STD" I+O!TANT&

f the study we find fails to satisfy the first three minimum standards in able C.', we4d

probably be better off abandonin" it and continuin" our search. But if we are satisfied

that it meets these minimum "uides, we need to decide if the association between

e/posure and outcome is sufficiently stron" and convincin" for us to do somethin"

about it. By this, we mean looin" at the ris or odds of the adverse effect with (as

opposed to without) e/posure to the treatment6 the hi"her the ris or odds, the stron"er

the association and the more we should be impressed by it. $e can use the "uides in

able C.3to determine if the valid results of the study are important.
http://127.0.0.1:1818/chaps/chap6-1.htm#ref3http://127.0.0.1:1818/chaps/chap6-1.htm#reftab6-1http://127.0.0.1:1818/chaps/chap6-1.htm#reftab6-3http://127.0.0.1:1818/chaps/chap6-1.htm#ref3http://127.0.0.1:1818/chaps/chap6-1.htm#reftab6-1http://127.0.0.1:1818/chaps/chap6-1.htm#reftab6-3


30/42

Table 6.3 8re the valid results of this harm study importantF

). Ghat is the magnitude of the association between the eposure and outcomeF

0. Ghat is the precision of the estimate of the association between the eposure and theoutcomeF

1. What is the magnitude of the association between the e/posure and

outcome?

Is noted above, %uestions of etiolo"y can be answered by several different study

desi"ns. +ifferent study desi"ns re%uire different methods for estimatin" the stren"th

of association between e/posure to the putative cause and the outcome of interest. n

randomized trials and cohort studies, this association is often described by calculatin"the ris (or incidence) of the adverse event in the e/posed (or treated) patients relative

to that in the une/posed (or untreated) patients. his relative ris (


31/42

the proportion with the outcome in each. But, in caseGcontrol studies, we can4t

calculate the 9 of them had a history of caffeine consumption, then aA>9 and cA'9. f

'99 patients without the outcome are assembled and it is found that N8 of them

received the e/posure, then bAN8 and dA88 and the O< becomes&

his means that the odds of e/periencin" the adverse event for people who had a

history of caffeine consumption was '' times that of those who didn4t have the same

e/posure.

O< and


32/42

chapter, a well-done randomized trial is susceptible to less bias than either a cohort or

caseGcontrol study. herefore, we4d be satisfied with a smaller increase in ris from a

randomized trial than from a cohort or caseGcontrol study. Because cohort and, even

more so, caseGcontrol studies are susceptible to many biases, we want to ensure that

the O< is "reater than that which would result from bias alone. $e mi"ht not want to

label an O< from a caseGcontrol study as impressive unless it is FN for minor adverse

events, and we4d set this value at pro"ressively lower levels as the severity of the

adverse event increases. here is less potential bias in cohort studies and therefore

we mi"ht re"ard a relative ris of F3 as convincin" for more severe adverse events.

Hrofessor Des rwi" has provided another useful tip when looin" at the stren"th of the

association. t re%uires us to find a report that includes some ad5ustment for potential

confounders. e su""ests that we compare the unad5usted measure of association

with one in which at least one nown confounder has been ad5usted out. f this

ad5ustment produces a lar"e decline in the


33/42

rates&

2or an O< derived from a caseGcontrol study, the calculation is more comple/

(remember, we can4t determine 0incidence1 directly in a caseGcontrol study). ts scary

formula reads&

if O< '&

if O< F'&

where HEE< is the patient e/pected event rate (the adverse event rate amon"

individuals who are not e/posed to the putative cause). $e4ve made this easier for you

by cranin" out some typical HEE


34/42

2racticing 34M in real+time

he above formula is scary and we don4t use it very often. f the answer we4re looin"

for isn4t in able C.N, we use the EBM calculator to %uicly convert an O< to an JJ.

2irin" up the EBM calculator on the website (http&??www.cebm.utoronto.ca?) or from the

accompanyin" !+, we can insert our O< of 9.>9 and our HEE< of 9.998 and, at the

clic of a button, the JJ is calculated (see 2i"ure C.'). $e can also download this

calculator to our H+I (personal di"ital assistant) for %uic usa"e.

. What is the precision of the estimate of the association between the e/posure

and outcome?

n addition to looin" at the ma"nitude of the


35/42

confidence interval is close to ' and, as you will recall, when the O< appro/imates ',

the adverse event is no more liely to occur with than without the e/posure to the

suspected a"ent.

Once we have decided that the evidence we have found is both valid and important,

we need to consider if it can be applied to our patient (able C.C).

CAN THIS VA%ID AND I+O!TANT EVIDENCE ABOT HA!+ BE A%IED TO

O! ATIENT&

1. s our patient so different from those included in the stud& that its results

cannot appl&?

Is emphasized in previous chapters, the issue is not whether our patient fulfills all the

inclusion criteria for the study we found, but whether our patient is so different from

those in the study that its results are of no help to us. ee p. '3C for further discussion

of this issue.


36/42

. What is our patient5s ris# of benefit and harm from the agent?

o apply the results of a study to an individual patient, we need to estimate our

patient4s ris of the adverse event if she were not e/posed to the putative cause.here4s a hard way and an easy way to tacle this. he hard way re%uires searchin"

for "ood evidence on pro"nosis6 the much easier way re%uires estimatin" the ris

relative to that of une/posed individuals in the study. ust as with JJs in !hapter 8,

we can e/press this as a decimal fraction (called 0f1)& if our patient is at half the ris of

study patients, fA9.86 if our patient is at three times the ris, fA3. he study JJ can

then be divided by 0f1 to produce the JJ for our individual patient. 2or e/ample,

suppose a study found we4d only need to treat '89 people with a statin to cause one

additional cancer. But if we thin our patient is at twice the ris of the study patients,

fA* and '89?* "enerates an JJ of @8. f, on the other hand, we thou"ht our patient

was at one-third the ris (fA9.33), the JJ for patients lie ours becomes N88.

Table 6.6 4uides for deciding whether valid important evidence about harm can beapplied to our patient

). Is our patient so different from those included in the study that its resultscannot applyF

0. Ghat is our patient#s ris6 of benefit and harm from the agentF

. Ghat are our patient#s preferences, concerns, and epectations from thistreatmentF

. Ghat alternative treatments are availableF


37/42

n situations such as this when we4re considerin" the use of a medication, the JJ

needs to be balanced a"ainst the correspondin" JJ summarizin" the benefit of this

treatment. he resultin" crude 0lielihood of bein" helped vs. harmed1 (D, see !h.

8) by this treatment can provide the startin" point for the last step, described in the

ne/t section.

$. What are our patient5s preferences, concerns, and e/pectations from this

treatment?

t is vital that we incorporate our patient4s uni%ue concerns and preferences into any

shared decision-main" process. n the case of potentially harmful therapy, and 5ust as

in !hapter 8, we can as our patient to %uantify her values for both the potential

adverse event and the tar"et event we hope to prevent with the proposed therapy. he

result is a severity-ad5usted lielihood of bein" helped or harmed by the therapy. f we

are unsure of our patient4s baseline ris, or if she is unsure about her values for the

outcomes, a sensitivity analysis can be done. hat is, different values for relative

severity could be inserted, and we and our patient could determine at which point the

decision would chan"e.

-. What alternati"e treatments are a"ailable?

2inally, we and our patient could e/plore alternative mana"ement options. s there

another medication we could consider s there any effective non-pharmacolo"ical

therapy available

1eturning to our patient and having reviewed the evidence about caffeine consumption and urgeincontinence, she is considered to have heavy caffeine consumption. The bottom line from the

study suggests that heavy caffeine consumption may increase the ris6 of detrusor instability, butthis association appears wea6. Eowever, after discussion with our patient, she would li6e toconsider this option further in an attempt to reduce her symptoms, and together we have outlined aproposal to loo6 at an n-of-) study to determine if a reduction in caffeine consumption will alleviateher symptoms +see "h. 5 for detailed discussion of n-of-) studies/. Ge have also discussed otherris6 factors that were identified in studies and which may be contributing to her incontinence,including multiparity and history of forceps delivery. If no symptom resolution occurs withmodification of caffeine consumption, we#ve agreed to consider a trial of pelvic floor muscle trainingwhich is a non-pharmacological intervention that has been proven to be beneficial in a recentsystematic review.


38/42

!ACTICIN$ EB+ IN !EA%9TI+E

2ormulatin" a %uestion, and then retrievin", appraisin", and applyin" relevant

evidence from the primary literature will usually tae lon"er than the avera"e clinical

appointment. here are several ways to tacle the time constraints, focusin" on findin"

shortcuts to facilitate practicin" EBM. Is we4ve already mentioned (in !h. *), if we can

find our answer in a hi"h-%uality pre-appraised resource, we4re ahead of the "ame6

but, unfortunately, many of these resources don4t yet have a broad covera"e of topics,

so we may not find anythin" relevant. f we4re lucy to have a local librarian or

consultant pharmacist who can help us trac down the evidence, we4d as them for

some help. Ilternatively, we can share tass with other collea"ues, either locally or

virtually usin" e-mail discussion "roups. $e4d also consider asin" our patient to boo

a return visit in a wee to review the evidence with them at that time, "ivin" us some

time to review the evidence between visits. f the same clinical issues arise commonly,

we could develop our own patient information leaflets briefly summarizin" some of the

evidence. n the same way, we could eep our !Is (critically appraised topics) (with

e/piry dates;) on file for our own needs if we e/pect the clinical problem to arise a"ain.

2inally, dependin" on how interested our patient is in bein" part of the shared decision-

main" process, they could be involved in each step alon" the way.

A #INA% NOTE

Material in this chapter can be applied to %uestions about whether treatments do more

harm than "ood, as well as to %uestions of causation around lifestyle issues such as

caffeine consumption, and around environmental issues such as livin" close to

hydroelectric power lines. he principles are all the same, but, in lifestyle issues, the
http://top.banner.showabstract%28%279570171%27%29/http://top.banner.showabstract%28%2710862848%27%29/


39/42

harm is contrasted with the satisfaction or other "ains derived from pursuin" the

putatively 0unhealthy1 practice. Ind with environmental issues, discussion must

consider interventions to reduce the ris and their associated cost-effectiveness and

feasibility.

!E#E!ENCES

'. Irya DI, Myers +D, acson J+. +ietary caffeine intae and the ris for

detrusor instability& a caseGcontrol study. Obstet Lynecol *9996 >C& =8G>.

*. $ald J, $att !, Daw M>'.

Critical Appraisal ( 7 )
http://top.banner.showabstract%28%2712387654%27%29/http://top.banner.showabstract%28%2712387654%27%29/


40/42

On the step 3 EBM process, after we found the resources about the evidence,

we have to do the critical appraisal that evidence for its validity (closeness to the

truth), impact (size of the effect), and applicability (usefulness in our clinical

practice).

he !ritically appraisin" that evidence to studies of the he #tility of

diagnostic, he effect oftherapy, Prognosisof disease and Causative(etiolo"y of

disorders).

7). C!ITICA% A!AISA% #O! THE!A"


41/42

Ifter retrievin" evidence, we need to decide if it4s valid and important before we can

apply the evidence to our individual patients. he order in which we consider validity and

importance depends on individual preference. $e could start by appraisin" its validity,

ar"uin" that, if it isn4t valid, who cares whether it appears to show a hu"e effect

Ilternatively, we could determine its clinical importance, ar"uin" that, if the evidence

doesn4t su""est a clinically important impact, who cares if it4s valid $e can start with

either %uestion, so lon" as we remember to follow-up one favorable answer with the

other %uestion. o illustrate our discussion, we4ll consider the followin" patient.

C!N!CA SCENAR!"8 75-year-old man is seen in our office after being discharged from hospital 0 wee6s previously.

@uring this admission he underwent a carotid endarterectomy after suffering a transient ischemicattac6 +TI8/, and being diagnosed with significant carotid stenosis. Eis hospital stay wasuncomplicated and his discharge medications included metoprolol 5' mg twice daily for hypertension

and aspirin ) mg daily. Today, he has brought us an article from the Internet describing the benefitsof statins for stro6e prevention and he wonders what this medication is and if he should ta6e it. Jurnote from his last visit showed that his total cholesterol was 5mmolH?, E@?-cholesterol was0.'mmolH?, and ?@?-cholesterol was 0mmolH?. Eis eamination was unremar6able.

Based on this scenario, we posed the followin" %uestion& n a patient with history of I,

carotid endarterectomy, hypertension, and normal lipid profile, does therapy with a statin

decrease ris of stroe #sin" HubMed !linical Uueries (see !h. * for a description of

this search en"ine) we identified the recent M

'

which mi"ht help us answer this%uestion. ($e also found the M

ARE T#E RES$TS "F T#!S !N%!&!%$A ST$%' &A!%

'. $as the assi"nment of patients to treatment randomized*. $as the randomization concealed3. $ere the "roups similar at the start of the trialN. $as follow-up of patients sufficiently lon" and complete8. $ere all patients analyzed in the "roups to which they were randomized

ome finer points&C. $ere patients, clinicians, and study personnel ept blind to treatment@. $ere "roups treated e%ually, apart from the e/perimental therapy

1. Was the assignment of patients to treatment randomized?

#ntil recently, it was believed that hormone replacement therapy (


42/42
http://127.0.0.1:1818/chaps/chap5-1.htm#ref3http://127.0.0.1:1818/chaps/chap5-1.htm#ref4http://top.banner.openfn%28%27ch05fn.htm/#FN1');http://top.banner.openfn%28%27ch05fn.htm/#FN2');http://top.banner.openfn%28%27ch05fn.htm/#FN3');

Documents

Critical Appraisal.1.2.3,4