Critical Appraisal.1.2.3,4

  • View

  • Download

Embed Size (px)

Text of Critical Appraisal.1.2.3,4

  • 8/13/2019 Critical Appraisal.1.2.3,4


    Critical Appraisal

    On the step 3 EBM process, after we found the resources about the evidence,

    we have to do the critical appraisal that evidence for its validity (closeness to the

    truth), impact (size of the effect), and applicability (usefulness in our clinical


    he !ritically appraisin" that evidence to studies of the he #tility of

    diagnostic, he effect oftherapy, Prognosisof disease and Causative(etiolo"y of


    (1). Critical Appraisal for Diagnosis and Screening.

    $e have to answer three %uestions about dia"nostic tests&

    '. s this evidence about the accuracy of a dia"nostic test valid

    *. +oes this (valid) evidence show that this test can accurately distin"uish

    patients who do and do not have a specific disorder

    3. ow can apply this valid, accurate dia"nostic test to a specific patient

    he post-test probabilities after positive (upper curve) and ne"ative (lower

    curve) test results for the ran"e of possible pre-test probabilities of disease. he

    first %uestion, on validity, ass if we can believe the information in the "raph. he

    second %uestion, on importance, ass if the results show clinically worthwhile

    shifts in uncertainty (the further apart the post-test curves, the lar"er this shift).

    he third %uestion means we need to understand how the test results mi"ht

    chan"e our dia"nostic uncertainty on application, not only for the patients in the

    study but, more importantly, for a particular patient.

  • 8/13/2019 Critical Appraisal.1.2.3,4




    avin" found a possibly useful article about a dia"nostic test, how can we

    %uicly critically appraise it for its pro/imity to the truth he %uestions listed in

    able 3.* are for individual reports, but we can also apply them to the

    interpretation of a systematic review (overview) of several different studies of the

    same dia"nostic test for the same tar"et disorder.

    Is t'is eidence ao*t a diagnostic test alid&

    '. +eas*re,ent-was the reference (0"old1) standard measured independently,

    i.e. blind to our tar"et test

    *. !epresentatie- was the dia"nostic test evaluated in an appropriate

    spectrum of patients (lie those in whom we would use it in practice)

    3. Ascertain,ent-was the reference standard ascertained re"ardless of the

    dia"nostic test result

    (2ourth %uestion to be considered for clusters of tests of clinical prediction rules&

    was the cluster of tests validated in a second, independent "roup of patients)

    1. Measurement: Was there an independent, blind comparison with a

    reference (gold) standard?

    he patients in the study should have under"one both the dia"nostic test in

    %uestion (say an item of the history or physical e/amination, a blood test, etc.)and the reference or 0"old1) standard (an autopsy or biopsy or other confirmatory

    0proof1 that they do or do not have the tar"et disorder).

    ometimes investi"ators have a difficult time comin" up with clear-cut reference

    standards (e.". for psychiatric disorders), and we4ll want to "ive careful

  • 8/13/2019 Critical Appraisal.1.2.3,4


    consideration to their ar"uments 5ustifyin" the selection of their reference

    standard. Moreover, we caution you a"ainst the uncritical acceptance of

    reference standards, even when they are based on 0e/pert1 interpretations of

    biopsies6 for e/ample, in an Evidence Base Medicine note, 7enneth 2lemin"

    reported that the de"ree of a"reement over and above chance in readin" breast,

    sin and liver biopsies is less than 89:; he results of one test should not be

    nown to the fol who are applyin" and interpretin" the other. (2or e/ample, the

    decision to complete the biopsy should not be dependent on the result of the

    dia"nostic test under study, and the patholo"ist interpretin" the biopsy that

    comprises the reference standard for the tar"et disorder should be 0blind1 to the

    result of the blood test that comprises the dia"nostic test under study.) n this

    way, investi"ators avoid the conscious and unconscious bias that mi"ht

    otherwise cause the reference standard to be 0over-interpreted1 when the

    dia"nostic test is positive and 0under-interpreted1 when it is ne"ative.

    . !epresentati"e: Was the diagnostic test e"aluated in an appropriate

    spectrum of patients (li#e those in whom we would use it in practice)?

    +id the report include patients with all the common presentations of the tar"et

    disorder (includin" those with its early manifestations), and patients with other

    commonly confused dia"noses tudies that confine themselves to florid cases

    vs. asymptomatic volunteers (a dia"nostic 0case-control1 study) are useful only

    as a first crude chec of the test, because when the dia"nosis is obvious to the

    eye we don4t need any dia"nostic test. he really useful articles will be set in the

    dia"nostic dilemmas we face, and include patients with mild as well as severe

    symptoms, early as well as late cases of the tar"et disorder, an amon" both

    treated and untreated individuals.

    $. %scertainment: Was the reference standard ascertained regardless of

    the diagnostic test result?

  • 8/13/2019 Critical Appraisal.1.2.3,4


    $hen patients have a ne"ative dia"nostic test result, investi"ators are tempted

    to fore"o the reference standard, and when the latter is invasive or risy (e.".

    an"io"raphy) it may be wron" to carry it out on patients with ne"ative test results.

    o overcome this, many investi"ators now employ a reference standard for

    provin" that a patient does not have the tar"et disorder6 this re%uires that the

    patient doesn4t suffer any adverse health outcome durin" a lon" follow-up despite

    the absence of any definitive treatment (for e/ample, convincin" evidence that a

    patient with clinically suspected deep vein thrombosis did not have this disorder

    would include no ill-effect durin" a prolon"ed follow-up despite the absence of

    antithrombotic therapy).

    f the report we4re readin" fails one or more of these three test, we4ll need toconsider whether it has a fatal flaw that renders its conclusions invalid. f so, it4s

    bac to more searchin" (either now or later6 if we4re already used up our time for

    this wee, perhaps we can interest a collea"ue or trainee in tain" this on as an

    0educational prescription1 . On the other hand, if the report passes this initial

    scrutiny and we decide that we can believe its results, and we haven4t already

    carried out the second critical appraisal step of decidin" whether these results

    are important, the we can proceed to the ne/t section.




    Sensitiit2 specificit2 and li3eli'ood ratios

    n decidin" whether the evidence about a dia"nostic test is important, we will

    focus on the accuracy of the test in distin"uishin" patients with and without the

    tar"et disorder. $e4ll consider the ability of a valid test to chan"e our minds from

    what we thou"ht before the test (we4ll call that the 0pre-test1 probability of some

    tar"et disorder) to what we thin afterwards (we4ll call that the 0post-test1

  • 8/13/2019 Critical Appraisal.1.2.3,4


    probability of the tar"et disorder). +ia"nostic tests that produce bi" chan"es

    from pre-test to post-test probabilities are important and liely to be useful to us

    in our practice.

    +ata from& Luyatt L, O/man I+, ali M, et al. Len ntern Med '>>*6 @&'N8-83

    Hrevalence A (ac)?(abcd) A =9>?*[email protected]> A 3' :

    Hositive predictive value A a?(ab) A @3'?'99' A @3:

  • 8/13/2019 Critical Appraisal.1.2.3,4


    Je"ative predictive value A d?(cd) A '899?'[email protected]= A >8:

    ensitivity A a?(ac) A @3'?=9> A >9:

    pecificity A d?(bd) A '899?'@@9 A =8:


  • 8/13/2019 Critical Appraisal.1.2.3,4


    Is you can see from able 3.3, our patient4s result (C9 mmol?D) places her in the

    top row of the table, either in cell 0a1 or in cell 0b1. Qou mi"ht note from able 3.3

    that >9: of patients with iron deficiency have serum ferritin in the same ran"e as

    our patient a?(ab)P6 that property, the proportion of patients with the tar"et

    disorder. n dia"nostic studies, we sometimes see a description of the proportion

    of patients who don4t have the tar"et disorder and who have ne"ative (or normal)

    test results6 this is called the 0specificity1. 9:?'8: A C) to be seen in someone with iron deficiency anemia as insomeone without the condition6 that ratio is called the 0lielihood ratio1 for a

    positive test result (D

  • 8/13/2019 Critical Appraisal.1.2.3,4


    you did it ri"ht if you wind up with an answer for post-test probability that is

    identical to its e%uivalent, the positive predictive value)

    (6). Critical appraisal for rognostic

    $hether posed by our patients, collea"ues or ourselves, we fre%uently need

    to consider %uestions about pro"nosis. 2or e/ample, a patient newly dia"nosed

    with Ilzheimer4s dementia mi"ht as, 0$hat4s "oin" to happen to me1 Is

    clinicians, we mi"ht consider& hould we screen for microalbuminuria in a patient

    with diabetes mellitus hould a patient with microalbuminuria be treated with an

    an"iotensin-convertin" enz