Critical Appraisal of a Diagnostic Test Study

Dr. Sherif Omar

Dr. Zaid Al-hamadani PGY3 Family Medicine ResidentDr. Omnia Darweesh PGY2 Family Medicine Resident

Basic structure for a diagnostic study?

Through the following steps• Series of patients (of interest)• New test• Reference (Gold) standard• Compare the results of the index test with the

reference standard, blinded

Critical appraisal process for the diagnostic test

Three easy steps

A/ Are the results valid?B/ What are the results?C/ Will they help me care of my patients?

1. Was there a clear question for the study to address? A question should include information about:

– the population – the test – the setting – the outcomes

A/ Are the results of the study valid?

Population: Sera from 787 consecutive patients tested at the VCT centre in September-October 2012 were included in the analysis.

Test (Intervention): Rapid tests versus ELISA for HIV screening

Setting: Voluntary Counselling and Testing Facility of a tertiary care centre in North India.

Outcomes: Our study highlights that RDTs fare poorly compared to ELISA as screening assays

2. Was there a comparison with an appropriate reference standard? HINT: Is this reference test(s) the best available indicator in the circumstances?

-All 787 sera were tested by RDTs (test in question) and all of them also tested by and compared to the standard screening method (ELISA).

-Results were also confirmed by doing Western Blot (confirmatory gold standard method).

3. Did all patients get the diagnostic test and the reference standard? Consider:

– Were both received regardless of the Results of the test of interest? – Check the 2 x 2 table

Yes all the population received the two tests in comparison.

4. Could the results of the test of interest have been influenced by the results of the reference standard? Consider:

– Was there blinding? – Were the tests performed independently? (Review bias)

-The evaluation was conducted in a blinded fashion with the RDTs and ELISA performed by different technical personnel and the status of the sera as per the RDT based algorithm not revealed to the personnel performing the ELISA.

5. Is the disease status of the tested population clearly described? Consider:

– Presenting symptoms – Disease stage or severity – Co-morbidity– Differential diagnoses (Spectrum bias)

-As the aim of the diagnostic tests is screening of HIV, which is done in an asymptomatic population, so the answer to these questions is “No”.

-HIV is a silent infection for many years, and thus the aim of screening is towards people representing the general population, for example:

•All people coming to live in Qatar regardless of age, sex, race or ethnicity•Pre-employment screening tests•Pre-marital screening tests•Military enrollment

6. Were the methods for performing the test described in sufficient detail? HINT: Was a protocol followed?

-The study itself was performed in three phases:-The staff were trained and pretest informed consent was taken.

Phase one:The first RDT (SD Bioline) applied for all sera, then the results were labeled as reactive or non reactive. Then the reactive tests were tested further by the two other RDTs.

Phase two:All sera then were tested by ELISA (standard screening test).

Phase three:All the positive tests from ELISA and RDTs were compared with Western Blot for confirmation of results.

7. What are the results? Consider:

– Are the sensitivity and specificity and/or likelihood ratios presented? – Are the results presented in such a way that we can work them out?

B/ If so, what are the results?

-Thirty-six serum samples were reactive by the first test. -On subsequent evaluation of all the 787 samples by Microlisa-HIV, 40 HIV reactive samples were identified (all confirmed as positive by western blot), 9 of which had been reported as nonreactive by SD Bioline HIV-1/2 3.0.

-Thus the first RDT had missed 9 (22.5%) HIV reactive samples (also confirmed to be positive by western blot) and its sensitivity on comparison with ELISA was 77.5%.

-In addition, SD Bioline HIV-1/2 3.0 registered 5 false positive results (negative by ELISA and western blot) giving a specificity of 99.3%.

8. How sure are we about these results? Consider:

– Could they have occurred by chance? – Are there confidence limits?– What are they?

-No Confidence Limit mentioned in the study

9. Can the results be applied to your patients/ the population of interest? HINT: Do you think you patients / population are so different from those in the study that the results cannot be applied? Such as age, sex, ethnicity and spectrum bias.

C/ Will the results help me and my patients/population?

10. Can the test be applied to your patient or population of interest? Consider:

– Think of resources and opportunity costs – Level and availability of expertise required to interpret the tests – Current practice and availability of services

11. Were all outcomes important to the individual or population considered? Consider:

– Will the knowledge of the test result improve patient wellbeing – Will the knowledge of the test result lead to a change in patient management?

12. What would be the impact of using this test on your patients/population?