CRITICAL APPRAISAL OF

LITERATURE

Rahmatina B. Herman Fakultas Kedokteran Universitas Andalas

Introduction

Research involves gathering, collecting and analyzing of data to produce meaningful information

Many of research did not perform in good quality

Even many of studies were biased and the results were untrue

It can lead to false conclusions

What is Critical Appraisal?

Critical appraisal is an important element of evidence-based medicine

Critical appraisal is the process of systematically examining research evidence to assess its validity, results, and relevance before using it to inform a decision (Hill and Spittlehouse, 2001)

Critical appraisal is the process of carefully and systematically examining the research paper to judge its trustworthiness, and its value and relevance in a particular context (Burls 2009)

..What is Critical Appraisal?

Critical appraisal is a systematic approach to the results of scientific papers:

- reading,

- understanding,

- interpreting,

- identifying the limitations,

- deciding upon the usefulness

..What is Critical Appraisal?

Critical appraisal is an essential step in the process of putting research into practice:

- Asking questions about an articles research methodology,

- Scrutinizing its data collection and analysis methods,

- Evaluating how the presented findings can help us to determine whether the conclusions can influence us in decision-making

..What is Critical Appraisal?

The aim of Critical Appraisal is helping people to develop the necessary skills to make sense of scientific evidence based on validity, results and relevance.

Why Do We Need Critical Appraisal Skill

Problem to obtain scientific information:

Vast and expanding literature.

Limited time to read.

Reason to read Different reasons to read mean different

strategies. - Keeping up to date. - Answering specific clinical questions. - Pursuing a research interest.

Stages of Obtaining Information

I. Clarify your purpose for reading

- Keeping up to date Skimming the main journals and summary bulletins

- Answering specific questions Finding good quality literature on the subject

- Pursuing a research interest Extensive literature searching.

..... Stages of Obtaining Information

II. Specify your information need

- What kind of reports do I want?

- How much detail do I need?

- How comprehensive do I need to be?

- How far back should I search?

The answers to these questions should flow from the reasons for reading

.....Stages of Obtaining Information

III. Identify relevant literature

- Selectivity is the key to successful critical appraisal

IV. Critically appraise what you read

- Separating gold from ore

- Time is limited, stop reading the unimportant information

- Simple checklists enable the useful information to be identified.

Kind of Information in A Research Paper

During Critical Appraisal, try to find the following information: 1. Research question 2. Study design 3. Selection issues 4. Outcome factors and study factors; method to measure 5. Confounders 6. Statistical method 7. Statistical Results 8. Conclusions 6. Study limitations 7. Application the findings into daily clinical practice or

research

An ideal abstract contains at least the red color information

Abstract

After you read the abstract:

Do you want to know more after reading the abstract?

Are there reasons to doubt the findings without reading the whole article?

Introduction Section

The introduction sections must help us to identify:

- key concepts,

- goals,

- subjects, and

- themes of the research.

Methodology Section

The Methodology will give us a step-by-step description of exactly how the study was carried out.

Study design: Different research questions require different

study designs.

For example the best design for studies evaluating the effectiveness of an intervention or treatment is a Randomized controlled trial

..Methodology Section

Where the study done

Data collection: - From whom data was collected - Primary or secondary data - How data collected - How good the data

How the study control for differences between the groups being compared

..Methodology Section

Sample: Is there any selection bias? In an intervention study, how were subjects

recruited and assigned to groups?

Studies are subject to bias and also confounders.

It is important that researchers take steps to minimize this bias and confounders.

For example use of a control group, randomization, blinding

Sample size to produce significant results

..Methodology Section

Methods to measure the outcome factors and study factors

- Whether there was any error in measurement

- Consider the sensitivity and specificity of each detection method

How good the measures:

- Validity

- Whether the measures well established in either prior research or through pilot testing by the researcher

..Methodology Section

What is statistical method used?

- Statistical method used should be appropriate and be able to identify and characterize the effect of potential factors

- The effects of confounder(s) should be controlled.

- Probability : Find the P values, and confidence interval (CI)

- Significances - significance or not significance means

- Bias play devils advocate

..Methodology Section

Whether important potential confounders were considered

For example: - You found a significant association between smoking

and arteriosclerosis risk - You want to compare the proportion of cigarette

smokers between arteriosclerosis cases and controls - You found more smokers in arteriosclerosis cases,

and more males in arteriosclerosis cases than controls - Since a proportion of smokers in males usually higher

than females, your finding might be distorted because of the different sex distribution

After adjusting the effect of sex distribution, there was no difference in

the association between smoking and GC risk.

Discussion and Conclusion Section

Must be detailed and precise in discussing the findings

How well are the results related to other research on the same topic?

- Is there a review of how these results compare or contrast with prior research?

- If this report found something different from previous research, then its important to question on appraising the reliability of the findings.

..Discussion and Conclusion Section

What are the authors conclusions?

- Whether the results reported in the conclusions consistent with what is reported in the tables

- Whether the interpretation consistent with what the actual findings

This section will also explain:

- The limitations of the research

- Possible implications or application of the findings into daily clinical practice or research

Example of Guides in CA

Are the results of the study valid? > Primary: - Whether the exposure status clearly defined and

measured - Whether the outcome of interest measured - What was the follow up time and was it

adequate to measure the outcome of interest? - Whether the outcome measured in the same

way for both exposed & not exposed > Secondary: - How much was the attrition? (Loss to follow up) - Are there confounders that the investigator did

not address?

..Example of Guides in CA

What were the results? - How strong is the association between exposure

and outcome?

- How precise is the estimates of the risk ? (Confidence Interval and p value for statistical test of significance)

Will the results help me? - Whether the results applicable to my

population?

- What is the magnitude of the risk ?

- Should I attempt to stop the exposure?