Critical Appraisal 1.ppt

Dr Samira Alsenany

Dr SA 2012

Purpose of this seminar The purpose of this seminar is to provide

a brief overview of the critical appraisal process.

A critical review must identify the strengths and weaknesses in a piece of research and this should be carried out in a systematic manner (Eachus, 2003)

Dr SA 2012

Critical Appraisal“Critical appraisal is the process of

systematically examining research evidence to assess its validity, results, and relevance before using it to inform a decision” (Hill and Spittlehouse, 2001, p.1).

Dr SA 2012

Critical Appraisal Critical appraisal is an essential step in the

process of putting research into practice. Asking questions about an article’s research methodology, scrutinizing its data collection and analysis methods, and evaluating how its findings are presented will help you to determine whether that article’s conclusions should influence practical decision-making.

Dr SA 2012

When should you Critically Appraise?

• Conducting literature reviews for grant proposals.

• Evaluating the effectiveness, costs, and benefits of health programs.

• Establishing new health programs.

• Implementing policies; and Public health decision making, especially at the senior management level.

Dr SA 2012

What is criticalappraisal?

Critical appraisal is an essential part of evidence-based.

Critical appraisal allows us to make sense of research evidence and thus begins to close the gap between research and practice.

The most appropriate design for studying the effectiveness of a specific intervention or treatment.

Dr SA 2012

Appraising systematicreviews

Once you have conducted a literature search and obtained full text articles, you can begin the critical appraisal process.

Consider the following questions to assess the quality of the article

Dr SA 2012

Questions to AskWhat kind of reports do I want?

Is this article relevant to my issue and setting?

How much detail do I need?

How comprehensive do I need to be?

How far back should I search?

Dr SA 2012

Questions to AskIs it of interest?Why was it done? How was it done?What has been found?What are the implications?What else is of interest?

Dr SA 2012

Critically Appraise What You Read.Simple checklists enable the useful information to be identified.

Dr SA 2012

ChecklistsChecklists for particular types of

literature are a quick and easy way of learning critical appraisal.

They all have 3 stages:Basic questions.Essential appraisal. Detailed appraisal

Dr SA 2012

Critical Appraisal: Notes and Checklists

Available at: http://www.sign.ac.uk/methodology/checklists.html

Scottish Guidelines Network (SIGN) has developed a number of critical appraisal tools in support of their creation of evidence-based guidelines. These appraisal tools used by SIGN to conduct systematic reviews have been made available for public use on their website, including methodology evaluation checklists for each of the following 6 types of studies: systematic reviews/meta analyses, RCTs, cohort studies, case-control studies, diagnostic studies, and economic evaluations.

Dr SA 2012

Questions to AskThe abstract

Use the information found in the abstract to answer the questions below:

• Are your issues discussed there?• What are the main findings of the research?• Do you want to know more after reading the abstract?• Was the research done in a similar setting to yours?• Does it address a related question? (Even research

that covers your issue indirectly can be useful.)• Are there reasons to doubt the findings without

reading the whole article?

Dr SA 2012

Questions to Ask

You may conclude that the study is not reliable merely by reading the abstract.

If this is the case, move on to another article! Critical

Dr SA 2012

Questions to AskWhy was it done?

IntroductionShould a clear statement of the purpose of

the study.Focus on the time, place and circumstances of the

population and setting studied.How similar or different is the study population or setting to

yours? Is a difference likely to matter for the issue at hand?The absence of such statements can imply that the authors

had no clear idea of what they were trying to find out.

Or they didn’t find anything but wanted to publish!

Dr SA 2012

Questions to AskHow was it done?

Brief but should include enough detail to enable one to judge quality.

Must include who was studied and how they were recruited.

Basic demographics must be there. How the data was collected (interviews, focus

groups, questionnaires, surveys, observations, etc.).

Methods

Dr SA 2012

Questions to Ask Results

Three broad issues need to be considered the results of the study :

A Are the results of the study valid?B What are the results?C Will the results help locally

The major challenge.

What do I think this really means?

Dr SA 2012

Questions to AskResults

What has it found?The data should be there statistics.Is the sample large enough to produce significant

results? Is the effect size clinically or operationally relevant?

Are the aims in the introduction addressed in the results?

Look for illogical sequences, bland statements of results.

Flaws and inconsistencies.All research has some flaws , the

impact of the flaws need to assessed.Dr SA 2012

Questions to AskHow good are the measures?• Do the measures accurately reflect what the

researcher was trying to measure(validity)?• How clear and appropriate are these

measures? Too broad? Too narrow?• Are they actual measures or proxy measures?• Are the measures well established in either

prior research or through pilot testing?

Dr SA 2012

Questions to Ask Discussion

The discussion section is more detailed and precise than the abstract, and will explain:

A review of how these results compare or contrast with prior research? If this report found something different from.

The limitations of the research and possible implications which are not mentioned in

the abstract.

Dr SA 2012

Questions to Ask implications

What are the implications?Abstract / discussion.

The whole use of research is how far the results can be generalised.

All authors will tend to think their work is more important than the rest of us!

What is new here? What does it mean for health care? Is it relevant to my speciality?

Dr SA 2012

Recommended Reading

The following 3 reading series were published in core medical journals and contain in-depth

information on critical appraisal and evidence-based practice.

From the British Medical Journal (BMJ): How to Read a Paper: The Basics of Evidence-Based

MedicineAvailable at: http://www.bmj.com/collections/read.dtl Evidence-Based Nursing NotebookAvailable at: http://ebn.bmj.com/cgi/collection/notebookFrom the Journal of the American Medical Association (JAMA): User Guides to Medical LiteratureAvailable at: http://www.shef.ac.uk/scharr/ir/userg.html

Dr SA 2012

Online Critical Appraisal ToolsThe Centre for Methods and Tools (http://www.nccmt.ca/) is currently developing a

critical appraisal framework that is specifically tailored topublic health research. This guide will be the definitive critical appraisal resource for

public health practitioners in Canada.• AGREE InstrumentAvailable at: http://www.agreetrust.org/instrument.htm The purpose of the Appraisal of Guidelines Research & Evaluation (AGREE)

Instrumentis to provide a framework for assessing the quality of clinical practice guidelines. Itoriginates from an international collaboration of researchers and policy makers who

worktogether to improve the quality and effectiveness of clinical practice guidelines byestablishing a shared framework for their development, reporting and assessment.• A Beginner’s Guide to Judging Research StudiesAvailable at http://www.cihr-irsc.gc.ca/e/34192.htmlThis is a succinct guide to critical appraisal written by John Frank, Scientific Director atthe Canadian Institutes of Health Research. Although this article focuses on makingsense of media reports of new health research, the astute questions Frank poses can beapplied directly to research studies themselves.

Dr SA 2012

Online Critical Appraisal Tools• CASP Critical Appraisal ToolsAvailable at: http://www.phru.nhs.uk/Pages/PHD/resources.htmThe UK-based Critical Appraisal Skills Programme (CASP), a division

of the NHS’s Public Health Resources Unit, has developed a customized critical appraisal checklist for each common type of research study. Direct links to each of the 6 appraisal tools are

listed below: • Qualitative Research http://www.chsrf.ca/kte_docs/casp_qualitative_tool.pdf • Review Articles (including Systematic Reviews) http://www.phru.nhs.uk/Doc_Links/S.Reviews%20Appraisal

%20Tool.pdf • Case Control Studieshttp://www.phru.nhs.uk/Doc_Links/Case%20Control

%2011%20Questions.pdf

Dr SA 2012

Thanks

Dr SA 2012