WHOGUIDANCE FOR THE DEVELOPMENT OF EVIDENCE-BASED VACCINE RELATED

RECOMMENDATIONS

August 2011

Position Papers(PP) on Vaccine Preventable Diseases: 4 sections

• Introduction• Information on the respective disease (disease

epidemiology, the pathogen, the disease)

• Information on the available vaccines (composition, safety, immune response, efficacy and effectiveness, cost effectiveness, other relevant issue)

• WHO position on optimal vaccine use (based on best available evidence, social values, preferences)

Importance of quality of evidence

• Quality of study design crucial for conclusions

• RCTs are considered gold standard; faulty randomization or blinding may reduce the quality of an RCT below that of a well designed observational study.

• Quality of evidence reflects extent to which confidence in the estimation of effect is adequate to support a particular decision.

GRADE for assessing evidence

• Grading of Recommendations Assessment, Development and Evaluation (GRADE): adopted by WHO and 50 other organizations.

• WHO Vaccine Position Papers uses GRADE method to score quality of evidence.

• SAGE also make strategic recommendations without formal GRADE scoring but follow GRADE approach

GRADE Process: Systematic Literature Review

• 5 criteria (limitations in study design, inconsistency, indirectness,

imprecision, publication bias) used to downgrade quality of evidence .

• 3 criteria (magnitude of effect, dose-response gradient, ability of study to

limit biases ,control for confounding) used to upgrade quality of evidence.

• Based on this score & other factors (balance betn benefits &

risks, social values ,preferences, cost resources), recommendations

are made and scored as strong or weak.

• A strong recommendation can still occur with low or very low quality evidence — it is the net result of how all the other factors come into play that is important (Clear explanation would be needed)

GRADE Applicability

• GRADE tables are only applied to issues regarding the

effectiveness & safety of vaccines;generally created

for overall vaccine efficacy/effectiveness and safety

• Occasionally used for the duration of protection, schedule considerations, and use in subpopulations, such as specific age or risk groups or HIV-infected populations.

Key Activities involved in SAGE Recommendations

For EBR all steps always to be conducted except 5 i.e. formal scoring

Methodology for framing Questions for Systematic Review: PICO

• Patient Population

• Intervention of interest

• Comparator

• Outcomes of interestUse of rotavirus vaccines (Definition of questions)

• Population: healthy infants 2–6 mo, or HIV-children; • Intervention: 2 different vaccines, 2 or 3 doses; • Comparator: absence of vaccination, prevention (hygiene), oral rehydration; • Outcome: morbidity, hospitalization, consultations, parental work loss,

mortality, nosocomial infections

1. Definition of Questions

Task for working Group in the beginning

• Consensus on Key PICO questions for detailed literature review

• Questions of efficacy/effectiveness, safety and burden of the disease are key questions for developing recommendations for vaccine use

• Formulation of the questions to be graded is of utmost importance (validated by entire group)

An initial rating of the importance of outcomes should precede the review of the evidence.

1. Definition of Questions

Issues for Consideration for Reco.Epidemiologic Features of the Disease

1. Definition of Questions

Issues for Consideration for Reco. Clinical characteristics

1. Definition of Questions

Issues for Consideration for Reco.

Vaccine & Immunization Characteristics1. Definition of Questions

Issues for Consideration for Reco. Economic considerations

1. Definition of Questions

Issues for Consideration for Reco.

Other considerations

• Health-system opportunities and existence of, and interaction with, other existing intervention and control strategies.

• Social impacts.

• Legal considerations;

• ethical considerations.

1. Definition of Questions

Identification of critical question to which GRADE to be applied

• GRADE scoring needs to be applied only to critical questions, preferably five questions.

• The formal scoring is only appropriate for questions regarding an intervention (e.g. vaccine use) and not for disease burden, economic considerations or strategic recommendations (e.g. research gaps, decision to pursue an eradication goal, etc)

2. Identification of critical Questions

Systematic review of the literature

• Data may be published or unpublished & should be extracted using a data extraction tool

• RCTs are considered the gold standard for assessing the effect of an intervention; for vaccine effectiveness and safety observational studies, including outbreak investigations, disease, surveillance and post-market surveillance data represent important sources

3. Systemic review of literature

Identifying study limitations through risk of bias in a summary table

4. Review the quality of evidence

Checklists for reviewing study quality4. Review the quality of evidence

Scoring of the quality of evidence

• Studies enter GRADE system at a particular level:all RCTs enter at level 4 & observational studies and surveillance data enter at level 2 .

• Then downgraded (Limitation, Inconsistency, Indirectness, Imprecision, Reporting bias) or upgraded ( Large effect/strength of association, populatn effect, mitigated bias or confounding)

Final scores can’t exceed 4 points or drop <1.

5. Scoring the quality of evidenceCategorization of studies

Quality of evidence rating5. Scoring the quality of evidence

Quality of evidence rating

5. Scoring the quality of evidence

The GRADE tables explicitly provide the score of the outcomes critical to the recommendation

5. Scoring the quality of evidence

Template of a GRADE table used to score the quality of evidence

5. Scoring the quality of evidence

Levels of Evidence

Am Fam Physician 2002;65:251-8.