Embed Size (px)
DESCRIPTION
Failure to meet standards. T r u s t. Reform required. Unsatisfactory quality. Cannot trust. Failure to meet standards. T r u s t. Reform required. Unsatisfactory quality. Cannot trust. Failure to meet standards. T r u s t. Reform required. Unsatisfactory quality. - PowerPoint PPT Presentation
Citation preview
How to produce top quality national Guidelines
Thomy Tonia, MScGuidelines Methodologist
Today’s schedule
• Why do we need evidence-based guidelines• Step by step introduction to the GRADE
approach • How to adapt guidelines for use on a national
level
1990“Appropriateness Guidelines describe accepted indications for using particular medical interventions and technologies, ranging from surgical procedures to diagnostic tests”
2011“Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances"
2013“Clinical practice guidelines are statements that include recommendations intended to optimize patient care. They are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options”
Grading of Recommendations Assessment Development and Evaluation
The GRADE working group began in the year 2000 as an informal collaboration of people with an interest in addressing the shortcomings of present grading systems in health care. Our aim is to develop a common, sensible approach to grading quality of evidence and strength of recommendation
Why should *I* bother?
PICO
Clinica
l questi
on
Rate
importa
nce
Select
outcomes
Very low
Low
Modera
te
High
Formulate recommendations:•For or against (direction)•Strong or weak (strength)
By considering:Quality of evidenceBalance benefits/harms
Values and preferences
Revise if necessary by considering:Resource use (cost)
Quality
rating
outcomes
across
studies
OutcomeOutcomeOutcome
Outcome
Critical
Important
Critical
Not important
Gra
de
dow
n o
r up
Outcome
Important
Overa
ll q
ualit
y o
f evid
ence
- Separates quality of evidence (high, moderate, low, very low) and strength of recommendation (strong, weak/conditional)
- Systematic, explicit, transparent
- Considers patients’ values and preferences very important for local contexts!
Main characteristics
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Hints
Setting the scope manageable
Choosing the Guideline panel (content experts; methodologist; patients; primary care physicians) and Chair!
Guideline Panel
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Hints
Population
Intervention
Comparison
OutcomeIn patients hospitalized for COPD exacerbations is initial treatment with IV corticosteroids compared to oral corticosteroids better (reduction in length of hospital stay)?
Are IV corticosteroids effective for treating COPD exacerbations?
Population Intervention ComparisonOutcome
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Theory
1 PICO question/ recommendation
7-10 PICO questions/ Guideline
No PICO question, no recommendation!
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Hints
-Importance- NOT evidence driven
-Guided by patients’ needs and values
- What do younger doctors need guidance for?
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Theory
Q3- oral vs iv steroids AL CC OP RS Total
Treatment failure 9 5 9 9 8
Mortality 9 9 7 9 9
Hospital readmission 9 9 8 8 9
Length of hospital stay 9 5 6 9 7
QoL 4 6 4 1 4
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Hints
Systematic literature review
All relevant electronic databases
Handsearching of journals
Two assessors
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Theory
Pragmatic GRADE approach
Search for recent systematic reviews and build up on them
Search main database(s) only
One assessor
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Hints
Select studies according to predefined criteria
Extract outcomes of interest
Meta-analyse, when applicable
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Theory
RevMan
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Theory
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Hints
Grading per outcome and not per study!
Quality in GRADE means more than risk of bias
Expert opinion is not a type of evidence
A particular quality of evidence does not necessarily imply a particular strength of recommendation
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Hints
- Quality the extend of our confidence that the estimates of the effect arecorrectadequate to support a particular decision/ recommendation
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Hints
RCTs: high quality
Risk of bias
Inconsistency
Indirectness
Imprecision
Publication bias
Grading per outcome, not per study!
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Hints
Observational studies: low quality
Large magnitude of effect
Dose-response relation
All plausible confounding would result in an overestimate of effect
Grading per outcome, not per study!
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Hints
Risk of bias
• Lack of allocation concealment • Lack of blinding
• Large loss to follow-up
• No ITT
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Hints
Inconsistency
• Variability/ heterogeneity of results
• Possible reasons? (intervention, definition of outcomes, quality of studies etc)
• Similarity of point estimates
• Overlap of CIs
• Statistical criteria (p value for test for heterogeneity, I2)
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Theory
Inconsistency
-Differences in direction do not constitute a criterion for rating down!
-Sub-group analyses even if statistical heterogeneity is small
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Theory
Inconsistency
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Hints
Indirectness
• Generalisability, transferability,
applicability
• Differences in the components of PICO
questions
• Indirect comparison
• Differences in population (children/adults),
intervention (intravenous/oral), outcomes
of interest (surrogate outcomes)
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Hints
Imprecision
• Confidence interval
• Estimate of effect includes both
appreciable benefits and not
appreciable benefits (or even harms)
• If the recommendation would differ
if the upper vs the lower boundary
of the CI represented the truth,
consider rating down
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Hints
Publication bias
• Difficult to estimate!
• Failure of reporting studies that were undertaken
• Delayed reporting of negative trials
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Theory
•Risk higher when only a few
small studies that show positive
effect are available
Theory Hints
Guideline Panel
Recommendations
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Strong (we recommend…) vs
Conditional (we suggest….)
+ -
Benefits vs Downsides
Evidence Quality
Values and Preferences Costs
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
+ -
Strong recommendation more likely as the difference between desirable and undesirable consequences becomes larger.
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Strong recommendation more likely with higher quality evidence
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Strong recommendation is more likely as the variability (or uncertainty) about patient values and preferences decreases
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Theory
Values and preferences: SRs?Include patient representatives at the guideline panel or as reviewersPatient surveys
If none of the above, describe the values and preferences that the panel placed on each outcome“This recommendation places a relatively high value on the reduction of mortality and a relatively low value on quality of life”
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
A weak recommendation is more likely as the incremental costs of an intervention (more resources consumed) increase
• Explicit and transparent
• Distinguish between quality of the evidence and grade of recommendations
• Patient important outcomes
• Benefits vs harms
• Values and preferences of patients
•Time and resource demanding
•Does not guarantee consistency across graders and does not eliminate the need for judgement
•Has been developed mainly for intervention Qs and not for diagnosis
Adapting an exıstıng Guideline
Process
The process provides a systematic approach to adapting guidelines produced in one setting for use in a different cultural and organisational context.
The adapted guideline addresses specific health questions relevant to the context of use and is suited to the needs, priorities, legislation, policies and resources in the targeted setting.
Flexible, transparent and explicit
Definition
Set-up
Adaptation
Finalisation
Hints
•Tasks to be completed before starting (i.e. identifying necessary skills and resources)
Set-up
Adaptation
Finalisation
Hints
1. Check whether adaptation is feasible
2. Establish an organizing committee
3. Select a guideline topic (possible criteria: prevalence of condition, costs, practice variations)
Set-up
Adaptation
Finalisation
Hints
4. Identify necessary resources and skills (clinical knowledge, administrative expertise, methodological expertise, managerial skills)
5. Complete tasks for the set-up phase: terms of reference, COI, consensus process, potential endorsement bodies, authorship, dissemination and implementations strategies)
6. Write adaptation plan
Set-up
Adaptation
Finalisation
ExamplesSteps
Set-up
Adaptation
Finalisation
Hints
Selecting topic, identifying questions, searching for guidelines, assess guidelines, prepare draft)
Set-up
Adaptation
Finalisation
Hints
7. Determine the health questions (PIPOH)
Set-up
Adaptation
Finalisation
Examples
Population concerned and characteristics of disease
Intervention(s) of interest
Professionals to whom the guideline will be targeted
Outcomes, including patient outcomes, systems outcomes and/or public health outcomes
Health care setting and context in which the guideline will implements
Steps
Set-up
Adaptation
Finalisation
Hints
7. Determine the health questions (PIPOH)
8. Search for guidelines and other relevant documents (search strategy, inclusion+exclusion criteria)
9. Screened retrieved guidelines10.Reduce number of retrieved
guidelines, if large
Set-up
Adaptation
Finalisation
Examples
Use rigour scores of the AGREE instrument!
Steps
Set-up
Adaptation
Finalisation
Hints
11. Assess guideline quality
Set-up
Adaptation
Finalisation
Examples
AGREE II instrument framework for assessing the quality of clinical practice guidelines23 itemsDomains: Scope and purpose
Stakeholder involvementRigour of developmentClarity of presentationApplicabilityEditorial independenceOverall assessment
Steps
Set-up
Adaptation
Finalisation
Hints
11. Assess guideline quality12. Assess guideline currency13. Assess guideline content
Set-up
Adaptation
Finalisation
ExamplesSteps
Set-up
Adaptation
Finalisation
Hints
11. Assess guideline quality12. Assess guideline currency13. Assess guideline content14.Assess guideline consistency
(search strategy and selection of evidence; consistency between evidence and summaries/ interpretation of evidence; consistency between interpretation and recommendations)
Set-up
Adaptation
Finalisation
Hints
15. Assess acceptability and applicability of the recommendations
Set-up
Adaptation
Finalisation
Examples
- Does the intervention meet patients views and preferences in the context of use?
- Are the intervention/equipment available in the context of use?
- Is the necessary expertise available? - Are there any constraints (e.g. legal,
resources) that would impede the implementation?
- Is the recommendation compatible with the culture and values in the setting where it is to be used?
Steps
Set-up
Adaptation
Finalisation
Hints
15. Assess acceptability and applicability of the recommendations
16.Review assessments17. Select between guidelines and
recommendations to create an adapted guideline
Set-up
Adaptation
Finalisation
Examples
a) Reject the whole guidelineb) Accept a whole guideline and all of its
recommendationsc) Accept the evidence summary of the
guidelined) Accept specific recommendationse) Modify specific recommendations
Steps
Set-up
Adaptation
Finalisation
Examples
Care must be taken when modifying existing guidelines and/or recommendations not to change the recommendations to such an extent that they are no longer in keeping with the evidence upon which they should be based.
Steps
Set-up
Adaptation
Finalisation
Hints
15. Assess acceptability and applicability of the recommendations
16.Review assessments17. Select between guidelines and
recommendations to create an adapted guideline
18. Prepare draft adapted guideline
Set-up
Adaptation
Finalisation
Hints
Feedback from stakeholders, establish process for review and update, create final document
Set-up
Adaptation
Finalisation
Hints
19. External review- target audience of the guideline
Set-up
Adaptation
Finalisation
Examples
- Foster ownership and commitment of intended users towards the guideline
- Ensure those likely to use the guideline review it and provide feedback (acceptance, identification of areas not covered, ensure recommendations are clear)
- Allow managers and policy makers to consider resources and prepare
- Act as first dissemination step
Steps
Set-up
Adaptation
Finalisation
Hints
19. External review- target audience of the guideline
20. Consult with endorsement bodies
21. Consult with source guideline developers
22. Acknowledge source documents23. Plan for aftercare of the adapted
guideline24. Produce final document
Useful resources- Guideline production/ GRADE
• GRADE Working Group: http://www.gradeworkinggroup.org/ •JCE series on GRADE: http://www.gradeworkinggroup.org/publications/JCE_series.htm •Cochrane handbook: http://handbook.cochrane.org/ • Review Manager: http://ims.cochrane.org/revman • GRADEpro: http://ims.cochrane.org/gradepro • Tutorials, videos etc: http://cebgrade.mcmaster.ca/
Useful resources- Guideline adaptation
• Guidelines International network (GIN): http://www.g-i-n.net/
• GIN Adaptation working group: http://www.g-i-n.net/working-groups/adaptation
• AGREE II: http://www.agreetrust.org/ • ADAPTE manual and toolkit:
http://www.g-i-n.net/document-store/working-groups-documents/adaptation/adapte-resource-toolkit-guideline-adaptation-2-0.pdf
Thank you for your attention!
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Theory
RR: 0.85 (0.75-1.02)
Mortality
Drug A Usual Care
Threshold for clinical decision making
Questions
Outcomes
Literature Search
Evidence
Grading
Recommendations
Strong recommendation with lower quality evidence
Antipyretics for children with chicken poxa.Aspirin b. ParacetamolHigh quality evidence for equivalenceLow quality evidence for increased side effects with aspirin (Reye Syndrome) Strong recommendation for paracetamol
