View
27
Download
0
Category
Tags:
Preview:
DESCRIPTION
Issues and challenges to scoping and focusing the question. ESQUIRE Qualitative Systematic Review Workshop University of Sheffield 8 September 2011. Janet Harris Cochrane Qualitative Research Methods Group. Session aims. - PowerPoint PPT Presentation
Citation preview
Issues and challenges to scoping and focusing the question
ESQUIRE Qualitative Systematic Review Workshop University of Sheffield8 September 2011
Janet HarrisCochrane Qualitative Research Methods Group
Session aims
• To identify the issues and challenges in formulating different types of qualitative questions for qualitative evidence synthesis
The issue: What question?• The challenges
•Utility•Relevance•Timeliness•Explanatory value
How effectiveness questions are developed
Refine definition of
problem
Identify possible
causes and effects
Identify problem
Test solution
s
Margerunm-Leys http://www-personal.umich.edu/~jmargeru/conceptmap/types.htm
Clinical expertise + research evidence are used to:
Effectiveness question: In adults with Low Back Pain does provision of individual education reduce pain, improve functioning and promote return to work?
P: Adults with back pain (BP)
O: Reduce pain, improve
functioning, return to work
I: Information on how to self-manage BP
Assumed
solution:Patients
need written and/or verbal
education delivered
by a health professiona
l
Problem and
solution (as defined
by the researcher
s)
Possible results from a search using an informing question
Problem Definition(s)
Effects
CausesInforming question
(Background question)
Single studies / evidence syntheses identifying factors that might make us rethink assumptions
Explore assumptions with informing questions
P: Important characteristics
relating to I and O
O: Factors that may dilute
potential effectiveness
I: Factors that affect
implementation
Informing question
(Background question)
Refine PConsider subgroup analysis
Consistency of
implementation
Such as participants’
views
Back pain patients’ experiences of receiving education for self management
Informing questions may identify theories• Theories are explanations for a phenomenon or
an event e.g. Why something happens in relation to another event
• They can be grand theories, they can be middle range theories, or they can be single-case theories
• Middle range theory goes beyond individual studies, providing a cross cutting explanation for patterns that are seen across settings and different types of people
• Whenever possible, interventions testing effectiveness should be based on theory
Issues raised from informing questions
• There may be ‘essential ingredients’ that need to be included in interventions• For example, the search on back pain education
identified an issue with appropriateness and relevance of information
• Informing questions need to produce convincing evidence about these ingredients to justify changing an effectiveness question.
• How much evidence is needed to change the original effectiveness question?
Revising the effectiveness question: An example
• In adults with Back Pain does provision of individual education by a health professional reduce pain, improve functioning and promote return to work?
• Revised effectiveness question:• P = adults with Low Back Pain • I = information explained by health professional• C= information explained by a health
professional with additional peer support• O = reduce pain, improve functioning and
promote return to work
Additional ingredients that theoretically increase relevance
Issues with the refined question• Is the quality of reporting for effectiveness studies
adequate?• For example, do the interventions describe how
information is explained by the health professionals? • Is the amount of time for the explanation documented? • Does the comparison describe the type of peer support
offered?
• Qualitative evidence can indicate how a question should be refined, but if effectiveness studies do not report the relevant information then the end product could be an empty review
Enhancing questions to explain the results of effectiveness reviews
• Monitors pathway
from interventi
on to outcomes
• May highlight problems
with Quan
studies
Enhancing question• Within-study data
• Parallel-study data
• Similar-study data
Enhancing questions: Issues and challenges
• Adequate data within the study about PICO characteristics?
• Adequate data in embedded qualitative studies?• Adequate data in parallel sibling studies?• Can ‘similar’ qualitative studies be found?• What does ‘similar’ mean – across embedded, sibling,
and independently conducted studies? • Comparability of data - how to synthesize data across
studies?
Utility of findings: Used to critique
design and PICO in
included studies;
subgroup analysis of
theory-based interventions
..
Extending question: Can explore an issue related to population, intervention, comparison or outcome
Cause: Low awareness of educational
approaches for behaviour change
Extending question:How do people make sense of and use Back Pain information? What educational theories promote better use of BP information?
Problem noted: Lack of theory-based education on how to self-
manage BP
Negative effect: Increased
utilization of services and absenteeism
Extending questions: Issues and challenges• Extending questions require a separate qualitative
evidence synthesis, either conducted within or alongside an effectiveness review. Funding needs to be justified by considering:• Timeliness: Is the information perceived to be critical to
using the results of the effectiveness review? • Relevance: Is the issue important to funders, policymakers
and effectiveness reviewers ?• Utility: did the scoping process specifically note issues that
need to be informed by qualitative research?• Explanatory value: Will the qualitative synthesis contribute
to explaining the outcomes of effectiveness studies and/or the broader context that influences effectiveness?
Extending questions: Issues and challenges• Extending questions require coordination of a team
including qualitative evidence synthesis (QES) members working alongside effectiveness review (ER) members
• The parallel processes need to be considered because• Findings from the QES may identify quality issues in the
effectiveness studies. At what point are these quality issues brought into the effectiveness review – at abstract review? Development of inclusion criteria? Critical appraisal? Metaanalysis?
• Reporting QES process and findings: Approaches are still being developed. At the same time, the software for Cochrane ERs – RevMan – does not yet have the facility for including Extending reviews.
Effectiveness review
Supplementing questions
Supplementing review:
Conducted independently
Before or after an ER
On a wider issue than
effectiveness
Effectiveness review
Supplementing question issues• Supplementing questions: Used in stand alone reviews
to produce information on a wider issue • Does peer support help people with chronic conditions?• Does peer support promote health literacy (help people to
understand and use health information)• Issues
• Timeliness: Will the theory be used over time?• Relevance: Does the theory address issues noted by ER
authors and those trying to implement review results (External validity)
• Utility: will the theory enable development of better effectiveness studies?
• Explanatory value: Can the theory be used to explain some of the differences in effect within and across existing ERs? Can the theory be used to inform development of future ERs?
Summary: What guides qualitative question formulation• The need to ensure that effectiveness questions are
informed by existing theory on what works (Informing questions)
• The need to provide real-time explanations for effectiveness studies (Enhancing questions)
• The need to further explore issues identified in the review by reviewing a broader range of research on effectiveness (Extending question)
• The need to explore the context surrounding a condition or an intervention, including situations and experiences that may not be directly related to effectiveness (Supplementing question)
Is it worth it?• When scoping and focusing the question, we need to ask:
Does this question pass the TRUE test?• Timely: Will the question still be relevant when the
review is finally completed? • Relevant: How will this question contribute information
that is relevant to health services policymakers?• Useful: Which of (the many) findings from the qualitative
scoping will be useful in focusing the review and/or explaining the results?
• Explanatory value: Does the synthesis contribute to explaining the outcomes of effectiveness studies and/or the broader context that influences effectiveness?
Further challenges
• Team skills• Team coordination• Informed and transparent decisions
about when and how to useQES/ER data
Recommended