View
1
Download
0
Category
Preview:
Citation preview
ACCEPTED
Gemma Mansell, MSc, Arthritis Research UK Primary Care Centre, Primary Care Sciences,
Keele University, Keele, Staffordshire, UK
Jonathan C Hill, PhD, Arthritis Research UK Primary Care Centre, Primary Care Sciences,
Keele University, Keele, Staffordshire, UK
Steven J Kamper, PhD, Musculoskeletal Division, The George Institute for Global Health,
University of Sydney, Australia. EMGO+ Institute, VU University Medical Centre,
Amsterdam, The Netherlands
Peter Kent, PhD, Spine Centre of Southern Denmark, Hospital Lillibælt, Institute of Regional
Health Services Research, University of Southern Denmark, Denmark
Chris Main, PhD, Professor of Clinical Psychology, Arthritis Research UK Primary Care
Centre, Primary Care Sciences, Keele University, Keele, Staffordshire, UK
Danielle A van der Windt, PhD, Professor of Primary Care Epidemiology, Arthritis Research
UK Primary Care Centre, Primary Care Sciences, Keele University, Keele, Staffordshire, UK
Name and Address for Correspondence: Gemma Mansell, Arthritis Research UK Primary
Care Centre, Primary Care Sciences, Keele University, Keele, Staffordshire, UK, ST5 5BG,
Tel: 01782 734877, Fax: 01782 733911, Email: g.mansell@keele.ac.uk
ACCEPTED
The manuscript submitted does not contain information about medical device(s)/drug(s). The
National Institute for Health Research (NIHR) under its Programme Grants for Applied
Research Programme (Grant Reference Number RP-PG-0707-10131) funds were received in
support of this work. Relevant financial activities outside the submitted work: employment,
grants, royalties.
1. Introduction
In this article we consider whether adjustments to study designs, particularly of clinical trials,
might provide important information to help explain how interventions for patients with low
back pain work. Back pain clinical trials have typically shown small effects associated with
various treatments as compared to no treatment or current usual care, which could mean that
these treatments are not very effective, (1; 2)
or that the treatments are only effective for
specific clinical subgroups and interventions therefore need better targeting. (3)
However,
researchers also need to consider new methods to optimise treatment outcomes by
investigating the processes which explain how or why specific treatments work. (4)
Such
studies have been referred to as mediation analyses, (5)
and this paper seeks to highlight why
and how future studies should be designed to incorporate mediation analyses.
The primary importance of mediation analysis is that it can help us understand how to
improve treatments for patients with back pain. For example, psychological factors such as
fear-avoidance and self-efficacy are known to be predictive of patient outcomes such as
disability and return to work, but it is less clear whether targeting an intervention specifically
at reducing fear-avoidance or increasing self-efficacy is able to further improve such
outcomes. By analysing the causal pathway between treatment, potential mediators and
outcomes, we can identify the key factors to focus on during treatment and also identify
ACCEPTED
ACCEPTED
which factors do not mediate outcome, in order to further streamline and improve
intervention efficacy and efficiency.
At the XII International Low Back Pain Primary Care Research Forum in Odense, Denmark
2012, a dedicated workshop was held in which various aspects of mediation study design
were explored and methodological considerations discussed. The workshop brought together
experienced and internationally recognised low back pain researchers and clinicians to
provide an overview of the importance of mediation analyses and explore its usefulness in
testing our assumptions about how our interventions may be influencing outcomes. We
provide a summary of the workshop findings and discuss the implications for embedding
mediation analysis methodology within future interventional research. We also offer
definitions for some commonly used and misused terms, provide examples of mediation
research using different study designs and incorporate information on the methodology of
mediation analysis from the wider literature, in order to formulate recommendations for
future mediation research in the field of low back pain.
It is important to acknowledge that while the focus of this article is on mediators of specific
treatment effects, the identification of mediator variables often comes from a programme of
work which may involve different study designs. This article therefore includes a discussion
of research using observational designs to explore mediating processes, although the
limitations of evidence from such studies is noted and ultimately randomised controlled trials
(RCTs) are required to confirm which factors mediate specific treatment effects.
2. Concept Definitions
When longitudinal data from an observational study (single groups of individuals observed
over time) or an intervention study (two or more groups of individuals, exposed to different
treatment conditions) are analysed, some factors may be consistently predictive of outcome.
ACCEPTED
ACCEPTED
However, although such factors may show a statistical association with outcome following
treatment, the relationship can take a number of different forms: as a prognostic factor, a
treatment effect moderator or a treatment effect mediator.
2.1. Prognostic Factors
Prognostic factors are baseline characteristics that are associated with outcome regardless of
treatment, such as high baseline pain and high baseline activity limitation, that are generally
known to be associated with poor outcome in spinal pain patients across a variety of different
treatment interventions. So in an RCT, a prognostic factor might predict outcome for patients
in both the intervention and control groups. In intervention studies, prognostic factors have
also been called nonspecific predictors of treatment outcome. (5)
In statistical terms, they are
baseline variables that are associated with outcome (they have a significant main effect only)
but do not correlate or interact with treatment allocation. (5)
2.2. Treatment Effect Moderators
A treatment effect moderator is a baseline variable that identifies people who do better or
worse with a given treatment. These factors specify for whom and under what circumstances
a treatment is most effective and are therefore central to stratified care that targets specific
subgroups of people. For example, treatment A may work better for people with high fear of
movement (who) or in people seeking primary care rather than secondary care (what
circumstances) compared to treatment B. In statistical terms, treatment effect moderators
stratify or subgroup the effect of a particular treatment and are indicated by a statistical
interaction between treatment allocation (treatment A versus B), time and subgroup (e.g. high
versus low fear). (5)
In some fields of research, treatment effect moderators are also referred
ACCEPTED
ACCEPTED
to as effect modifiers or predictors of differential treatment effect, but these terms refer to the
same thing. (6)
2.3. Treatment Effect Mediators
In contrast to prognostic factors and treatment effect moderators, which are both measured
only at baseline, treatment effect mediators are factors that change in response to a specific
treatment and should therefore not only be measured at baseline but also during and after
treatment. These factors help explain the relationship between treatment and outcome, and
may therefore help identify the causal mechanisms by which treatments work. The
identification of mediators can potentially improve treatment effects by strengthening the
influence of the mediating factors involved in producing the treatment effect. Hypothetically
it might be that in order for cognitive behavioural therapy (CBT) (the treatment) to reduce
activity limitation (the outcome), a patient’s self-efficacy (the mediator) needs to be
strengthened, and that in the absence of self-efficacy improving, CBT will not be as effective
as it otherwise might be. In statistical terms, changes in mediator scores are associated with
specific treatment allocation (i.e. differ between intervention groups), and change in the
mediator has a direct association with change in treatment outcome (5)
(see Figure 1).
3. Study design in Mediation Analysis
An attractive aspect of mediation analysis is its applicability to different study designs. The
analytical process can be applied to data from cross-sectional studies, longitudinal
observational cohorts and RCTs. However, different study designs do impose different
limitations on the nature of the research question that can be answered and also the certainty
with which causal inferences can be made.
3.1. Cross-sectional designs
ACCEPTED
ACCEPTED
Mediation analysis has often been used with data from cross-sectional studies. With a
convincing biological rationale and a priori defined hypotheses, these data can provide some
evidence in support of pre-existing theory or be used to identify promising potential pathways
that can then be tested and confirmed via more robust designs such as RCTs. However,
causal inference requires a specified temporal (time-based) ordering of the investigated
variables, and in cross-sectional studies where participants are only assessed at one time
point, no temporal link can be made. Many publications that report the use of this design do
acknowledge that the use of cross-sectional data precludes the inference of causation. (7); (8); (9)
3.2. Longitudinal designs
A number of authors have stipulated that longitudinal designs are required for the assessment
of change over time. ((10); (11))
Longitudinal data enable the researcher to control for previous
levels of the mediator and outcome variables which can strengthen the claim of a potential
causal association, but do not account for the potential influence of confounding variables
that are not measured or included in the mediation analysis. Confounding is a key issue in
mediation analysis using longitudinal and cross-sectional observational designs. Confounders
are variables that are correlated with treatment exposure (i.e. show different values between
treatment groups) or the independent variable and are associated with outcome, but unlike
mediating variables are not on a causal pathway between treatment and outcome.
A series of studies on people with spinal pain in Australia used longitudinal data from
observational cohorts ((12); (13))
or pooled data from an RCT (14)
to investigate the mediating
influence of psychological factors in the relationship between pain and disability. Three or
more waves of data may be useful for assessing temporality as this provides stronger
evidence in support of a causal pathway linking the variables, ((15); (16); (17))
although it is
ACCEPTED
ACCEPTED
acknowledged that this does increase the complexity of the analysis and requires more
complex modelling procedures. (17)
3.3. Experimental designs
Mediation analysis aims to identify causal associations, and experimental designs where
patients are randomly allocated to a certain intervention at a point in time provides perhaps
the strongest support for causal inference. It should be noted however that RCTs and
observational studies seek to address different types of questions in mediation research.
Studies using observational data investigate the relationships between variables and outcomes
relevant to people with a particular condition, and may support the generation of hypotheses
regarding potential mediators. Mediation studies that include allocation to one treatment or
another as the independent variable investigate the mechanism of action of a particular
treatment on outcome. It is this aspect of experimental designs which is of particular benefit
in mediation analyses as it allows the investigation of the key variables (mediators) to target
in future intervention studies. For example, studies of mediation analysis in RCTs have found
that graded exposure treatment exerted part of its treatment effect in people with chronic LBP
via reduction of catastrophising, (18)
and showed that improvement in back pain was due to an
increase in lumbar muscle endurance following an exercise program. (19)
Another advantage of experimental designs is their ability to reduce the effect of
confounding. When an RCT design is used, randomisation helps ensure that intervention
groups have similar characteristics at baseline and so any differences in outcome are less
likely to be as a result of one group differing substantially from the other to start with. ((16);
(20); (21)) This can help researchers to see whether any mediating effects would have occurred
in spite of treatment (e.g. natural resolution) or whether they are due to the treatment being
given and are therefore on a causal pathway. (21)
However, it has been recognised that the
ACCEPTED
ACCEPTED
RCT design does not necessarily account for all unmeasured variables that could impact on
outcome. (22)
This is because although participants may have been randomised to the
intervention itself, change in the mediator(s) and outcome(s) occurs after randomisation has
taken place (if the intervention is effective) (15)
meaning that there may still be factors other
than treatment that affect the relationship between the mediator and outcome variables. (23)
For example, although patients may have similar levels of fear-avoidance at baseline due to
the randomisation process, during treatment some patients may develop better therapeutic
relationships with their therapist than others which may lead to a greater change in fear-
avoidance, and therefore a greater change in outcome for these particular patients. In this case
the therapeutic relationship is a potential confounder because it may have an impact on fear-
avoidance in addition to the impact of treatment.
4. Issues around heterogeneity in primary care populations
Studies carried out in secondary or tertiary care populations, in which the condition is usually
clearly established, unambiguously identified, and of sufficient severity or impact to enable
the identification of therapeutic change, may report larger and more consistent effect sizes
than those conducted with primary care populations. (24)
A problem when investigating
primary care populations may be the greater clinical variability in symptoms and their impact
on the population under study. This could also influence mediation analyses as greater
heterogeneity in potential mediators and outcomes results in larger standard deviations in the
scores in the study population, making it harder to demonstrate a difference between
treatments. Perhaps more importantly, greater clinical variability may also imply that the
mechanism explaining treatment effect varies across patients, which can translate into
difficulty in identifying the role of mediators in explaining treatment effects. In order to
overcome this, larger sample sizes might be required so that the assessment of the effect of
ACCEPTED
ACCEPTED
mediators in different subgroups can be undertaken. A specific advantage of primary care
populations is that, from a public health perspective, the results may be more widely
generalizable.
5. Areas for Improvement to the Design of Future Mediation Studies
A number of ways to incorporate mediation analysis into future RCTs by making small
changes to their current design have been suggested. The list below outlines the main
suggestions but is by no means exhaustive; more detailed reviews of the mediation literature,
relating to both design and analysis issues, can be found elsewhere. ((11); (23); (25); (26); (27))
5.1. The importance of a theoretical foundation
Selecting mediators based on theory can help researchers choose the key factors to focus on,
(28) and will help formulate clear hypotheses regarding how a treatment might work before the
study begins, which may subsequently be supported or refuted by the results of the mediation
analysis. (29)
A theoretical basis will also help with the issue of temporality described above,
as it provides a framework for showing how variables might relate to each other in a
particular order. (30)
5.2. The Design of Studies to Test Specifically for Mediation
It is suggested that where an intervention seeks to change a particular mediator or set of
mediators, measures of the proposed mediators should be included in the study. (11)
However,
investigators should limit data collection to only include mediators that can potentially be
modified by treatment and for which there is some evidence for a mediating relationship with
the outcome measure. (28)
For example, researchers might hypothesise that an exercise
ACCEPTED
ACCEPTED
programme is effective due to its action of improving aerobic endurance in patients, or
alternatively via improvements in self-efficacy. In these cases, measures of aerobic capacity
and/or self-efficacy should be incorporated into the study design to specifically test these
hypotheses.
Sample size obviously has an effect on the power of a study to detect mediating effects.
While the required sample size is dependent on the choice of statistical technique, variability
of the measures and the expected strength of the associations between the variables, a
substantial sample may be necessary.. It is recommended that sophisticated mediation
analysis methods require at least 150 (31)
to 200 (32)
participants to provide reliable results.
One review (33)
found that out of 166 mediation analysis studies published in two key journals
in their field, 113 studies (68%) included sample sizes of above 150.
5.3. Timing of Assessments
In order to assess temporality, it has been suggested that both the mediator and outcome
should be assessed frequently over the period of time that change is expected to occur. ((16);
(24)) This again shows the importance of considering treatment mediators a priori, as a clear
hypothesis regarding when a mediator is likely to change is important in planning the
optimal timing for study measurement time-points. While studies that include assessments
pre- and post-intervention may provide some useful information on what mediating factors
are related to outcome, (16)
they are not able to determine which variable (mediator or
outcome) changed first. (11)
Following the example of the exercise intervention proposed in
Section 5.2 above, while changes in aerobic capacity are likely to take several weeks to
occur, self-efficacy gains may be realised in a shorter time-frame. These considerations will
need careful thought in order to plan appropriate timing of study measurements. Also, an
intervention may work differently for different people and the effects could occur at different
ACCEPTED
ACCEPTED
times. (11)
Although attempting to untangle this issue adds considerable complexity, multiple
appropriately timed measurements are important for exploring mediators of treatment.
5.4. Inclusion of Study Measures with Appropriate Measurement Properties
Measurement error is a particularly important problem when trying to assess whether change
in a particular factor is responsible for the subsequent change in outcome, as in mediation
analysis. Measurements need to be highly reliable when measuring change (34)
to ensure
accurate estimates. Measurement error can become magnified when a raw change score
(difference between pre- and post-test) is calculated as there is likely to be error associated
with both measurements. (34)
A measure’s sensitivity to change is also important, as this
relates to whether or not a measure can detect change over time and separate this from
measurement error. (35)
The use of measures with poor reliability or responsiveness in
mediation studies may affect the user’s ability to identify effects of mediating variables.
5.5. Incorporation of Exploratory Analysis
In the workshop on mediation analysis held during the XII Forum in Odense 2012 there was
general consensus that, aside from the points mentioned above, there is a need to return to
exploratory analysis (for example mediation analyses in uncontrolled studies) and qualitative
analysis to pinpoint the concepts that are most important in investigating treatment
mechanisms. The importance of qualitative research in helping identify key factors to test as
mediators has been highlighted previously, (11)
suggesting that this can be especially helpful
in areas where no theories are available and can be used initially to explore therapeutic
processes and their impact on therapist and patient. This implies that a programme of
research will often be needed to identify key mediators, with replication of findings to
confirm the importance of those mediators being an essential part of that process. ((19); (28))
ACCEPTED
ACCEPTED
6. Summary and Conclusions
The majority of trials investigating the effectiveness of primary care interventions for back
pain have shown small or at best, moderate effects of treatment ((36); (37))
and the field is
looking for better ways to improve outcomes for patients with back pain. Mediation analysis
aims to provide better insight into the causal pathways underlying treatment effects,
explaining why treatments work or don’t work, and potentially offering new opportunities to
improve patient outcomes by optimising the content or delivery of treatment. Until recently,
mediation analysis in clinical research was often limited to a descriptive evaluation of the
processes potentially underlying the effects of treatment in trials. ((38); (39))
Over the past few
years, interest in the use of more sophisticated approaches to mediation analysis has
increased, often guided by methods described in the psychological literature. In this article,
we have summarized the concepts and different designs used in mediation analysis and
explained the importance of experimental designs when investigating mediators of treatment
effect. We have also emphasized the importance of other considerations such as defining and
understanding constructs, selecting study measures with appropriate measurement properties,
and ensuring study measurement time-points are appropriately selected, in order to
investigate the longitudinal associations between mediating and outcome variables. We have
also outlined the relevance of observational and qualitative research in identifying potential
mediating factors. Based on the discussions during the 2012 workshop and supported by the
literature we have proposed a set of recommendations to support and improve the design of
mediation analysis in back pain research (Box 1), with the ultimate aim to improve the design
and delivery of intervention studies and optimize outcomes for patients with back pain.
ACCEPTED
ACCEPTED
Acknowledgements: Chris Main would like to thank Dr Kelvin Jordan for his statistical
advice.
References
(1) Van der Windt D, Hay E, Jellema P, et al. Psychosocial interventions for low back pain in
primary care: Lessons learnt from recent trials. Spine 2008;33: 81-9.
(2) Guzman J, Esmail R, Karjalainen K, et al. Multidisciplinary rehabilitation for chronic
low back pain: Systematic review. BMJ 2001;322: 1511-6.
(3) Haldorsen EMH, Grasdal AL, Skouen JS, et al. Is there a right treatment for a particular
patient group? Comparison of ordinary treatment, light multidisciplinary treatment, and
extensive multidisciplinary treatment for long-term sick-listed employees with
musculoskeletal pain. Pain 2002;95: 49-63.
(4) Shaw WS, Linton SJ, Pransky G. Reducing sickness absence from work due to low back
pain: How well do intervention strategies match modifiable risk factors? J Occup Rehabil
2006;16: 591-605.
(5) Kraemer HC, Wilson GT, Fairburn CG, et al. Mediators and moderators of treatment
effects in randomized clinical trials. Arch Gen Psychiatry 2002;59: 877-83.
(6) Kraemer HC, Stice E, Kazdin A, et al. How do risk factors work together? Mediators,
moderators, and independent, overlapping, and proxy risk factors. Am J Psychiatry 2001;158:
848-56.
(7) Woby SR, Roach NK, Urmston M, et al. The relation between cognitive factors and
levels of pain and disability in chronic low back pain patients presenting for physiotherapy.
Eur J Pain 2007;11: 869-877.
(8) Arnstein P, Caudill M, Mandle CL, et al. Self efficacy as a mediator of the relationship
between pain intensity, disability and depression in chronic pain patients. Pain 1999;80: 483–
91.
ACCEPTED
ACCEPTED
(9) Gheldof ELM, Vinck J, Van den Bussche E, et al. Pain and pain-related fear are
associated with functional and social disability in an occupational setting: Evidence of
mediation by pain-related fear. Eur J Pain 2006;10: 513–25.
(10) Liu LC, Flay BR, Aban Aya Investigators. Evaluating mediation in longitudinal
multivariate data: Mediation effects for the Aban Aya Youth Project drug prevention
program. Prev Sci 2009;10: 197-207.
(11) Kazdin AE. Mediators and mechanisms of change in psychotherapy research. Annu Rev
Clin Psychol 2007;3: 1-27.
(12) Kamper SJ, Maher CG, Costa L da CM, et al. Does fear of movement mediate the
relationship between pain intensity and disability in patients following whiplash injury? A
prospective longitudinal study. Pain 2012;153: 113-9.
(13) Costa L da CM, Maher CG, McAuley JH, et al. Self-efficacy is more important than
fear of movement in mediating the relationship between pain and disability in chronic low
back pain. Eur J Pain 2011;15: 213–9.
(14) Hall AM, Kamper SJ, Maher CG, et al. Symptoms of depression and stress mediate the
effect of pain on disability. Pain 2011;152: 1044–51.
(15) Collins LM, Graham JW, Flaherty BP. An alternative framework for defining
mediation. Multivar Behav Res 1998;33: 295-312.
(16) Lockhart G, MacKinnon DP, Ohlrich V. Mediation analysis in psychosomatic medicine
research. Psychosom Med 2011;73: 29-43.
(17) Maric M, Wiers RW, Prins PJM. Ten ways to improve the use of statistical mediation
analysis in the practice of child and adolescent treatment research. Clin Child Fam Psychol
Rev 2012;15: 177-91.
ACCEPTED
ACCEPTED
(18) Leeuw M, Goossens MEJB, van Breukelen GJP, et al. Exposure in vivo versus operant
graded activity in chronic low back pain patients: Results of a randomized controlled trial.
Pain 2008;138: 192–207.
(19) Mulroy SJ, Winstein CJ, Kulig K, et al. Secondary mediation and regression analyses of
the PTClinResNet database: Determining causal relationships among the international
classification of functioning, disability and health levels for four physical therapy
intervention trials. Phys Ther 2011;91: 1766-79.
(20) Bullock JG, Green DP, Ha SE. Yes, but what’s the mechanism? (Don’t expect an easy
answer). J Pers Soc Psychol 2010;98: 550-58.
(21) Murphy R, Cooper Z, Hollon SD, et al. How do psychological treatments work?
Investigating mediators of change. Behav Res Ther 2009;47: 1-5.
(22) Judd CM, Kenny DA. Process analysis: Estimating mediation in treatment evaluations.
Evaluation Rev 1981;5: 602-19.
(23) Emsley R, Dunn G, White IR. Mediation and moderation of treatment effects in
randomised controlled trials of complex interventions. Stat Methods Med Res 2010;19: 237-
70.
(24) Grissom RJ, Kim JJ. Effect sizes for research: Univariate and multivariate applications.
(2nd
Ed.) New York: Taylor & Francis (Routledge); 2012.
(25) Hayes AF. Beyond Baron and Kenny: Statistical mediation analysis in the new
millennium. Commun Monogr 2009;76: 408-20.
(26) MacKinnon DP, Fairchild AJ, Fritz MS. Mediation analysis. Ann Rev Psychol 2007;58:
593-614.
(27) MacKinnon DP, Fairchild AJ. Current directions in mediation analysis. Curr Dir
Psychol Sci 2009;18: 16-20.
ACCEPTED
ACCEPTED
(28) MacKinnon DP. Analysis of mediating variables in prevention and intervention
research. National Institute on Drug Abuse Monograph 1994;139: 127-53.
(29) MacKinnon DP. Integrating mediators and moderators in research design. Res Social
Work Pract 2011;21: 675-81.
(30) Mathieu JE, Taylor SR. Clarifying conditions and decision points for mediational type
inferences in organizational behaviour. J Organiz Behav 2006;27: 1031-56.
(31) Holbert RL, Stephenson MT. Structural equation modelling in the communication
sciences, 1995-2000. J Hum Commun Res 2008;28: 531-51.
(32) Tomarken AJ, Waller NG. Potential problems with “well fitting” models. J Abnormal
Psychol 2003;112: 578-98.
(33) Fritz MS, MacKinnon DP. Required sample size to detect the mediated effect. Psychol
Sci 2007;18: 233-9.
(34) Streiner DL, Norman GR. Health measurement scales: A practical guide to their
development and use.( 4th
Ed). Oxford: Oxford University Press; 2008.
(35) Terwee CB, Bot SDM, de Boer MR, et al. Quality criteria were proposed for
measurement properties of health status questionnaires. J Clin Epidemiol 2007;60: 34-42.
(36) Keller A, Hayden J, Bombardier C, et al. Effect sizes of non-surgical treatments of non-
specific low-back pain. Eur Spine J 2007;16: 1776-88.
(37) Machedo LA, Kamper SJ, Herbert RD, et al. Analgesic effects of treatments for non-
specific low back pain: A meta-analysis of placebo-controlled randomized trials.
Rheumatology (Oxford) 2009;48: 520-7.
(38) Jellema P, van der Windt DAWM, van der Horst HE, et al. Why is a treatment aimed at
psychosocial factors not effective in patients with (sub)acute low back pain? Pain 2005;118:
350-9.
ACCEPTED
Recommended