The use of randomized control trials in complementary therapies: exploring the issues

The use of randomized control trials incomplementary therapies: exploring theissues

Janet Richardson PhD BSc RN DipDN

Lecturer Practitioner in Research and Development,

The Florence Nightingale Division of Nursing and Midwifery,

King's College, James Clerk Maxwell Building,

57 Waterloo Road, London SE1 8WA, England

Accepted for publication 21 January 2000

RICHARDSONRICHARDSON J. (2000)J. (2000) Journal of Advanced Nursing 32(2), 398±406

The use of randomized control trials in complementary therapies: exploring

the issues

The current popularity of complementary therapies presents an interesting

challenge to nurses and midwives. If they are to deliver such therapies

themselves, or support patients in choosing appropriate therapies they will

need to consider the professional and legal issues, in particular those regarding

safety. Evidence for the effectiveness for complementary therapies is also a

requirement in order that their integration into nursing practice can be justi®ed.

Purchasers are currently hampered by the lack of credible evidence for

effectiveness and until that evidence is provided, access to such therapies

through the National Health Service (NHS) will remain limited. The form that

evidence should take has led to a lively debate about possible methodological

approaches. There appears to be a clash between the medical profession and

those working in the ®eld of complementary therapy research, with the medical

establishment advocating randomized control trials (RCTs). This contrasts with

the view held by some advocates of complementary therapies that the RCT

approach is reductionist and not applicable to such approaches. The pivot

of the debate on the methodological approaches for evaluating complementary

therapies is the contrast of two apparently different and diverse world-views,

and the assertion that methods developed in one world-view are not transferable

to the other. There is also some confusion within the ®eld of complementary

therapy over the applicability of RCTs to therapies such as acupuncture, and the

mistaken assumption that trials which include a control group, are also required

to be double-blind. This paper is based on the need for good quality evidence of

effectiveness in complementary therapy. It will set out the concerns associated

with the use of RCTs within complementary therapy, together with the bene®ts

and limitations of this approach. The paper will go on to review research

options and propose some suggestions for future methodological approaches.

Keywords: complementary therapies, randomized control trials, pragmatic,

explanatory, methodological, double-blind, health services research, nursing

Journal of Advanced Nursing, 2000, 32(2), 398±406 Integrative literature reviews and meta-analyses

398 Ó 2000 Blackwell Science Ltd

INTRODUCTION

There is little doubt that interest in complementary

therapies has increased in recent years, and that accom-

panying this increase is large-scale expenditure on

complementary medicines and practitioners. In the

United Kingdom (UK), estimates of the percentage of the

population that uses complementary therapies range from

14%±30% of the general population, with acupuncture,

osteopathy, chiropractic, herbal medicine and homeo-

pathy indicated as the most popular therapies (Sharma

1992). Consumer surveys indicate positive public atti-

tudes in other European countries with 20%±50% of the

European population consulting complementary practi-

tioners, and identifying acupuncture as one of the most

popular forms of complementary treatment (Fisher &

Ward 1994).

For nurses and midwives the increase in public interest

in complementary therapies is proving to be an interesting

challenge and practitioners are responding in a number of

different ways. Some advocate the use of therapies such as

massage and aromatherapy in hospital settings (Styles

1997). Stevensen (1997), for example, claims that comple-

mentary therapies can bene®t patients and suggests how

nurses can work with such therapies in the context of

United Kingdom Central Council for Nursing, Midwifery

and Health Visiting (UKCC) Professional Development

Categories. Whilst massage, re¯exology and aromatherapy

perhaps understandably are popular amongst nurses,

more invasive treatments such as acupuncture are also

receiving attention. In particular midwives are beginning

to offer acupuncture to pregnant and postnatal women

(West 1997), and nurses sometimes provide acupuncture

in chronic pain clinics.

For most nurses, however, rather than providing the

therapy, the primary issue will be related to giving advice

and support to their clients who are keen to integrate

complementary therapies into their healthcare. Dimond

(1995) suggests that midwives have a duty to ensure that

their clients are aware of the advantages and disadvan-

tages and the dangers of particular therapies. In doing this,

however, they will need to be aware of the legal and

ethical implications for their practice (Norton 1995,

Geddes & Henry 1997).

Arguably, if nurses are to advise or support their clients

in the choice to use a complementary therapy, they should

also be aware of the evidence for the effectiveness of the

treatment. The need for research evidence demonstrating

the effectiveness of complementary therapy, and the

methodological issues, have received some attention in

the nursing press (Ersser 1995, Botting 1997). Many

complementary therapies have not been adequately inves-

tigated (Hamilton & Bechtel 1996). So whilst nurses might

facilitate or incorporate such therapies in to their practice,

there is a corresponding need to develop a knowledge base

grounded in effectiveness and outcome measurement

(Norton 1995). Further, as long as purchasers of healthcare

are hampered by the lack of credible evidence for effect-

iveness, access to such therapies through the National

Health Service (NHS) will remain limited.

Five types of evidence have been posited ranging from

systematic reviews of well-designed randomized control

trials (RCTs) to opinions based on clinical evidence or

descriptive studies. The strongest types of evidence

ranked according to Bandolier (1995) are thought to be

provided by systematic reviews of well-designed RCTs

(type i), and strong evidence from at least one properly

designed RCT of appropriate size (type ii). Bandolier rates

the evidence from well-designed trials without random-

ization and time-series or matched case-control studies as

type iii. The evidence provided by well-designed non-

experimental studies or the opinions of respected author-

ities have been ranked as type iv and type v, respectively.

In complementary therapy the form the evidence should

take has led to a lively debate about possible methodolo-

gical approaches. The pivot of this debate is the proposed

contrast of two apparently different and diverse world-

views often referred to as paradigms (Patel 1987, Mercer

et al. 1995, Vickers 1996), and the assertion that methods

developed in one world-view (reductionist/biomedical)

are not transferable to the other (holistic). The arguments

put forward in this paper are based on the need for good

quality evidence of effectiveness in complementary ther-

apy if such therapies are to be integrated into nursing

practice. This paper does not attempt to present an

in-depth monologue of the arguments for and against

RCTs, and the author accepts that different views exist

regarding the use of RCTs within the nursing and medical

professions as well as in complementary therapy. It does,

however, set out the concerns associated with the use of

RCTs within complementary therapy, together with the

bene®ts of this approach. Ultimately, the position adopted

is one which advocates that the best `scienti®c' approach

to the inquiry is to adopt a methodology which is most

appropriate to the question and the context.

RANDOMIZED CONTROL TRIALS

The randomized control trial (RCT) is rooted in a `posit-

ivist' science. Positivism has its origins in the natural

sciences. It is concerned with events which can be

observed, the research requires a stable environment, is

`value free and objective', should be quanti®able, and aims

to establish causal relationships (Proctor 1998). The need

to eliminate error and sources of bias is paramount,

therefore tight control is required in order to provide

results which support or reject hypotheses and create

generalizable laws (Proctor 1998). The RCT has widely

Integrative literature reviews and meta-analyses Complementary therapies

Ó 2000 Blackwell Science Ltd, Journal of Advanced Nursing, 32(2), 398±406 399

become regarded as the principle method for obtaining a

reliable assessment of treatment effect and involves four

phases. The ®rst phase is primarily concerned with drug

safety, toxicity and side-effects. Phase 2 involves small-

scale investigations into effectiveness and safety. Phase 3

is a full-scale evaluation of treatment, comparing the new

drug or interaction with current standard (or placebo)

treatment. In this phase each patient is randomly assigned

to the treatment or placebo group. Phase 4 is the post-

marketing surveillance (Pocock 1983).

Before the Second World War there were no formal

requirements for clinical trials before a drug could be

freely marketed. The ®rst RCTs were performed in agri-

culture where various crops and fertilisers were assigned

in random arrangements. The ®rst properly randomized

control trial in medicine was conducted by the Medical

Research Council (MRC) in 1948. In this trial patients with

pulmonary tuberculosis were randomized to streptomycin

and bed rest or bed rest only; no `placebo' drug was used.

The thalidomide tragedy in the 1960s caused a tightening

of government regulations in the United States of America

(USA) and UK. Regulatory requirements for the USA Food

and Drug Administration require evidence from two

placebo controlled trials. This will generally provide good

evidence that a drug works under speci®c conditions, but

it is less clear how the drug will work under different

conditions and in patients who do not resemble those in

the trial. Other interventions such as surgical procedures

and diagnostic imaging, however, are often adopted on the

basis of little or uncertain evidence (Henry & Hill 1995).

Randomization and the use of a control group remain an

important component of trials attempting to assess the

effectiveness of an intervention. Uncontrolled trials have

the potential to provide a very distorted view of therapy. A

therapy may gain widespread acceptance on the basis of

uncontrolled trials prior to RCTs that may prove to be less

optimistic. Further, wide variations in response rates may

be due to patient selection and disease progression. In

nursing and complementary therapy the practitioner is an

integral part of the intervention. This is also true in much

orthodox medicine. For example, the skill of a surgeon is

likely to impact on the patient's prognosis, thus general-

izing results from highly skilled surgeons to routine

clinical practice may be problematic (Pocock 1983).

One of the major objections to the use of RCTs in the

evaluation of complementary therapies is the blinding

procedure (Anthony 1993). The purpose of blinding is to

exclude non-speci®c factors (placebo effects) which may

produce a desirable outcome, but which are not due

directly to the active intervention. Double-blind trials are

the `gold standard' of clinical trial research, yet double-

blind studies for interventions other than pharmaceutical

ones are very rare, and for some treatments it is totally

impossible to arrange a double-blind trial, for example in

cancer trials where complex drug dose calculations are

required (Pocock 1983). Anthony (1993) believes that

blind designs are impossible in complementary therapy as

the therapist is an integral part of the intervention.

There are, however, clear distinctions between blinding

and randomization, and good quality RCTs can be

conducted in the absence of a blinding procedure. Ulti-

mately the aim of randomization is to provide equivalent

groups for comparison. Where comparisons are made

between groups which are not equivalent, research can be

fundamentally ¯awed and lead to insupportable conclu-

sions (Bagenal et al. 1990, Hayes et al. 1990, James & Reed

1990, Monro & Payne 1990, Sheard 1990a, b). However

even careful randomization can lead to non-equivalent

group comparisons (Cassidy et al. 19901 ), which highlights

the importance of assessing group equivalence, not just at

entry to the study to establish an equivalent base-line, but

at each stage of the analysis.

There are a number of practical dif®culties in the

interpretation of clinical trials. For example, an overem-

phasis on signi®cance testing and inadequate sample sizes

increase the probability of a type 2 error. Repeated

signi®cance testing on accumulating data or the use of

multiple endpoints increases the probability of a signi®-

cant result due to chance. Further, care is required when

carrying out subgroup analysis, in particular the sample

size and power of the test should be suf®cient to detect a

signi®cant result. In general it is recommended that both

con®dence intervals (for the treatment difference) and

P-values are reported, as the use of con®dence intervals

re¯ects the true magnitude of the effect (Pocock 1983,

Pocock et al. 1987, Altman 1991). Problems may arise if

the practitioners participating are unrepresentative, if

patients or treatments are atypical, or if there is poor

recruitment. Furthermore it has been suggested that the

RCT will be out of context, and patient/practitioner beliefs

and wishes will be excluded (Black 1996). The importance

of asking patients' their views is now widely acknow-

ledged and such approaches can be incorporated into

research into complementary therapy (Kacperek 1997).

Jones (1995) suggests that the establishment of evi-

dence-based practice depends on research from both

qualitative and quantitative traditions. Qualitative

methods are well established in nursing (see for example

Morse 1989, Morse & Field 1996); case study approaches

have also been used to assess the effectiveness of comple-

mentary therapy (Ballard 1996, Brooker et al. 1997). It is

interesting to note that qualitative research methodologies

are becoming increasingly popular in orthodox medicine

(Fitzpatrick & Boulton 1994, Britten 1995, Pope & Mays

1995, Greenhalgh & Taylor 1997).

Sackett et al. (1991) emphasized the importance of

`clinical' as well as statistical signi®cance. This can mean

two things, the clinical signi®cance to the patient, or the

extent to which the results of a trial lead to changes in

clinical behaviour. Many RCTs compare treatments that

J. Richardson

400 Ó 2000 Blackwell Science Ltd, Journal of Advanced Nursing, 32(2), 398±406

will produce only moderate differences in outcome, but

these differences may be clinically important, hence the

need to combine trial data in systematic reviews and meta-

analysis (Sackett et al. 1991, Clarke & Stewart 1995).

Clinical research is ultimately about making a difference

to patient care, quality of life or health outcome.

In RCTs external validity is often poor, as the outcome is

in¯uenced by the characteristics of the provider, thus

Fitter & Thomas (1997) emphasize the importance of

evaluating the therapies under `normal' service condi-

tions, leaving practitioners free to give individualized

patient treatments. External validity requires attention

when conducting or evaluating clinical trials (Vickers

1995b). However, issues of external validity receive little

attention in the instruments developed to judge methodo-

logical quality of trials (Moher et al. 1995).

The fundamental underpinning of the RCT is to assess

and to isolate potential sources of bias. This includes

observation bias, sampling bias, non-speci®c factors

(placebo effects), natural disease progression (large

number of diseases are self-limiting), and cointerventions

(Vickers 1995c). A number of important methodological

innovations (such as quasi-experimental designs), encour-

age the application of controlled trials in clinical practice

for assessing the outcome of an intervention, by providing

a means by which potential biases can be identi®ed and

dealt with (see for example Cook & Campbell 1979).

EVALUATING COMPLEMENTARY THERAPIES:THE METHODOLOGICAL DEBATE

In spite of the current methodological debate in comple-

mentary therapy, RCTs have been conducted and a

number of systematic reviews exist (see for example

Richardson & Vincent 1986, ter Riet et al. 1990, 1991b,

Kleijnen et al. 1991a). However, Mercer et al. (1995)

suggest a number of potential barriers to the use of RCTs

in complementary therapy, which include con¯icting

concepts and theories of health and disease, lack of

agreement about diagnostic criteria, contrasting views

about the therapeutic process and different theories of

causation. Patel (1987) identi®es `paradigmatic' problems

and suggests that a difference of world-views presents

more of a problem for the integration of complementary

with orthodox medicine than lack of evidence. Vickers

(1996) explains that `paradigm' is a term borrowed from

the history and philosophy of science, and a shift from one

paradigm to another is a transition to different ways of

viewing the world.

It might be argued that paradigms in complementary

and orthodox medicine are widely different, therefore

research methods developed in one paradigm are not

transferable to the other and those working within a

paradigm are unable to look outside it. Black (1996)

suggests that the arti®ciality of the RCT may reduce the

placebo element of any intervention, failing to capitalize

on the non-speci®c treatment effects, therefore the trial

will inevitably re¯ect the minimum level of bene®t that

can be expected. However, it is possible to apply RCTs

without adopting a reductionist approach, for example a

number of RCTs have been applied to the evaluation of

aromatherapy and massage, psychotherapeutic tech-

niques, healing and prayer (Dunn et al. 1995, Vickers

1996).

In attempting to disentangle the issues relating to the

use of RCTs in the context of complementary (and to some

extent orthodox) therapies the research question becomes

paramount, as different questions require different

approaches (Canter & Nanke 1993, Vickers 1995a). A

report prepared by the National Institutes for Health (NIH)

concludes that the RCT is the gold standard and most

reliable method yet developed, and should always be used

where it is practical and ethical to do so. Complementary

therapies are often very individualistic in approach and

cannot always be standardized as a treatment for large

groups of individuals in the context of an RCT (Mercer

et al. 1995). Some interventions are impossible to blind,

and due to the nature of different diagnostic systems,

patient selection may be problematic. Further, one meth-

odological assumption of the RCT is that the majority of

patients have no preference for one treatment or another

(Fitter & Thomas 1997).

A number of ways of dealing with these problems have

been proposed. Wiegant et al. (1991) and Brewin &

Bradley (1989) set out procedures for patient selection.

Others have suggested alternative methodologies for evalu-

ating effectiveness such as single case designs, qualit-

ative approaches, outcome assessment and clinical audit,

and observational studies (Aldridge 1988, Reilly & Taylor

1993, St George 1994, Mercer et al. 1995, Black 1996,

Melchart et al. 1997). Heron & Reason (1984) have gone so

far as to categorize the RCT as a source of alienation Ð

because the individual is separated from what is going on

in their body and from decisions about treatment. As the

RCT remains the `gold standard' within conventional

medical research and has in some cases been applied to

the evaluation of complementary therapies, it is important

that this form of evidence is used, where possible, to

provide evidence of effectiveness.

The historical purpose of medical research has been to

understand the mechanisms of disease and produce new

treatments, not to question effectiveness or the implemen-

tation of those treatments in routine clinical practice

(Smith 1995). Many conventional treatments have not

been subjected to randomized control trials, and as a

result new interventions have been introduced on a wide-

scale on the basis of opinion rather than evidence from

RCTs (Smith & Rennie 1995). Treatments have been

introduced and used widely before being shown to be

harmful and of no bene®t, for example the use of insulin



coma therapy, leucotomy, and regressive electro-convul-

sive therapy (ECT) for the treatment of schizophrenia

(Andrews 1989).

Lack of evidence can prohibit the effective use of

resources. For example, the prevalence of pressure sores

in the UK is 5±9% of patients receiving hospital and

community care; treatment costs are estimated as high

as £755 million a year, but there is little evidence

regarding the effectiveness of available treatments

(Bandolier 1994).

In biomedicine the overall approach places an emphasis

on generating objective `hard data' to establish particular

cause±effect relationships for disease. The experimental

methodology ®ts closely with the theoretical assumptions

contained in the biomedical model, and has been adopted

as the `gold standard'. Strenuous efforts are made to

isolate independent and dependent variables within the

experimental research design and eliminate `non-speci®c'

variables. In assessing the impact of health care, however,

`cure' is not the only factor to be considered; increasingly

lay people place importance on how far treatment

enhances or sustains emotional well-being, a healthier

life-style and more satisfying relationships (Mercer et al.

1995). So arguably, clinical trials should include some

measure of quality of life and patients' perceptions

(Delamothe 1994, Sackett 1994).

CONTROLLING FOR PLACEBO EFFECTS

The terms `placebo' and `placebo effect' are often used

interchangeably; however, it is important to make a clear

distinction between the two terms. A placebo is an inert

substance used to persuade an individual that she/he has

received an active, usually therapeutic, intervention. In

contrast a `nocebo' is an inert substance which produces a

negative or harmful effect. The placebo-effect is an effect

rather than a substance. It is mediated by unde®ned

mechanisms which facilitate a therapeutic change, but

which are usually viewed as less valid than pharmaco-

logical or other àctive' interventions (Watkins 1994).

Lewith (1993) suggests that the placebo response is

consistent at 35á2% (� 2á2%), a ®gure which is generally

thought to come from Beecher (1955) who quotes the

average of his own 11 studies, all of which varied widely.

In fact the placebo response in double-blind studies is not

constant and has been shown to vary from 0% to 100%

(Ernst & Resch 1995, Wall 1996).

There is no evidence that the placebo-effect differenti-

ates between organic and mental illness, or that speci®c

groups of the population are `placebo responders' with a

particular type of mentality (Wall 1996). In studies of pain

(a multidimensional experience) all dimensions appear to

be equally involved in the placebo response, either

separately or together. The non-speci®c effects which

bring about a placebo response can be extremely powerful

and are complex in nature. Watkins (1994) suggests that

the power of the placebo-effect has been recognized and

used to enhance the well-being and state of mind of

patients since the time of Aristotle. It is thought to be an

essential component of all forms of medicine and healing

(Helman 1984, Lewith 1993).

The power of the placebo-effect has been demonstrated

in a number of studies (for example Dimond et al. 1958,

Cobb et al. 1959, Moscucci et al. 1987). Though the nature

and cause of such effects are poorly understood, they have

been attributed to an interaction of the self-healing

properties of the body, changes induced by therapist and

environment, the cultural context, the power of expect-

ancy and belief and physical/pharmacological interven-

tions (Helman 1984).

Studies suggest patients are searching for subtle clues

about what to expect and that these expectations affect

responses to active or inactive interventions (Bootzin

1985). This raises the question of the comparable nature

of the active intervention and placebo intervention.

Active interventions, for example morphine, may

produce immediate side-effects. It is often possible for

patients who participate in double-blind RCTs to guess

accurately which experimental group they have been

assigned to (for example on the basis of their experience

of side-effects), hence the importance of ensuring blind-

edness at outcome (Moscucci et al. 1987). So a thorough

placebo comparison might involve the use of àctive

placebos', that is the placebo substance should mimic

the active substance without containing the agent

designed to bring about the desired outcome (Finkel

1985, Wall 1996). In complementary therapy, nursing

and many forms of orthodox medicine identifying a

comparable manoeuvre against which to test an inter-

vention (for example acupuncture) is fraught with dif®-

culty (Wall 1996).

Placebos used in acupuncture trials include the use of

needles that are either rubbed against the skin, or glued to

it. However, it seems unlikely that patients would accept

these procedures as credible forms of treatment (Vincent

1993). Alternatively needles may be inserted at incorrect

(theoretically irrelevant) sites as in `sham acupuncture'.

However, there are some doubts as to whether acupunc-

ture and non-acupuncture sites are distinguishable, and

some studies have shown no difference between sham and

real acupuncture sites (Vincent 1993). Vincent suggests

the use of mock TENS treatment or minimal acupuncture

as a possible placebo solution; or, alternatively two types

of control trials, a no-treatment waiting list control, or

acupuncture as an addition to traditional treatment.

Lewith & Machin (1983) argue that sham acupuncture

cannot be considered a control in pain relief studies, as

there is some evidence that chronic pain is attenuated

through mechanisms such as diffuse noxious inhibitory

control.

J. Richardson


PRAGMATIC AND EXPLANATORY STUDIES

One useful approach is to make a distinction between

pragmatic trials, where the intention is to study the whole

policy of a particular approach to therapy, and explan-

atory trials which try to subdivide the policy, separating it

out into the constituent parts (Schwartz & Lellouch 1967).

The explanatory trial is aimed at understanding which

speci®c component of an intervention produces the

outcome; the pragmatic trial is aimed at decision, that is

it seeks to answer the question `Do patients bene®t from

this treatment?' or `Which of two treatments are prefer-

able?' (Schwartz & Lellouch 1967).

One of the differences between pragmatic studies and

explanatory studies is the extent to which protocol viol-

ations and major deviations (such as patient non-compli-

ance) are considered in the interpretation of the results

(Pocock 1983). Pocock suggests that all eligible patients,

regardless of compliance with the protocol, should be

included in the analysis where possible. This pragmatic

approach is sometimes called ìntention to treat analysis'

and provides a more valid assessment of treatment effect-

iveness in actual clinical practice. The alternative èxplan-

atory' approach con®nes the analysis to only those

patients who received therapy according to the protocol.

In most full-scale trials of drug therapy, and for the

majority of health care evaluation, the aim is to evaluate

treatment policies as they relate to actual clinical practice

rather than purely scienti®c evaluations of drug effect. In

such studies pragmatic trials are more important than

explanatory trials (Pocock 1983).

In pragmatic trials ànalysis by intention to treat' is

recommended, thus withdrawals and deviations are

accounted for in the analysis (Altman 1991). Ànalysis of

compliers only' can distort treatment comparisons. This is

particularly important if patients are withdrawn because

of side-effects (as these side-effects and treatment failures

will not then be reported in the analysis). In some trials it

is not easy to include withdrawals, as lack of response

data once a patient is withdrawn leaves no alternative but

to exclude them from the corresponding analysis. It is

therefore important to develop alternatives to the standard

intention-to-treat analysis that can compensate in some

way for non-compliance, for example sensitivity analysis

(Pocock & Abdalla 19982 ).

In the debate regarding the appropriate methodology for

assessing the effectiveness of complementary therapies,

the term pragmatic has a different meaning. It refers to a

research design that allows practitioners to treat patients

individually, in a way more in line with clinical practice,

and capitalizes on rather than restricts the non-speci®c

`placebo' effects (Thomas & Fitter 1997). Knipschild

(1993) suggests the use of trials comparing the best

orthodox treatment and complementary therapy for

patients who seem suitable for both. For example Meade

& Frank (1993) compared chiropractic and outpatient

management for the treatment of low back pain.

Prior to commencing a clinical research trial the nature

of the research questions needs to be clearly de®ned, and

the type of trial (pragmatic or explanatory) stated. Further,

the selected outcome measures should be appropriate,

valid and reliable. Reliability refers to the extent to which

a measure provides consistent results; validity is the

extent to which the results of a test pertain directly to

the characteristic being measured (see Bowling 1991). If

the study utilizes a qualitative approach it will need to

demonstrate rigour in processing and interpreting the data

(Burnard 1991, Forchuk & Roberts 1993, Greenhalgh &

Taylor 1997).

PATIENT SELECTION

Two major problems exist for patient selection to comple-

mentary therapy research trials. The ®rst problem also

relates to trials in orthodox treatments and originates from

the premise that patients play an active part in the

outcome of all treatments (Brewin & Bradley 1989). The

patient's motivation to follow treatment regimens is likely

to be in¯uenced by preferences before treatment is started.

The greater the need for participation (for example

following a special diet) the greater the motivation will

in¯uence outcome. In pragmatic trials the objective is to

discover the effect and outcome of `packages' of manage-

ment, even though the individual components of the

package cannot necessarily be reduced. Most treatments

are complex, consisting of a mix of supposedly active

components with contextual factors such as the way

treatment is given. In explanatory trials the contextual

factors are èqualised' by randomization and arti®cial

constraints. In pragmatic trials contextual factors are

optimized so approximating actual practice, and the role

expectations play in treatment outcome can be evaluated.

Brewin and Bradley suggest an alternative to the conven-

tional RCT is to optimize motivation by ascertaining

patients' preferences prior to randomization and allo-

cating patients to their preferred treatment choice.

Patients who express no preference are randomized to

either treatment or control group.

The second problem relates to patient selection that can

either be based on orthodox or complementary system

diagnosis. Wiegant (1991) proposes four possible ways of

dealing with this, from selection based on orthodox

diagnosis, to selection criteria de®ned by complementary

therapy diagnosis alone.

A pragmatic trial which accounts for patient preference

and which allows practitioners to treat patients on an

individual basis, goes some way to addressing the criti-

cisms of the application of RCTs to the evaluation of



complementary therapies. Blinding should be included in

the design where possible, and every effort made to

identify and control for potential bias in order to ensure

that the outcome is due to the intervention.

SUMMARY AND CONCLUSIONS

Many of the challenges facing research in complementary

therapies are consistent with those facing research in

nursing, indeed a number of nursing interventions (for

example massage) are also de®ned as complementary

therapy. There is clearly a need for good quality studies

evaluating the effectiveness of such therapies, but there is

also disagreement about the use of a `reductionist'

approach. Randomized-controlled trials are thought to be

the `gold standard' of clinical research. They do, however,

present a number of problems, in particular the approach

attempts to `control out' the non-speci®c (placebo) effects

which appear to play a signi®cant role in producing a

positive outcome; issues of external validity and general-

izibility to clinical practice are not straightforward. There

are also dif®culties in identifying suitable `placebo' inter-

ventions, and some evidence that patients are able to

identify whether they are in a placebo or active treatment

group.

The methodological debate appears to be quite complex,

with proponents of different approaches, and often even

different world-views. Many of the arguments focus on the

`gold standard' double-blind placebo control trial and the

possibility of this method of scienti®c inquiry being

transferable to complementary therapies (see Mercer et al.

19953 ). There is, however, a misconception that all RCTs

need to be blinded and that treatments within an RCT

cannot be individualized (Pocock 1993).

The purpose of randomization is to ensure that the

groups (treatment, placebo and/or control) are equivalent,

and that any differences between groups are due to the

interventions and not due to bias or chance. The use of

credible placebos in much of conventional and comple-

mentary therapies is impractical. However, it is practical

to evaluate the treatment on offer compared with either

the best available treatment or a no treatment control

group (Ernst & Resch 1995).

The decision regarding methodological approach and

the use of a placebo-controlled trial will depend on the

research question. Ultimately the methodology should be

appropriate for the question. The question could ask

`What is the patient's experience of therapeutic massage?'

or Ìs osteopathy more effective than physiotherapy for the

treatment of back pain?' Clearly the two questions require

different approaches.

The provision of an evidence-based health service

requires not only explanatory trials to identify the

precise nature of what causes a therapeutic effect, but

also pragmatic trials, which capitalize on the therapeutic

intervention as a `complete package', involving individ-

ualized treatments and an interaction with a health care

provider. Where possible, such pragmatic trials should

be randomized and/or the experimental groups assessed

for equivalence in order to identify potential sources of

bias.

Avis & Robinson (1996) suggest that methodological

positions proposing quantitative or qualitative research

should be put aside in favour of an ìnterdisciplinary'

approach to research. This might, for example, involve the

use of pragmatic clinical trials, which incorporate a

control group, but allow treatment to be carried out in

line with everyday clinical practice. Such trials could

involve group comparisons, as with the conventional RCT,

or adopt a single-case control approach. Greater emphasis

should be placed on the use of con®dence intervals, the

size of the treatment effect and its clinical signi®cance.

Patients who are withdrawn from the study should be

accounted for in the analysis. Where randomization is not

possible, procedures are available for assessing group

equivalence and dealing with biases (Cook & Campbell

1979). Qualitative approaches can provide a rich source of

data that allows the experience of the subject to be

reported and can be used independently or alongside

quantitative approaches. The use of objective measures

can be combined with a subjective patient assessment.

However, such measures will need to be appropriate,

valid, reliable and sensitive to changes over time (Guyatt

et al. 1987).

Nurses need to be aware of the research issues and how

to judge the quality of research evidence in complement-

ary therapy, in order that they can provide appropriate

advice to patients, and move towards integrating effective

therapies into their practice.

References

Aldridge D. (1988) Single-case research designs. Complementary

Medical Research 3, 37±46.

Altman D.G. (1991) Practical Statistics for Medical Research.

Chapman & Hall, London.

Andrews G. (1989) Evaluating treatment effectiveness. Australian

and New Zealand Journal of Psychiatry 23, 181±186.

Anthony H.M. (1993) Clinical research: questions to ask and the

bene®ts of asking them. In Clinical Research Methodology for

Complementary Therapies (Lewith G.T. & Aldridge D. eds),

Hodder & Stoughton, London4 .

Avis M. & Robinson J. (1996) Continuing dilemma in health-care

research. Nursing Times Research 1, 9±11.

Bagenal F.S., Easton D.F., Harris E., Chilvers C.E.D. & McElwain

T.J. (1990) Survival of patients with breast cancer attending

Bristol Cancer Help Centre. Lancet 336, 606±610.

Ballard A.E. (1996) Traditional and complementary therapies

used together in the treatment, relief and control of Chron's

disease and polyarthritis. Complementary Therapies in Nursing

and Midwifery 2, 52±54.

J. Richardson


Bandolier 65 (1994) Pressure Sores. Five Reviews and an RCT.

Anglia and Oxford Regional Health Authority, Oxford.6

Bandolier 12 (1995) Evidence-Base Everything. Anglia and Oxford

Regional Health Authority, Oxford.7

Beecher H.K. (1955) The powerful placebo. Journal of the Amer-

ican Medical Association 159, 1602±1606.

Bowling A. (1991) Measuring Health: A Review of Quality of Life

Measurement Scales. Open University Press, Buckingham.

Black N. (1996) Why we need observational studies to evaluate

the effectiveness of health care. British Medical Journal 312,

1215±1218.

Bootzin R.R. (1985) The role of expectancy in behavior change. In

Placebo: Theory, Research and Mechanisms (White L., Tursky

B. & Schwartz G. eds), Guilford Press, New York, pp. 198±2038 .

Botting D. (1997) Review of literature on the effectiveness of

re¯exology. Complementary Therapies in Nursing and

Midwifery 3, 123±130.

Brewin C.R. & Bradley C. (1989) Patient preferences and rando-

mised clinical trials. British Medical Journal 299, 313±315.

Britten N. (1995) Qualitative interviews in medical research.

British Medical Journal 311, 251±253.

Brooker D.J., Snape M., Johnson E., Ward D. & Payne M. (1997)

Single case evaluation of the effects of aromatherapy and

massage on disturbed behaviour in severe dementia. British

Journal of Clinical Psychology 36, 287±296.

Burnard P. (1991) A method of analysing interview transcripts in

qualitative research. Nurse Education Today 11, 461±466.

Canter C. & Nanke L. (1993) Emerging priorities in complement-

ary medical research. In Clinical Research Methodology for

Complementary Therapies (Lewith G.T. & Aldridge D. eds),

Hodder & Stoughton, London.

Cassidy J.D., Lopes A.A. & Yong-Hing K. (1990) The immediate

effect of manipulative vs. mobilization on pain and range of

motivation in the cervical spine: a randomised controlled trial.

Journal of Manipulative and Physiological Therapeutics 16,

279.9

Clarke M.J. & Stewart L.A. (1995) Obtaining data from randomised

controlled trials: how much do we need for reliable and

informative meta-analyses? In Systematic Reviews (Chalmers I.

& Altman D.G. eds), BMJ Publishing, London, pp. 37±4710 .

Cobb L.A., Thomas G.I., Dillard D.H., Merendino K.A. & Bruce

R.A. (1959) An evaluation of internal mammary artery ligation

by a double blind technique. New England Journal of Medicine

20, 1115±1118.

Cook T.D. & Campbell D.T. (1979) Quasi-Experimentation Design

and Analysis Issues for Field Settings. Houghton Mif¯in,

Boston.

Delamothe T. (1994) Using outcomes research in clinical practice.


Dimond B. (1995) Complementary therapy and the mother's

wishes. Modern Midwife 5, 34±35.

Dimond E.G., Kittle C.F. & Crockett J.E. (1958) Evaluation of

internal mammary ligation and sham procedure in angina

pectoris. Circulation 18, 712±713.

Dunn C., Sleep J. & Collett D. (1995) Sensing an improvement: an

experimental study to evaluate the use of aromatherapy,

massage and periods of rest in and intensive care unit. Journal

of Advanced Nursing 21, 34±40.

Ernst E. & Resch K.L. (1995) Concept of true and perceived

placebo effects. British Medical Journal 311, 551±553.

Ersser S.J. (1995) Complementary therapies and nursing research:

issues and practicalities. Complementary Therapies in Nursing


Finkel M.J. (1985) Placebo controls are not always necessary. In

Placebo Theory, Research and Mechanisms (White L., Tursky

B. & Schwartz G.), The Guilford Press, New York11 .

Fisher P. & Ward A. (1994) Complementary medicine in Europe.


Fitter M.J. & Thomas K.J. (1997) Evaluating complementary

therapies for use in the National Health Service: `Horses for

Courses'. Part 1: The design challenge. Complementary Ther-

apies in Medicine 5, 90±93.

Fitzpatrick R. & Boulton M. (1994) Qualitative methods for

assessing health care. Quality in Health Care 3, 107±113.

Forchuk C. & Roberts J. (1993) How to critique qualitative research

articles. Canadian Journal of Nursing Research 25, 47±56.

Geddes N. & Henry J.K. (1997) Nursing and alternative medicine.

Legal and practice issues. Journal of Holistic Nursing 15,

271±281.

Greenhalgh T. & Taylor R. (1997) Papers that go beyond numbers

(qualitative research). British Medical Journal 315, 740±743.

Guyatt G.H., Walter S. & Norman G. (1987) Measuring change over

time: assessing the usefulness of evaluation instruments.

Journal of Chronic Diseases 40, 171±178.

Hamilton D. & Bechtel G.A. (1996) Research implications for

alternative therapies. Nursing Forum 31, 6±10.

Hayes R.J., Smith P.G. & Carpenter L. (1990) Bristol Cancer Help

Centre. Lancet 336, 1185.

Henry D. & Hill S. (1995) Comparing treatments (Editorial). British

Medical Journal 310, 1279.

Helman C. (1984) Culture, Health and Illness. Butterworth

Heinemann, Oxford.

Heron J. & Reason P. (1984) New paradigm research and holistic

medicine. British Journal of Holistic Medicine 1, 86±91.

James N. & Reed A. (1990) Bristol Cancer Help Centre. Lancet

336, 744.

Jones R. (1995) Why do qualitative research? (Editorial). British

Medical Journal 311, 2.

Kacperek L. (1997) Patients' views on the factors which in¯uence

the use of an aromatherapy massage outpatient service. Comple-

mentary Therapies in Nursing and Midwifery 3, 51±57.

Kleijnen J., ter Riet G. & Knipschild P. (1991a) Acupuncture and

asthma: a review of controlled trials. Thorax 46, 799±802.

Kleijnen J., Knipschild P. & ter Riet G. (1991b) Clinical trials of

homoeopathy. British Medical Journal 302, 316±323.

Knipschild P. (1993) Trials and errors. British Medical Journal

309, 1706±1707.

Lewith G.T. (1993) Every doctor a walking placebo. In Clinical

Research Methodology for Complementary Therapies (Lewith

G.T. & Aldridge D. eds), Hodder & Stoughton, London12 .

Lewith G.T. & Machin D. (1983) On the evaluation of the clinical

effects of acupuncture. Pain 16, 111±127.

Meade T.W. & Frank A.O. (1993) Manipulation and low back pain:

an example of principles and practice. In Clinical Research

Methodology for Complementary Therapies (Lewith G.T. &

Aldridge D. eds), Hodder and Stoughton, London, pp. 411±42013 .



Melchart D., Linde K. & Weidenhammer W. (1997) Systematic

clinical auditing in complementary medicine: rationale,

concept, and a pilot study. Alternative Therapies in Health

and Medicine 3, 33±39.

Mercer G., Long A.F. & Smith I.J. (1995) Researching and

Evaluating Complementary Therapies: The State of the Debate.

Collaborating Centre for Health Service Research, Nuf®eld

Institute for Health, Leeds.

Moher D., Jadad A.R., Nichol G. et al. (1995) Assessing the quality

of randomised controlled trials: an annotated bibliography of

scales and checklists. Controlled Clinical Trials 16, 62±73.

Monro J. & Payne M. (1990) Bristol Cancer Help Centre. Lancet

336, 743±744.

Morse J.M. (1989) Qualitative Nursing Research: A Contemporary

Dialogue. Sage, California.14

Morse J.M. & Field P.A. (1996) Nursing Research. The Application

of Qualitative Approaches. Stanley Thornes, Cheltenham.

Moscucci M., Byrne L., Weintraub M. & Cox C. (1987) Blinding,

unblinding, and the placebo effect: an analysis of patients'

guesses of treatment assignment in a double-blind clinical trial.

Clinical Pharmacology & Therapeutics 41, 259±265.

Norton L. (1995) Complementary therapies in practice: the ethical

issues. Journal of Clinical Nursing 4, 343±348.

Patel M.S. (1987) Problems in the evaluation of alternative

medicine. Social Science Medicine 25, 669±678.

Pocock S.J. (1983) Clinical Trials: A Practical Approach. John

Wiley & Sons, Chichester.

Pocock S.J. (1993) Error and bias in single group and controlled

data trials. In Clinical Research Methodology for Complement-

ary Therapies (Lewith G.T. & Aldridge D. eds), Hodder &

Stoughton, London, pp. 35±3715 .

Pocock S.J., Hughes M.D. & Robert J.L. (1987) Statistical problems

in the reporting of clinical trials. New England Journal of

Medicine 317, 426±432.

Pocock S.J. & Abdalla M.16 (1995) The hope and the hazard of using

compliance data in randomized controlled trials. Statistics in

Medicine 17, 303±317.

Pope C. & Mays N. (1995) Reaching the parts other methodologies

cannot reach: an introduction to qualitative methods in health

and health services research. British Medical Journal 311,

42±45.

Proctor S. (1998) Linking philosophy and method in the research

process: the case for realism. Nurse Researcher 5, 73±90.

Reilly D. & Taylor M. (1993) Developing integrated medicine:

Report of the RCCM Research Fellowship in Complementary

Medicine, The University of Glasgow 1987±90. Complementary

Therapies in Medicine 1(Suppl. 1), 1±50.

Richardson P.H. & Vincent C.A. (1986) Acupuncture for the

treatment of pain: a review of evaluation research. Pain 24,

15±40.

Sackett D.L. (1994) Understanding clinical trials (Editorial).


Sackett D.L., Haynes R.B., Guyatt G.H. & Tugwell P. (1991)

Clinical Epidemiology: A Basic Science for Clinical Medicine.

Little, Brown, Boston.

Schwartz D. & Lellouch J. (1967) Explanatory and pragmatic

attitudes in therapeutic trials. Journal of Chronic Disease 20,

637±648.

Sharma U. (1992) Complementary Medicine Today: Practitioners

and Patients. Tavistock/Routledge, London.

Sheard T.A.B. (1990a) Bristol Cancer Help Centre. Lancet 336,

683.

Sheard T.A.B. (1990b) Bristol Cancer Help Centre. Lancet 336,

1185±1186.

Smith R. (1995) The scienti®c basis of health services. British

Medical Journal 311, 961±962.

Smith R. & Rennie D. (1995) And now, evidence based editing.

British Medical Journal 311, 826.

Stevensen C. (1997) Complementary therapies and their role in

nursing care. Nursing Standard 11, 49±53.

St George D. (1994) Towards a research & development strategy

for complementary medicine. Homoeopath 54, 254±257.

Styles J.L. (1997) The use of aromatherapy in hospitalised

children with HIV disease. Complementary Therapies in

Nursing and Midwifery 3, 16±20.

ter Riet G., Kleijnen J. & Knipschild P. (1990) Acupuncture and

chronic pain: a criteria-based meta-analysis. Journal of Clinical

Epidemiology 43, 1191±1199.

Thomas K.J. & Fitter M.J. (1997) Evaluating complementary

therapy for use in the National Health Service: `Horses for

courses'. Part 2: alternative research strategies. Complementary

Therapies in Medicine 5, 94±98.

Vickers A.J. (1995a) A basic introduction to medical research.

Part ii: an overview of different research methods. Comple-

mentary Therapies in Nursing and Midwifery 1, 113±117.

Vickers A.J. (1995b) What conclusions should we draw from the

data? British Homoeopathic Journal 84, 95±101.

Vickers A.J. (1995c) A basic introduction to medical research.

Part i: what is research and why do it? Complementary

Therapies in Nursing and Midwifery 1, 85±88.

Vickers A.J. (1996) Research paradigms in mainstream and

complementary medicine. In Complementary Medicine: An

Objective17 Appraisal (Ernst E. ed.), Butterworth Heinemann,

Oxford, pp. 3±11.

Vincent C. (1993) Acupuncture as a treatment for chronic pain.

In Clinical Research Methodology for Complementary Ther-

apies (Lewith G.T. & Aldridge D. eds), Hodder & Stoughton,

London, pp. 293±29418 .

Wall P.D. (1996) The placebo effect. In The Science of Conscious-

ness. Psychological, Neuropsychological and Clinical Reviews

(Velmans M. ed.), Routledge, London19 .

Watkins A.D. (1994) The role of alternative therapies in the

treatment of allergic disease. Clinical and Experimental Allergy

24, 813±825.

West Z. (1997) Acupuncture within the National Health Service:

a personal perspective. Complementary Therapies in Nursing


Wiegant F.A.C., Kramers C.W. & van Wijik R. (1991) Clinical

research in complementary medicine: the importance of patient

selection. Complementary Medical Research 5, 110±115.

J. Richardson


Documents

The use of randomized control trials in complementary therapies: exploring the issues