Scientific research should be based on a predeterminedresearch question. On the basis of this question the researcherchooses the type of research that offers the best possibility ofanswering the research question. If the research questionfocuses on the onset or course of a disease, the best choice isan observational study design. If the research questionfocuses on the effectiveness of interventions, an experimentalstudy design or, specifically, a randomised controlled trial ismore appropriate. Critical consumers of scientific literaturemust determine whether the study design chosen is the bestdesign with which to address the research question at issue.

The conduct of scientific experiments symbolises whatresearchers do. In simple language the word ‘experiment’refers to any type of research. For instance, the professionalathlete experiments with various makes of sports shoes.However, scientific experiments have a number of specificcharacteristics which concern how systematic observationsare made under controlled conditions. Many scientistsstipulate one key characteristic of experiments: the researchconditions are manipulated (Rothman and Greenland 1998).However, ethical considerations and practical problems oftenmake scientific experimentation impossible. For example, itis not ethical to expose people to a risk factor just to be ableto study how much influence this factor has on an illness. Inmany cases, such restrictions make it necessary forresearchers to carry out non-experimental (observational)research. There are also methodological considerations whichcan influence the decision to conduct non-experimentalstudies.

From this discussion it is clear that there is no rigid hierarchyin the ‘strength’ of the various study designs, even though arandomised controlled trial is often considered to be the‘strongest’ design and case-control studies are oftenconsidered the ‘weakest’. This simple hierarchy ignores thefact that the research question determines the choice of studydesign. The differences between experimental and non-experimental designs will be illustrated using examples ofresearch on low back pain.

Research question

Scientific research should always be based on apredetermined research question. The researchers then choosethe type of research that offers the best possibility ofanswering the research question. Within the field of low backpain many research questions are still to be answered. Forexample, it is not yet known why some people who do notreceive therapy recover from their back pain and some do not.We also still do not know which factors really induce lowback pain, or which factors can prevent low back pain. Thereare also many unanswered questions with regard to the mostoptimal type of therapy. Some types of therapy arediscouraged (e.g. more than two days of bed rest), butquestions about the optimal therapy remain unanswered (VanTulder et al 1999).

Non-experimental research

If the research focuses on the prevalence or the incidence oflow back pain, observational (descriptive) studies are mostappropriate. ‘Observational’ means that the researcher does notinterfere with events, but charts them systematically. In thisway, Frymoyer reported that 60–90% of the general populationhad had an episode of low back pain at some time, and that theannual incidence of low back pain was 5% (Frymoyer 1988).Such studies can also investigate specific populations. Forinstance, Van Ravensberg and colleagues found that 27% of thepatients who visited a physiotherapist had back complaints(van Ravensberg, Oostendorp and Elvers 1995).

Cohort studies An unanswered question in the field of lowback pain concerns the impact of work-related factors. To beable to give a valid answer to this question, the researchercould carry out a specific type of observational study, a cohortstudy, in which a group of people (cohort) is monitored overtime. Important aspects of a cohort study are appropriateselection of subjects from the study population, exposuremeasurement, and follow-up measurement. Figure 1 showsthe basic design of a cohort study.

Celebrating 50 years of excellence — Australian Journal of Physiotherapy 2004 Vol. 50 189

Research Notes

The architecture of scientific research

RWJG Ostelo1,2, HCW de Vet2 and HJM van Beek3

1VU University Medical Centre, EMGO Institute 2University of Professional Education Amsterdam, Hogeschool van Amsterdam 3University of Professional Education Brabant, Hogeschool van Brabant

This anniversary issue of the Australian Journal of Physiotherapy brings the first of a new feature called ResearchNotes. Research Notes are short explanations of selected aspects of research methodology. Some will be simpleand accessible introductions and others will deal with more complex issues.

The forerunner of Research Notes was a series of articles published by leading Dutch physiotherapy researchersin Nederland Tijdschrift voor Fysiotherapie. A collection of these articles was published as a book. The editors ofthat book have kindly arranged for translation of selected articles for publication as Research Notes in the AustralianJournal of Physiotherapy. These translated Research Notes will be supplemented by contributions from otherexperts in research methods.

The first Research Note, by Ostelo, de Vet and van Beek, is titled ‘The architecture of scientific research.’ It providesan overview of some of the main methods used in quantitative clinical research. Subsequent Research Notes willlook in more detail at specific aspects of the design and analysis of clinical research.

If a cohort study is carried out to investigate the relationshipbetween work-related factors and the onset of low back pain,appropriate selection implies, in particular, that people whoparticipate in the study should not have low back pain at thestart of the study. Subsequently, by means of inclusion andexclusion criteria, the researcher will try to select asspecifically as possible those people about whom he or shewishes to make a statement. For instance, the researcher canrestrict the investigation to people in a certain age group sothat the influence of age will not be a confounder. In any case,if people also have a high risk of low back pain because oftheir age, it is no longer possible to study the pure relationshipbetween work-related factors and low back pain. Theexposure measurement (extent of exposure) will take placeafter inclusion. This will focus on factors in which theresearchers are interested, e.g. heavy and frequent lifting,lifting with trunk rotation, or the work-stress that isexperienced. Follow-up measurements record who has andwho has not developed low back pain. It is then possible tocalculate the magnitude of the risk of developing back pain,for instance among workers who are exposed to heavy andfrequent lifting and those who are not.

At this point it is worthwhile mentioning the concept of‘confounding’. Within the context of a cohort study,confounding is distortion of the observed relationshipbetween a particular risk factor and the development of thecomplaint at issue due to one or more other risk factors. Afactor is a confounder if it: a) is a risk factor for the complaintin question, and b) appears to be distributed unevenly over thecategories of the risk factor being studied (Rothman andGreenland 1988). For instance, the fact that some people areexposed to heavy and frequent lifting is an importantpotential confounder when the aim of the research is to findout whether there is a relationship between postureabnormalities in the spine and low back pain. Of course,heavy and frequent lifting is related to low back pain, and itis also possible that there is an association between the factor

of heavy and frequent lifting and posture abnormalities. A fewsteps can be taken to minimise confounding in the design ofa study (e.g. by ensuring that the potential confounder isdivided equally over the various groups by means of‘matching’). It is also possible, to some extent, to adjust in theanalysis for confounders.

One advantage of a cohort study is its prospective character:the risk factors are usually measured before the onset of thecomplaints. Moreover, measurements are performed atindividual level so that the researchers can determineprecisely which participants are exposed and whether theseare the people who eventually develop back pain. However, acohort study also has limitations. For instance, determiningthe length of the follow-up is often a problem: for how longmust a person be exposed to heavy and frequent lifting beforegetting low back pain? Investigating a rare disease with acohort study is not very efficient, because a great number ofpeople would have to be monitored in order to eventuallyidentify only a few cases. A cohort study is, however, anappropriate design with which to investigate rare exposurefactors, because these can be selected. Another disadvantageis that this type of research takes a long time and the

Celebrating 50 years of excellence — Australian Journal of Physiotherapy 2004 Vol. 50190

Research Notes

Population

Study population(workers)

Exposure factormeasurement

(heavy and frequent lifting)

Exposed (heavy lifting)

Back pain (Yes/No)

Not exposed (no heavy lifting)

Back pain (Yes/No)

Cases(people with

disease)

Retrospectivemeasurement of risk factor

Controls(people free of

disease)

Retrospectivemeasurement of risk factor

Inclusion and exclusion criteria

Figure 1. Basic design of a cohort study.

Table 1. Main differences between cohort and case-controlstudies.

Case-control study Cohort study

Usually retrospective Usually prospectiveRelatively cheap Often costlyQuick results Often long waiting timeStudy population relatively Study population relativelysmall largeAppropriate for rare diseases Appropriate for common

diseasesNot appropriate for rare Appropriate for rareexposures exposuresCan study only one disease Can study more than onebut multiple exposure factors disease but only a limited(explorative) number of exposure factorsOften difficult to measure Less problems with exposure exposure measurementsA major source of bias is A major source of biasconfounding due to is in selective drop-inappropriate selection out and change in of control subjects procedures over prolonged

study periods

Population

Figure 2. Basic design of a case-control study.

researchers should really already have some idea of whichrisk factors can play a role.

Case-control studies Some researchers avoid this disadvantageby ‘turning the study around’. Instead of monitoring healthyindividuals over time (prospectively) they use a retrospectiveapproach. They include cases (in our example, patients whoalready have low back pain) that represent cases from somehypothetical population that produced the cases that wereselected. Then, the objective in selecting controls is to chooseindividuals representative of those who, had they developed thedisease, would have been selected as cases. Additionally, it isof major importance to select these controls independently ofthe risk factor under study. Once cases and controls are selectedthe measurement goes ‘back in time’ in order to identify thepotential risk factors. This is called case-control design and anexample is illustrated in Figure 2.

Case-control studies are obviously a much quicker (andcheaper) research method because it is not necessary to waituntil the participants develop low back pain. However, thisstudy design introduces additional sources of bias. Forinstance, there is the possibility of recall bias: how manypeople can correctly assess how much stress they experiencedin their work five years ago? In other words, when risk factorsare measured subjectively there is the possibility of biasbecause the measurements may be influenced by diseaseitself. If the risk factor can be measured objectively, andcannot be modified by the disease itself, this is less of aproblem. (An example may be measurements of body height.)Table 1 summarises the main differences between a cohortstudy and a case-control study.

Pre-experimental research

So far the research questions we have examined have focusedon the course or onset of low back pain. Another type ofresearch question focuses on the effectiveness of therapy, forexample: ‘What is the effect of physiotherapy on patientswith low back pain?’ In the first instance, when it is notknown at all what effects the therapy has on patients,researchers could study the course of low back pain over timein people who receive a certain type of therapy. Theresearchers select patients with low back pain, describe the

patients’ main characteristics, record the baseline situation,and then describe the situation when the therapy has beencompleted. This type of research is often referred to as pre-experimental research, because there is no control group. Theclinical outcomes are known in patients who received thetherapy, but the outcome can not solely be attributed to thetherapy. Perhaps the course would have been the same if thepatients had received a different type of therapy, or hadsimply awaited the natural course of events. In addition,measurement errors could suggest that there was an effecteven though this was not the case.

Quasi-experimental research

Alternatively, the researcher could also monitor a group ofpatients who do not receive the therapy: a control group. Thispotentially allows the researcher to determine if the outcomesare better with therapy than without therapy. However, thereis a problem when control groups are assembled fromnaturally existing groups because then it is often unclear whatprocedure is responsible for the fact that some patientsreceive the therapy and others do not. Studies which assemblenaturally existing control groups are sometimes referred to asquasi-experimental research. A quasi-experimental studywhich, for example, compares outcomes in patients referredfor exercise therapy with those who were not referred fortherapy may be biased by the referral procedure. In fact,patients the general practitioner considered to be ‘serious’cases were probably referred to the physiotherapist, whereasthe mild cases may not have been referred. For this reason thetwo groups were not comparable at the start of the study. Thiscan be taken into account to a certain extent in the analysis,but it is better if the researcher ensures that the groups arecomparable at the start of the study (that is, it is better if theresearcher manipulates who is in the therapy and controlgroups). Figure 3 shows the basic design of pre-experimentaland quasi-experimental studies.

Experimental research

Randomised controlled trials A randomised controlled (orclinical) trial is the design that researchers should choose if,for instance, they wish to compare the effect of physiotherapywith treatment provided by the general practitioner for low

Celebrating 50 years of excellence — Australian Journal of Physiotherapy 2004 Vol. 50 191

Research Notes

Treatment A

Post-treatmentmeasurement

Follow-upmeasurement

Treatment B

Post-treatmentmeasurement

Follow-upmeasurement

Population

Baselinemeasurement

Treatment

Post-treatmentmeasurement

Follow-upmeasurement

Population

Baselinemeasurement

Pre-experimental study Quasi-experimental study

Figure 3. Basic design of pre-experimental and quasi-experimental studies.

back pain. Figure 4 shows the basic design of a randomisedcontrolled trial.

First of all, as in observational research, suitable people mustbe included. Subsequently the researcher must ensure that, bymeans of a randomisation procedure, each patient has anequal chance of being allocated to one of the two groups. Inprinciple randomisation ensures that at the start of the studythe two groups are comparable, which implies that the peoplein both groups, on average, have an equal chance of recovery.After the intervention the researcher evaluates the effect ofboth types of treatment, using predetermined measurementinstruments. In order to evaluate the long-term effects as well,a follow-up measurement often takes place after severalmonths (depending on the research question).

Specific features of observational studies and experimentalstudies will be considered in more detail in subsequentResearch Notes.

ReferencesFrymoyer JW (1988): Medical progress, back pain and sciatica.

New England Journal of Medicine 318: 291–300.

van Ravensberg CD, Oostendorp RAB and Elvers JWH (1995):Inventory of physiotherapy in primary care. [Dutch.] NederlandTijdschrift voor Fysiotherapie105: 36-58.

Rothman KJ and Greenland S (1998): Modern Epidemiology (2nded.) Washington: Lippencott-Raven Publishers.

Van Tulder MW, Koes BW, Assendelft WJJ and Bouter LNM(1999): The Effectiveness of Conservative Treatment of Acuteand Chronic Low Back Pain. Amsterdam: EMGO Institute.

Celebrating 50 years of excellence — Australian Journal of Physiotherapy 2004 Vol. 50192

Research Notes

Figure 4. Basic design of a randomised clinical trial (RCT).

Population

Study population (patients with

back pain)

Baselinemeasurement

Randomisation

Treatment(general practitioner)

Post-treatment(recovered yes/no)

Treatment(physiotherapist)

Post-treatment(recovered yes/no)

Follow-up(recovered yes/no)

Follow-up(recovered yes/no)

Inclusion and exclusion criteria

* This research note was previously published as: De architectuur van wetenschappelijk onderzoek (2001). Nederland Tijdschriftvoor Fysiotherapie 111: 22–25; and as a chapter in Ostelo RWJG, Verhagen AP and de Vet HCW (Eds) (2002): Education inScientific Research. [Dutch.] Houten, Bohn Stafleu van Loghum.