Re

earclica

berthit B

WOR

od

WwoThbomalartimanwhAnscientific research methods. Since sound research meth-ods are the backbone of a good study, the methods mustbe solid to ensure that the results are valid. If the methodsaregode

The purpose of this Research Methods Handbook isto provide basic information about common researchtechniques, how to conduct a good study, how to writea m

wi

neisonCOtaivemostaanwhstaPearls and pitfalls are discussed and tips are given. Wedare say that if you follow these simple guidelines,you will have a much greater chance of getting your

helped out with all practical details and negotiated allnecessary contracts. At Arthroscopy, Managing EditorHank Hackett and Jason Miller from Elsevier made

TA

meSE

0danuscript and, we hope, how to get it published.The work is divided into several sections, startingth an overview on evidence-based medicine; much-

things happen. Special thanks to Hank for his profes-sional editing work on all chapters, keeping track ofthe time frame, and all other practical details.

This work is an ISAKOS project, done on behalf ofthe ISAKOS Scientific Committee, and we would like tothank all Committee members, many of them co-authors,for their part in getting this done. Special thanks to MarioFerretti, Stephan Lyman, Rob LaPrade, Bruce Levy,Nick Mohtadi, Kevin Shea, Michael Soudry, and StefanoZaffagnini. We also extend our thanks to all other co-

he authors report no conflict of interest.ddress correspondence to Jn Karlsson, M.D., Ph.D., Depart-

nt of Orthopaedics, Sahlgrenska University Hospital/Mlndal,-431 80 Mlndal, Sweden. E-mail: jon.karlsson@telia.com.

2011 by the Arthroscopy Association of North America749-8063/1194/$36.00not good from the beginning, the outcome will not beod either, and the manuscript will not be publishedspite the investigators best effort.

manuscript published.A few words of thanks. First and foremost we thank

Michele Johnson, ISAKOS Executive Director, whoISAKOS Scientific Committee

A Practical Guide to Resand Pub

Editors: Jn Karlsson, M.D., Ph.D., RoNorimasa Nakamura, M.D., Ph.D., and Mo

FORE

Why Is a Research Meth

hy is this work needed, and who would benefitfrom it? First of all, we must realize that this

rk is on a high but at the same time moderate level.e aim is to put together a Research Methods Hand-ok that can be of practical help to those writingnuscripts for submission to Arthroscopy and simi-journals. We are referring to people working fulle, taking care of patients, with busy outpatient clinics

d fully booked surgical schedules. These are personso do not devote the majority of their time to research.d in most cases they do not have any major training inauoi:10.1016/j.arthro.2011.02.001

Arthroscopy: The Journal of Arthroscopic and Related Surgerysearch Methods Handbook

h: Design, Execution,tion

G. Marx, M.D., M.Sc., F.R.C.S.C.,handari, M.D., Ph.D., F.R.C.S.C.

D

s Handbook Needed?

eded information for all clinicians. The second sectionconcerned with study methods, with special focusstudy designs. Important scientific methods, likeNSORT and STROBE, are explained in greater de-

l. The third section is on biostatistics. This section isry practical, written with the clinician in mind. Com-n statistical methods are explained and the aim is toy practical and pragmatic. We are still cliniciansd not statisticians. And the idea is to help clinicianso are conducting a study and not to make themtisticians. The last section is on manuscript writing.thors, with special thanks to Sabine Goldhahn. Mohit

S1, Vol 27, No 4 (April, Suppl 2), 2011: pp S1-S112

Bhandari, one of the greatest clinician researchers wehathi

in-co

thema

readers of Arthroscopy as well as other journals willbeJNChISA

FRPr

ION

M

TgretioanjoilesgeweEvtop

cincoHo

corner store. As you are on the way to the site of thecrime, you consider the last robbery you investigated ata store on the other side of town. Reminding yourself of

happened; as you arrive at the scene, you have alreadypieced together the majority of the case. But whatabout this footprint? Does that change your hypothesisas to what went on . . . ?

demy

beha

These examples aim to illustrate, in albeit rudimen-tary terms, the paradigm shift that our field has been

best available research and their clinical expertise withthe specific patients values.

The first steps of EBM (the evidence) are verysimsecodevchtheme

S2 J. KARLSSON ET AL.Now lets consider instead that you are this sametective but have since watched a Sherlock Holmesstery video and have taken some of his words to heart.

You are en route to the site of this same robbery. Whiledriving there, you try to clear your mind of the robberyyou investigated last week. You want to approach thisilar to steps used in detective work as shown in theond example. This section will introduce the meth-

s with which to approach a problem and track downidence. The medicine piece of EBM is where thingsange. When it comes to solving the problem with

evidence you have gathered, one could argue thatdicine in fact has better tools at hand than a detec-how the last crime was conducted, you proceed todevelop a theory as to how the current robbers enteredthe store, their path throughout the store, what theystole, and how they escaped. Yes, that must be how it

undergoing for the past decade. Over the last severalyears, medical professionals and health professionalshave begun using EBM in their practice: integratingve ever met, deserves special thanks; without his work,s project would never have been possible.Finally, Gary Poehling and James Lubowitz, Editor-Chief and Assistant Editor-in-Chief of Arthros-py, supported the project from the start and sharedir knowledge and vast experience in the section onnuscript writing. Thank you both. We hope that the

SECT

What Is This Evidence-Based

he British Medical Journal recently surveyed theglobal medical community to determine the

atest medical breakthroughs since its first publica-n in 1840.1 It was an incredible period of innovationd change, when antibiotics were discovered, entirents were replaced with anything from ivory to stain-s steel, internal imaging was developed allowing sur-ons to see inside the body noninvasively, and vaccinesre developed and implemented on a global scale.idence-based medicine (EBM) was noted as 1 of the15 medical breakthroughs in the last 160 years.

BIAS DETECTIVES

Many have compared the use of evidence in medi-e to the use of evidence in the legal setting.2 Let us

nsider the classic character Detective Sherlocklmes and a legal example to set the stage.

You are a detective called to a robbery of a localnefit from this work.KARLSSON ROBERT G. MARX NORIMASA NAKAMURA

air Co-chair Co-chairKOS Scientific Committee

EDDIE H. FUesident of ISAKOS

2

edicine and Why Bother?

new case with no preconceived ideas or theories.As Sherlock said, Never guess. It is a shockinghabit . . . .2 You arrive at the site of the crime andbegin locating evidence: a black glove here, a footprintthere, a broken window in the front of the store, and awide-open door at the back. You attempt to collect allthe evidence you can find before developing a hypoth-esis as to the events of the robbery. Your mind recallsa quote from the detective video the night before, . . .dont theorize before you have all the evidence.2Remembering how observation was second nature toHolmes, you ensure you collect all the facts and recordall that you observe even if the information does notappear immediately relevant. Now its just a matter ofsitting down with the evidence and solving the crime.

Which one of these approaches would stand uptter in court? Which one would the store owner beppiest about in terms of having justice served?

REFRAMING THE PARADIGM TOEVIDENCE-BASED MEDICINE

tive would have available. This chapter will exploretheevscewh

W

towclireswiThcenthemathainvplaan

W

thenociatodiascepoing

W

exWthedeulaint

ticcaprotareMex

icine and across the world, including orthopaedics atanbenaJoicaOras

paacstojousanbejouregshsioon

ex20indtiotothe

Ea

theveanevreppahisenclu6-ylonofgeaftsusiowaofOnansu

S3RESEARCH METHODS HANDBOOKse tools. Lastly, applying this solution based on theidence you have gathered for a patients specificnario has no parallels to detective work; this isere our clinical expertise really comes into play.

hat Is Meant by Best Evidence?

If research is going to be used as evidence putard a hypothesis, which will in turn be applied to a

nical scenario, it should aim to be best evidence. Thisearch has to be relevant with regard to content but alsoth regard to what type of patient is being considered.is can range from research in basic science to patient-tered clinical research, from efficacy and safety ofrapeutic regimens to the power of certain prognosticrkers. The most updated clinical research does moren simply suggest new approaches, it can in fact oftenalidate older diagnostic tests and treatments and re-ce them with ones that are more efficacious, powerful,

d accurate and safer.

hat Is Meant by Clinical Expertise?

Even if research can invalidate older tests and replacem with newer tests, in terms of the best approach,thing can replace clinical expertise. Without a clini-ns ability to use his or her skills and past experiencesidentify the issues and a patients health status andgnosis, as well as risks and benefits present in thenario, we would be hard pressed to have a starting

int at which to apply mounting evidence from meet-s, symposia, and peer-reviewed journals.hat Is Meant by Patient Values?

Each patient is a unique person with his or her ownpectations, concerns, priorities, and preferences.hen each patient brings these unique components to

clinical encounter, it is not in vain. For clinicalcisions to be relevant and able to serve this partic-r patient, his or her unique considerations must beegrated into the decision-making process.

EBM THROUGH THE AGES

The term evidence-based medicine, a medical prac-e paradigm first introduced in the early 1990s, firstme to light as a component of the medical residencygram at McMaster University in Hamilton, On-

io, Canada.3 What started as the introduction ofnlightened skepticism to a group of residents atcMaster University led to an explosion of researchtending this initial idea to many specialties in med-international level. The methodology of EBM hascome a key component of orthopaedics with jour-ls such as Arthroscopy, The Journal of Bone andint Surgery, Indian Journal of Orthopaedics, Clin-l Orthopaedics and Related Research, and Actathopaedica embracing evidence-based orthopaedicsstandard vernacular in their proceedings.The concepts we now consider associated with theradigm of EBM may have roots in ancient historicalcounts of authoritative teaching and passing on ofries in ancient times or the emergence of personalrnals and the introduction of textbooks in Renais-ce times.4 In the early 1990s knowledge began toshared more easily in textbooks, and peer-reviewedrnals began making an appearance in the field withard to clinical practice. It was in the 1970s when a

ift in modern technology and essentially an explo-n in the field of informatics led to the emergence ofline journals and large databases.Claridge and Fabian4 provide a variety of specificamples of EBM emerging through the ages in their05 report on the history of EBM. These examplesicate a gap in knowledge and a subsequent ques-n, an approach to finding evidence, and an answerthe clinical query based on said evidence. Some ofse examples are summarized in Fig 1.

rly Evidence in Orthopaedics

During the time of these more recent developments,orthopaedics community was in the midst of de-

loping its own evidence in the same way. Hopped Bhandari5 present an interesting example of earlyidence in orthopaedics by discussing a particularort from the Proceedings of the American Ortho-

edic Association in 18896 in their article on thetory of evidence-based orthopaedics. This report,titled Hypertrophy of One Lower Extremity, in-des a case study regarding the treatment of aear-old child with a leg three-quarters of an inchger than the other.6 After failing to slow the growththis leg using a rubber bandage, the surgeon sug-

sted a shoe lift for the patients comfort. However,er the patient had later been examined by anotherrgeon, who diagnosed him with congenital occlu-n and dilation of the lymph channels, amputations recommended and carried out. After publicationthis case, a discussion with other specialists ensued.e surgeon described a similar leg-length discrep-

cy presentation in a 21-year-old woman. After con-ltation with a colleague who was also unsure of the

nathesustr

tofrotioproothcaan

well, would often implement a given treatment oftecsta

beona cpuinvthenopascaidebe

M

apintospca

relveuaincplaUnexica

coinbytivphtiewiincjoiGrtheanBaPr

Em

tha

FIGCla

S4 J. KARLSSON ET AL.ture of the problem, a high shoe was also given topatient. A third case was brought up by yet another

rgeon where a similar presentation was treated byetching of the sciatic nerve.With 3 experts offering 3 very different opinions ashow to proceed with such a presentation rangingm a shoe lift to sciatic nerve stretching to amputa-n, how were readers expected to know which ap-ach to use themselves? Historically, as in manyer specialties, information obtained on a case-by-

se basis by experts was passed on to other doctorsd learners who, knowing the experts reputation

URE 1. Examples of EBM through the ages adapted fromridge and Fabian.4hnique into their practice with a reinforced under-nding of its value.Despite these differing expert opinions undoubtedlying a common scenario in all specialties at this time,e contributor suggested a new approach to this lack ofonscience. Would it not be in accordance with the

rposes of this association to appoint a committee toestigate this subject, taking patients . . . and treatingm.6 This is an early example of anecdotal evidencelonger being sufficient as evidence on which to base

tient treatment. It was instead determined that larger-le trials would allow these surgeons to objectivelyntify the superior treatment and to demonstrate the

nefits of one approach versus the next.

odern-Day EBM

From hearsay practices in ancient times to the firstpearance of the randomized controlled trial (RCT)the early 20th century and from anecdotal evidencethe development of evidence through trials in manyecialties including orthopaedics, we arrive at whatn be referred to as modern-day EBM.In the early 1970s, Cochrane7 criticized the lack ofiable evidence behind a plethora of health care inter-ntions commonly accepted at the time. Rigorous eval-tion of these interventions highlighted the need for anrease in evidence in medicine after this publication,nting the seed for EBM. David Sackett of McMasteriversity used the term critical appraisal to describe

tracting evidence from systematically examined med-l literature in the early 1980s.8The actual term evidence-based medicine wasined by Dr. Gordon Guyatt of McMaster University1990. Initially a term intended for educational useinternal medicine residents in McMasters innova-e residency program, EBM gained popularity withysicians and residents in a variety of subspecial-s.9 An initial group of physicians from McMasterth a particular interest in critical appraisal grew tolude specialists from a variety of institutions whoned forces to create the Evidence-Based Workingoup. This group became responsible for adoptingidea of EBM and presenting it in the pivotal report

nouncing it as a new medical paradigm: Evidence-sed Medicine: A New Approach to Teaching theactice of Medicine.10

ergence of EBM

There were many specific changes during this timet really set the stage for the rapid widespread rec-

ognition of EBM. There were realizations of gaps incliEBAlanwa

Wmoinfrea

hasispaCothaouthiinf

skpratiofinboresofvaingouteatraasiingtioanresgivsu

hasuedastraloinfformaitycoclu

clinical studies and an increased need for objective, in-foresticathejoujouthatot15

oftoresinrevevticseaFoMfiranmibiladLiEBandeemwhof

thecalcumapreancacstejou

S5RESEARCH METHODS HANDBOOKnical decision making preceding the coining ofM, creating a real need for this paradigm shift.

ongside this, recent developments in technologyd perspectives fostered an environment where EBMs really able to blossom in tackling these gaps.As we approached the era of the Evidence-Basedorking Group introduced earlier, it was becomingre and more evident that traditional venues forormation were no longer sufficient. Several specificlizations set the stage for this spread of EBM.

Regardless of specialty, all physicians and surgeonsve a continuous need for valid information on diagno-, therapy, prevention, and prognosis for numeroustients with countless conditions and circumstances.vell et al.11 suggested on the basis of their research,t new information is needed 2 times for every 3tpatients. Another study, performed in 1991, added tos suggestion stating that physicians may require newormation up to 5 times per inpatient.12Regardless of the increasing need, the tools andills surgeons have typically been left with once inctice are no longer sufficient to acquire informa-n as needed. In the past, traditional venues fording information such as medical and surgical text-oks have been based on expert opinion rather thanearch and are in fact frequently wrong. The volumeinformation in these sources combined with the

riability in their validity makes them an overwhelm-source of information to sift through. In addition,

tside of written information, traditional didacticching is also often ineffective when it comes tonslating knowledge into clinical practice. All of thisde, for many clinicians, the main barrier to engag-

in best research to find answers to clinical ques-ns is time. With a busy clinic, operating room time,d call schedule, finding time to sit down, search forources and assimilate information to study anyen topic has often fallen outside the scope of most

rgeons typical daily schedules.There have been various recent developments thatve really allowed these previously insurmountable is-s to be tackled, allowing EBM to become a day-to-

y reality for full-time clinicians and surgeons. Newategies for searching for and appraising research,ngside an increase in the quality and the availability oformation, have brought evidence-based practice to theefront. In addition to the increase in amount of infor-tion, we must also acknowledge the increases in qual-of research. When improvements in research are

nsidered, a few main examples stand out. This in-des an increase in recognition of the importance ofmed consent from patients, as well as a trend forablishing globally based gold standards for best med-l practice.13 A study by de Solla Price showed thatre has been an increase in the number of scientificrnals by 7% per year. At this rate, the number ofrnals has doubled every 10 to 15 years, suggestingt by the early 21st century, we were approaching aal of 60,000 to 70,000 journals worldwide, of which,000 were strictly biomedical.14Although this seems like an insurmountable amountinformation, developments in technology have ledprograms that can bring the newest valid, reliableearch from a variety of sources in a concise formata matter of seconds. The availability of systematiciews, medical databases, the Cochrane Library, and

idence-based journals, for example, focusing on ar-les of immediate clinical use, has brought best re-rch and clinical decision making closer than ever.r example, in 1997, when the National Library ofedicine announced it was offering free access to thest-line Web-based medical databases MEDLINEd PubMed, usage jumped 10-fold, to a total of 75llion searches annually.15 Availability and accessi-ity of information have also increased with thevent of second-line databases such as the Cochranebrary, UpToDate, and Best Evidence along withM-related journals such as the ACP Journal Club

d Evidence-Based Medicine (these resources will betailed further later on). These changes, alongside theergence of the idea of lifelong learning, explainy there has been such a sudden surge in the conceptEBM not only in theory but also in practice.

THE PRACTICE OF EBM

As discussed earlier, a doctors clinical competence iscombination of 3 main aspects: knowledge, techni-

/clinical skill, and the ability to make decisions. Themulative factor of this combination is the ability toke appropriate, systematic, and unbiased decisions todict prognosis and interpret the results of examination

d laboratory tests to overall achieve therapeutic effi-y. In 1995 Haynes and Sackett16 summarized the keyps of practicing EBM in the opening editorial of thernal Evidence-Based Medicine as follows.1. Formulate the problem and convert the informa-

tion needs into answerable questions.2. Search for and assimilate in the most efficient

way possible, the best evidence with which toanswer these questions. This information comes

from the clinical examination, laboratory tests,

tratimimfroco

ofortpauathroffinnicfurpoagwicocatMOrbesuf

Amingtoknpreanthiinplevaweco

Asking a Well-Built Research Question

apropanepromawiasoalobefacsk

clitionoqu

Co

PI

an

Hac

Predm

e

Diao

ds

Thw

tPro

c

a

S6 J. KARLSSON ET AL.diagnostic imaging, or published literature.3. Once collected, appraise the evidence critically

for both its validity and its applicability to thecurrent clinical question.

4. Apply the results of this search and appraisal inpractice to both the clinical question and patientcontext.

5. Evaluate the above steps.

These steps will be outlined in further detail, illus-ting how surgeons taking as little as 30 minutes ofe per week for their professional development can

plement EBM into their practice to answer anythingm the everyday common complaint to the lessmmon complaint to the rare presentation.17

EBM AT WORK

Knee pain is among the most common complaintspatients seen by both primary care physicians andhopaedic specialists. Despite how often this type oftient presents, many clinicians still struggle with eval-ting knee pain. After developing a clinical pictureough discussion of the nature of the pain, mechanisminjury, patients history, relevant physical examinationdings, and preliminary diagnostic imaging, many cli-ians are still unsure of how to proceed with regard tother investigation. With no clear diagnosis at thisint, does this patient need a magnetic resonance im-ing (MRI) scan? Will this add to the clinical picture, orll this not provide any new information? When itmes to efficiency and economy of practice and allo-ion of resources, being able to determine whether anRI scan is required in this presentation is essential.dering an MRI scan because you always do orcause a mentor has always suggested it is no longerficient evidence to warrant proceeding.Recent research by Matzkin et al.18 presented at the

erican Association of Orthopaedic Surgeons meet-in 2011 has produced an evidence-based algorithm

determine the need for an MRI scan in evaluation ofee pain. By considering duration of symptoms,sence of an effusion, laxity, joint-line tenderness,

d the degree of radiographic degenerative change,s algorithm will indicate the need for an MRI scanthis situation. This algorithm is an excellent exam-

of how evidence derived from well-conducted,lid, and reliable research is coming to the surface asspeak, influencing our standard of care in the most

mmon presentations.As mentioned, formulating, building, and focusingclinical question comprise the first step in an ap-ach to using EBM in practice. Every time we see a

tient, for the first or fifth time, there is a need forw information about some component of our ap-ach: the presentation, diagnosis, prognosis, ornagement. These gaps in knowledge combinedth our limited time to devote to research necessitatefocus on efficiency. Our gaps in knowledge canmetimes seem rather large, so with this in mind,ngside our limited time to devote to this, we mustas efficient as possible in our search. The first keytor in keeping this step efficient is to become

illed at formulating answerable clinical questions.Questions commonly arise regarding anything fromnical findings, differential diagnoses, manifesta-ns, harm, and etiology to therapy, prevention, diag-stic tests, and prognosis. Examples of such commonestions are shown in Table 1.

mponents of a Good Question1. The patient context, problem, or population in

question2. The potential intervention, exposure, or maneuver3. The approach/option to which this intervention

is compared4. Clinical outcome of combining the above 2 fac-

tors considered in a specific timelineThese 4 components, identified with the acronymCO, are detailed below.Patient Characteristics: To set a good context fory question, clinicians must first identify and consider

TABLE 1. Common Questionsrm/etiology: Questions of identifying and understanding theause for a condition or disease.vention: Questions related to reducing the chance of a diseaseeveloping. This involves identifying and understandingodifiable risk factors associated with the condition as well as

arly screening techniques and standards.gnostic test: Questions related to selection and interpretationf diagnostic tests and, from this, how to confirm or exclude aiagnosis. This involves consideration of a tests specificity,ensitivity, likelihood ratios, cost, risks, and so on.

erapy: Questions related to selecting appropriate treatments,eighing the associated risk/benefits, and efforts/costs of using

hem.gnosis: Questions related to estimating the likely clinicalourse for a given patient over time and any complicationsssociated with this.

the patients characteristics. This involves demographicinfsocdecliinccarhe

tioexveor

asselsintit itesan

theWantretypgalittco

forpalar

improved functional outcome when compared with

leadenolescothe

groavtifscrsig

hiepy

Desig

Me view uSys tensive

m marizedq

Ra ental ste more gt

Pro cy ofb

Co nt is fw al, andr

Ca on area reliablo

Ca are revc le fore

S7RESEARCH METHODS HANDBOOKormation such as age, sex, and race, alongside theirial situation, resources, and values. In addition to this

mographic information, characteristics specific to thenical situation such as diagnoses or condition must beluded. The setting (inpatient, outpatient, rural, tertiarye, and so on) must then be considered. Is this a public

alth issue or an individual patient issue?Intervention: In forming a well-built clinical ques-n, the intervention must then be included. What is itactly that is being considered as a potential inter-ntion? This could be a medication, a diagnostic test,any other type of treatment.Comparison: A treatment or test can really only beessed relative to or in comparison with somethinge. One side of this comparison will be the potentialervention, and the other will be that against whichs being compared. This comparison may be anothert or treatment, the current standard treatment, watchd wait, or even no treatment at all.Outcome: Once the above are determined within

clinical question, include the outcome as well.hat is the desired effect you want to achieve? Is thereeffect you want to avoid? This can involve not onlyatment effects but also side effects. Outcome willically be divided into a primary outcome and surro-

te outcomes (measurements that on their own holdle value for the patient but are associated with out-mes that are considered very important to patients).Instead of asking, Is operative treatment indicated

a fractured clavicle? ask, In an active adulttient with a completely displaced midshaft clavicu-

fracture, would primary plate fixation result in

TABLE 2. Study

ta-analysis: A combination of all of the results in a systematic retematic review: On the basis of a specific clinical question, an exethodology. These studies are then reviewed, assessed, and sum

uestion at hand.ndomized (clinical) control trial: A prospective, analytic, experimnvironment. A group of similar individuals are divided into 2 orreatment[s]) and the outcomes are compared at follow-up.spective, blind comparison to a gold standard: To show the efficaoth the test being investigated and the gold standard test.

hort study: A large population with a specific exposure or treatmeith a similar but unaffected group. These studies are observation

easons aside from the exposure.se-control study: Patients who have a specific outcome or conditipproach used to identify possible exposures. These are often lessften correlational rather than causative.

se series/report: Reports on the treatment of an individual patientontrol group for comparison. Case reports do, however, have a roxist in these cases.noperative treatment at 1 year of follow-up?By using the PICO model to develop a specific andmplete clinical question, the task of finding bestidence becomes more plausible and efficient.

nding the Evidence in the Literature

Developing techniques for searching for evidence maym daunting. Considering that MEDLINE adds 4,500ords to its database on a daily basis, a physician in anye field would need to read 18 articles per day, 365 daysear, to be able to keep up with this amount of re-rch11: hence, daunting. This type of reading schedule

not plausible for any busy clinician or surgeon. Add tos that, in fact, only 10% of these articles are consid-d to be high quality and clinically relevant, and thisk seems even less plausible.11 In reality, however, byrning how to effectively approach a search for evi-nce, learning where to look and what techniques to usew, this job on a day-to-day basis becomes increasinglys daunting. In this section we will discuss various keyncepts, tips, and approaches to develop ways to find

evidence in an efficient and effective way.When first approaching the vast and continuallywing number of scientific and medical articles

ailable, an easy first step is to understand and iden-y the different types of research study designs. De-iptions of the different type of research study de-ns are listed in Table 2.From here, the types of research are placed in ararchy based on their value. Figure 2 illustrates theramid, or hierarchy of evidence, that reflects the

ns Definedsing accepted statistical methodology.

literature search is conducted identifying studies of soundaccording to the predetermined criteria related to the

udy that uses data generated typically in the clinicalroups (1 acting as a control and the other[s] receiving the

a test, patients with varying degrees of an illness undergo

ollowed over time. The outcomes of this group are comparedthey are not as reliable because the 2 groups may differ for

compared with those who do not. This is a retrospectivee than RCTs and cohort studies because their findings are

iewed. These have no statistical validity because they use nonovel and rare presentations, because no large populationsno

coev

Fi

seerecona yseaisthieretas

relprethaarcthetypinganingAsresclipamilesthrmo

latbeonterTaanevabeffitocli

fortexsorelOvpli

thesomu

S8 J. KARLSSON ET AL.ative authority of the different types of researchsent in the biomedical field. It is important to notet although there are various versions of this hier-hy and there is no universally accepted version,re is still agreement on the strength of certain keyes of research relative to the others. By understand-how different types of research compare to one

other, those that are most useful for a busy practic-clinician with a specific question can be targeted.you move up this pyramid, the quality of the

earch increases, in that it is most relevant to thenical setting and has a reduced risk of bias com-red with modes of research lower down the pyra-d. In addition, research higher up the pyramid putss onus on the searcher with regard to filteringough original data, making such research a muchre efficient means of locating information.

Filtered Resources: With a clinical question re-ed to the course of action/management of a patient,it related to diagnosis, treatment, prognosis, and so, filtered resources should be consulted first. Fil-ed resources (examples of which are shown inble 3) consider a question posed by clinical expertsd topic specialists and then provide a synthesis ofidence to come to a conclusion based on all avail-le research. Using filtered information is much morecient because the searching clinician does not needindividually appraise each piece of evidence. Thenician still has a responsibility to evaluate the in-mation with regard to the specific patient and con-t in question. To aid with this portion, these re-

urces also back up information with links to theevant literature and resources. When searching inid and PubMed, clinical filter options can be ap-ed to aid in finding EBM research.Unfiltered Resources: If an appropriate answer to

clinical question is not found in the filtered re-urces, the primary literature or unfiltered resourcesst be considered. Unfiltered resources also providemost recent research and can be used to determine

FIGURE 2. Hierarchy of evi-dence. This image separates thedifferent types of research into3 categories: background infor-mation, unfiltered information,and filtered information.

TABLE 3. Examples of Filtered Resourcesystematic reviews and meta-analyses

Cochrane Database of Systematic Reviews (The CochraneCollaboration)Database of Abstracts of Reviews of Effects (DARE;National Institute of Health Research)

ritically appraised topics (evidence syntheses)Clinical evidenceInfoPOEMs (Canadian Medical Association)ACP PIER (Physicians Information and EducationResource; American College of Physicians)National Guideline Clearinghouse (Agency for HealthcareResearch and Quality)

ritically appraised individual articles (article synopses)Evidence UpdatesBandolierACP Journal Clubthe

SX

X

CXXX

X

CXXX

whsinrelthastuquciemois

thesecproanan

scagaapofMranadtypity

The most updated of these detailed scales can be

rieabantheto

cugopatiopeplapathecothrshaa cbe

TABLE 4. SORT Rating System

Co

ABC

E Ra

Co

A very uuality ss, 1 lar

B likelyay cha

ity studwith s

C very liect andtudies

D fect is

arch evtudies

S9RESEARCH METHODS HANDBOOKether any new strides have been made in this areace the conclusions in the filtered resources wereeased. The challenge with unfiltered resources ist the onus is put on the clinician to evaluate eachdy to determine its validity and applicability to theery at hand. Searching for these resources effi-ntly and subsequently appraising what is found takere time and skill, which is why filtered information

typically considered first.MEDLINE is considered the database of choice forhealth sciences because it provides both primary andondary literature for medicine and other allied healthfessionals. In these instances, RCTs, meta-analyses,

d systematic reviews are considered the gold standardd should be considered first.Ratings of Quality of Evidence: Various ratingles have been developed to help the busy clinician

uge the quality of research based on an externallyplied rating before starting the critical appraisal steppracticing EBM. The Centre for Evidence-Based

edicine in Oxford provides 3 different rating scalesging from 1 to 5, each number and occasionally

ded letter identify the level of evidence based one of research design and various measures of qual-, such as confidence intervals and randomization.

de Definition

Consistent, good-quality patient-oriented evidenceInconsistent or limited-quality patient-oriented evidenceConsensus, disease-oriented evidence, usual practice,

expert opinion, or case series for studies ofdiagnosis, treatment, prevention, or screening

TABLE 5. GRAD

de Quality of Evidence

High Further research is Several high-q In special case

Moderate Further research isof effect and m

One high-qual Several studies

Low Further research isestimate of eff

One or more sVery low Any estimate of ef

Expert opinion No direct rese One or more scessed at www.cebm.net.Strength of Recommendation Taxonomy (SORT)ith codes A, B, and C) is a straightforward ratingstem,19 shown in Table 4.Grading of Recommendations Assessments, Devel-ments and Evaluation (GRADE) is a rating systemveloped by the GRADE Working Group in 2007,20own in Table 5.

tegration With Clinical Expertise Into PracticeArguably the most important aspect of EBM, or theal of EBM if you will, is to integrate best evidenceth clinical expertise for best treatment of a patient.e ability to integrate best evidence with clinical expe-nce into practice is 2-fold: (1) one must be comfort-le and capable in utilizing EBM in his or her practice,d (2) one must be able to understand and incorporate

patients needs and wants to establish the best coursefollow in terms of treatment and management.When using the approach to practicing EBM dis-ssed in this chapter, it is important to recall that theal is to combine evidence, clinical experience, andtients rights and perspectives to determine the solu-n. The importance of patients perspectives, beliefs, ex-ctations, and goals for life and health cannot be down-yed, because the approach to care in this EBM model istient centered. By considering how patients think about

available options and their relative benefits, harms,sts, and inconveniences when determining optionsough evidence and clinical expertise, we engage inred decision making. With this approach, we can makeompromise between these 3 factors to determine the

st approach to a given patient in a given context.

ting System

Definition

nlikely to change our confidence in the estimate of effect.tudies with consistent resultsge, high-quality multicenter trialto have an important impact on our confidence in the estimatenge the estimate.yome limitationskely to have an important impact on our confidence in theis likely to change the estimate.

with severe limitationsvery uncertain.

idencewith very severe limitationsac

(wsy

opdesh

In

gowiTh

There have been various resources developed toheavtisical guidelines based on best evidence have beendeveloped in a variety of specialties, providing a goodstarting place for commonly encountered scenarios.

Ev

fifth step: self-evaluation. After working through theEBM steps on a particular clinical question, it isimportant to review each of the 5 steps and evaluatewhanincoonMe

imwhing

tant techniques we can use to better ourselves asclipropraself-evaluation, are a part of this lifelong learning,while continually aiming to increase our knowledgebase of the best evidence. What good is this evidence,

specific circumstances, and circumstances where evi-dence is not available would present quite a challenge.

regprafocevcinorchpaglobesetretalsclieacapeffilooevles

ION

Ev

Tion

prointen

S10 J. KARLSSON ET AL.ether it was completed in its entirety, effectivelyd efficiently. By continuously self-evaluating, gapsa clinicians EBM skill set can be identified. A

mplete and helpful list of important questions to askeself in evaluation can be found in Evidence-Baseddicine: How to Practice and Teach EBM.17

Are my questions specific and answerable? Am Iable to form questions throughout the day and savetime to target them later? Is the importance ofasking good questions coming across in my teach-ing? Am I modeling this?Do I know the best resources for my questions? DoI have appropriate access to these resources? Am Isearching from a wide variety of resources?Am I critically appraising the evidence I have found?Am I accurately and efficiently applying measuresintroduced here (likelihood ratio [LR], number neededto treat [NNT], relative risk reduction [RRR])?Can I work through particular concerns about man-agement and integration to relate this evidence to aparticular patient? Can I accurately and efficientlyadjust my findings to fit my unique patient?As mentioned earlier in this chapter, one of theportant concepts that has fostered an environmentere EBM can blossom is the idea of lifelong learn-. Alongside self-evaluation, one of the most impor-

SECT

Levels of

raditionally, surgical indications and treatmentdecisions have been based largely on expert opin-

and personal experience. Although EBM has beenCONCLUSIONSThis section has just but scraped the surface withard to the impact of this paradigm shift in medicalctice. This new approach to clinical decision makingused around the sound application of best research

idence is becoming so common in all fields of medi-e that you would be hard pressed to find a physiciansurgeon not familiar with RCTs, meta-analyses, Co-rane reviews, or evidence-based guidelines. As ortho-edics moves forward with the momentum of thisbal EBM movement, evidence-based orthopaedics is

coming a term, concept, and way of life in the clinicalting for all in the field. As discussed, it is not only therieval and appraisal of evidence that are important, buto how this evidence can be applied to a specificnical situation considering societal values, as well ash patients individual perspective. By learning how to

proach searching for evidence in an effective andcient manner and by learning where to look, how tok, and what you are looking for, the task of using

idence in everyday clinical practice becomes less ands daunting.

Lauren E. Roberts, M.Sc.Jn Karlsson, M.D., Ph.D.

Mohit Bhandari, M.D., Ph.D., F.R.C.S.C.

3

idence

claimed as one of the greatest achievements inernal medicine over the past 150 years,21 its influ-ce has been slow to seep into the surgical literaturealuationAn important component of practicing EBM is the

however, without professional wisdom? Without pro-fessional wisdom obtained through ongoing profes-sional development, evidence cannot be adapted tolp busy clinicians to identify and integrate the bestailable research evidence with their clinical exper-e and the patient perspective discussed above. Clin-nicians is to encourage and engage in continuingfessional development. Developments in how wectice EBM, identified and updated through ongoing

because of the unique challenges of surgical trials. In20Jocreindseaaneatre

mianofgivMinhopotiovaRestawhitoMmebepreanme

W

assderesacoheof

biocoligonstuallou

man examination. In this way, the primary researchquDatitrathethe

levtioanwifacrestanevisclures

St

tascapesio

cuthetesthecaAC(e.oustuca

uacofroevplerecpawocatie

sigpre

S11RESEARCH METHODS HANDBOOK03 levels of evidence were first introduced into Theurnal of Bone and Joint Surgery, reflecting in-ased awareness of the importance of quality in anividual study.22 This recognition of high-level re-rch propelled the orthopaedic community to design

d accomplish better studies,23,24 which in other ar-s of medicine have ultimately led to significantatment advances.25Levels of evidence are important not only in deter-ning whether a study is of higher quality thanother, but they give the reader an immediate sensehow much weight the results of the study should been.26,27 The Oxford Centre for Evidence-Based

edicine has created a detailed hierarchy of evidence,which the highest level remains a meta-analysis ofmogeneous, high-quality RCTs.28 A significant pro-rtion of current orthopaedic studies are observa-nal studies. To ensure standards of reporting obser-tional studies, the STROBE (Strengthening theporting of Observational Studies in Epidemiology)tement was created, which assists investigatorsen reporting observational studies and supports ed-rs and reviewers when evaluating these studies.29ore recently, Grades of Recommendation Assess-nt, Development, and Evaluation (GRADEs) have

en introduced to allow for a transparent and com-hensive method to grade the quality of evidence

d strength of recommendations about the manage-nt of patients.30

HOW LEVELS OF EVIDENCE AREASSIGNED

hat Is the Primary Research Question?For a level of evidence to be assigned, one must firstess the primary research question. The level of evi-

nce is assigned specifically to whether the primaryearch question, well-defined in the purpose section ofmanuscript, was aptly addressed in the results andnclusion sections. Thus, asking a focused questionlps yield a more answerable question, and assignationlevel of evidence is relatively straightforward.For example, in a study comparing the use of aabsorbable interference screw versus washer-post

nstruct for tibial fixation in an anterior cruciateament (ACL) reconstruction, it would be ideal toly manipulate a single variable. In other words thedy includes the same surgeon, same technique, andpatients with the same isolated ACL injury. The

tcome would be a single data point, such as Lach-estion is focused on answering 1 specific question:oes tibial fixation of the graft affect the postoper-ve Lachman examination? If a difference in tibialnslation is found between the 2 types of fixation,n a conclusion can be made as to whether or notre was a difference.

Conversely, it becomes very difficult to assign ael of evidence when the primary research ques-n is not well-defined or the conclusions do notswer the research question. Frequently, studiesll make conclusions based on their results, but int the conclusions were not related to the primaryearch question. Therefore it is extremely impor-t when designing or reviewing a study to first

aluate whether the research question for the studywell defined and then evaluate whether the con-sions of the study are related to that primaryearch question.

udy Designs

Once the primary question is determined, the nextk is to identify the study type. Levels of evidencen be divided into 4 different study designs: thera-utic, prognostic, diagnostic, and economic or deci-n analyses.31Therapeutic Study Type: Therapeutic studies fo-s on assessing the effect of a specific treatment on

outcome of a specific disease process. A practicalt to determine whether a study design is consideredrapeutic is if the factor being studied can be allo-

ted in a random fashion. For example, a study ofL reconstruction evaluating the effect of graft type

g., bone-tendon-bone v hamstring autograft) on thetcome of reconstruction would be a therapeuticdy because the graft type can be randomly allo-

ted.Prognostic Study Type: Prognostic studies eval-te the effect of patient characteristics on the out-me of a disease process. Prognostic studies differm therapeutic studies because the factors beingaluated cannot be randomly allocated. For exam-, a study of the effect of age on outcome of ACLonstruction in 2 different study groups (e.g.,

tients aged 30 years v patients aged 30 years)uld be considered a prognostic study because age

nnot be randomly allocated to 2 groups of pa-nts in the study.Diagnostic Study Type: Diagnostic studies are de-ned to assess whether a specific test is related to thesence or absence of a particular pathology. For

example, in patients with femoroacetabular impinge-mepeefftioamwome

sigmeapinop

petheplemeACsisicacainhpa

avan

stuapcisex(Ldletom

anevtemdoI etiogreimquallde

Level I

Le

Le

S12 J. KARLSSON ET AL.nt of the hip, the anterior impingement test can berformed for assessment. A study examining theect of the anterior impingement test and its rela-nship to femoroacetabular impingement is an ex-ple of a diagnostic study design. Another exampleuld be joint-line tenderness and its ability to detectniscus tear.

Economic Analyses: Economic analyses are de-ned to assess the cost-effectiveness of a certain treat-nt for a certain pathology. For example, in the case of

group of young patients with femoroacetabular im-gement, one might compare the cost-effectiveness of

en versus arthroscopic impingement surgery.Decision Analyses: Decision analysis studies arerformed to evaluate the outcome of a certainrapy to determine the ideal treatment. For exam-, in evaluating surgical versus nonsurgical treat-nt for patients aged greater than 40 years withL deficiency, an expected-value decision analy-

, which is a systematic tool for quantitating clin-l decisions, can be used to conclude ACL surgi-

l reconstruction as a preferred treatment.32 Anerent limitation of this study type is that actual

tients are not evaluated.

LEVELS OF EVIDENCE IN DETAIL

Several systems for rating levels of evidence areailable.33 The one chosen by The Journal of Boned Joint Surgery and Arthroscopy has 5 levels ofdy design for each of 4 different study types: ther-eutic, prognostic, diagnostic, and economic or de-ion modeling.21,22,25 Among study designs, thereists a hierarchy of evidence, with RCTs at the topevel I), controlled observational studies in the mid-, and uncontrolled studies and opinion at the bot-

(Level V).33Understanding the association between study designd level of evidence is important. Higher levels ofidence should be more convincing to surgeons at-

pting to resolve clinical dilemmas.21 Because ran-mized clinical trials are not always possible, Levelvidence may not be available for all clinical situa-ns. Therefore Level III or IV evidence can still be ofat value to the practicing orthopaedic surgeon. It isportant to consider that an answer to a clinicalestion must be based on a composite assessment of

available evidence. No single study provides afinitive answer.Therapeutic studies1. RCTs with (a) significant difference or (b) no sig-

nificant difference but narrow confidence intervals2. Systematic reviews of Level I RCTs (studies

were homogeneous)Prognostic studies

1. Prospective studies2. Systematic review of Level I studies

Diagnostic studies

1. Testing of previously developed diagnostic cri-teria in series of consecutive patients (with uni-versally applied reference gold standard)

2. Systematic review of Level I studies

Economic and decision analyses studies

1. Clinically sensible costs and alternatives; valuesobtained from many studies; multiway sensitiv-ity analyses

2. Systematic review of Level I studies

vel IITherapeutic studies1. Prospective cohort study2. Lesser-quality RCT (e.g., 80% follow-up, no

blinding, or improper randomization)3. Systematic review of Level II studies or Level I

studies with inconsistent resultsPrognostic studies1. Retrospective study2. Untreated controls from an RCT3. Systematic review of Level II studiesDiagnostic studies1. Development of diagnostic criteria on basis of

consecutive patients (with universally appliedreference gold standard)

2. Systematic review of Level I and II studiesEconomic and decision analyses studies1. Clinically sensible costs and alternatives; values

obtained from limited studies; multiway sensi-tivity analyses

2. Systematic review of Level II studies

vel IIITherapeutic studies1. Case-control study

2. Retrospective cohort study

Le

Le

E

Le

thetieenintengropsqupaofto

domized group. Preoperative and postoperative clini-capagroproissta

Le

thaaggodethaclipaanauwagrcyobdo

Le

repsigwhtwsptivinfcuityBathrallwesimtwhainpfernathe

Le

tiv

S13RESEARCH METHODS HANDBOOK3. Systematic review of Level III studiesDiagnostic studies1. Study of nonconsecutive patients (without con-

sistently applied reference gold standard)2. Systematic review of Level III studiesEconomic and decision analyses studies1. Analyses based on limited alternatives and

costs; poor estimates2. Systematic review of Level III studies

vel IVTherapeutic studies

Case series (no, or historical, control group)Prognostic studies

Case seriesDiagnostic studies

1. Case-control study2. Poor reference standard

Economic and decision analyses studiesNo sensitivity analyses

vel V

Therapeutic studiesExpert opinion

Prognostic studiesExpert opinion

Diagnostic studiesExpert opinion

Economic and decision analyses studiesExpert opinion

XAMPLES OF STUDIES OF DIFFERENTLEVELS OF EVIDENCE

vel I

In a study of a consecutive series of patients withdiagnosis of internal snapping hip syndrome, pa-

nts were randomized into 2 different methods ofdoscopic release of the iliopsoas tendon.34 Patientsgroup 1 were treated with endoscopic iliopsoasdon release at the lesser trochanter, and patients inup 2 were treated with endoscopic trans-scapular

oas release from the peripheral compartment. Aality randomization process included randomizingtients at the last possible time point, e.g., at the timesurgery. An a priori power analysis was performedensure adequate numbers of patients in each ran-l and imaging assessments were evaluated for alltients. No statistical difference was found betweenups. Therefore this RCT, with a defined and ap-priate sample size and narrow confidence intervals,characterized as a Level I study, even though notistically significant difference was determined.34

vel II

In a prospective cohort study, patients aged oldern 40 years were compared with a group of patients

ed younger than 40 years who underwent autolo-us chondrocyte implantation for isolated cartilagefects of the knee.35 The authors hypothesis wast the older group of patients would have inferiornical results compared with the younger group oftients. All patients were followed up for 2 years,d validated clinical outcomes were used. Thethors hypothesis was disproved, because theres no statistically significant difference in the 2

oups of patients treated with autologous chondro-te implantation. This prospective study does nottain a Level I designation because it is nonran-mized.

vel III

The efficacy of open versus arthroscopic Bankartair remains controversial; therefore the authors de-ned a retrospective case-control study to determineether there is a significant difference in cost be-een the 2 surgical procedures. In a Level III retro-ective case-control study, the authors retrospec-ely reviewed the medical records and billingormation of consecutive patients treated for re-rrent, post-traumatic anterior shoulder instabil-.36 They compared 22 patients who had opennkart repair with 20 patients who had ar-oscopic Bankart repair. Total operating times andcharges were obtained from records. Patients

re also clinically evaluated. This study foundilar shoulder scores and rates of dislocation be-

een the 2 groups. The arthroscopic Bankart repaird a lower cost, but if an obligatory overnightatient stay was taken into account, the cost dif-ence was negligible. Because of its retrospectiveture, this study was characterized as a Level III,rapeutic cohort study.36

vel IV

The purpose of a Level IV study is to retrospec-ely review the outcome of a group of patients

treated in a similar way. In a Level IV therapeutic caseserrectieantivdabatecturtecspca

Le

theduofInulawhwhevorion

moapcaoubecotheThduthitheassovtiotre(grcisinsrecsujud

Grade A indicates good evidence (Level I studieswiintorrecqutentiocofor

pastaimnoaidtic

prodeiciouitsuqufinIIIareI sstudameinstu

Lethestutiopraingmatha

S14 J. KARLSSON ET AL.ies study, the authors described transphyseal ACLonstruction with hamstrings performed in 26 pa-nts with open tibial and femoral physes.37 Clinicald radiologic outcomes were evaluated retrospec-ely. Their outcomes were well defined, with vali-ted knee scores and detection of any growth distur-nce on scanograms. They concluded that theirhnique yielded good outcomes and no growth dis-bances. Because the authors did not compare theirhnique with another technique, and given its retro-

ective nature, this represents a Level IV therapeuticse series.

vel V

In a Level V study, the authors showed the use of70 arthroscope for several arthroscopic proce-

res and in a number of circumstances in which itfers superior visualization to a 30 arthroscope.38this study the authors demonstrated their partic-r expertise with this arthroscopic instrument,ich may be interesting for arthroscopic surgeonso are not familiar with the 70 arthroscope. How-

er, because this study does not report any resultsclinical outcomes, it is considered expert opin-.

GRADES OF RECOMMENDATION

As surgeons, we often find multiple studies to bere convincing than a single article. Although the

propriate literature on a particular clinical questionn be identified in many ways, to search the literaturerselves is time-consuming and the search may notcomprehensive. Although review articles are often

mprehensive in the available evidence they include,conclusions that they contain can be uncertain.

erefore grades of recommendation have been intro-ced in the development of practice guidelines. Ins process a reviewer or organization can gather all

appropriate literature, appraise the literature byigning a level of evidence, and summarize theerall quality by allocating a grade of recommenda-n.26,30 This helps the reader by giving definitiveatment recommendations that should definitelyade A) or probably (grade B) guide treatment de-ions for their patients. In addition, a grade of I, orufficient or conflicting evidence not allowing aommendation for or against intervention, advises a

rgeon to treat patients based on his or her bestgment or on a case-by-case basis.th consistent findings) for or against recommendingervention. Grade B indicates fair evidence (Level IIIII studies with consistent findings) for or againstommending intervention. Grade C indicates poor-ality evidence (Level IV or V studies with consis-t findings) for or against recommending interven-n. Grade I indicates that there is insufficient ornflicting evidence not allowing a recommendation

or against intervention.

CONCLUSIONS

The purpose of this chapter is to provide the ortho-edic sports medicine surgeon with a better under-nding of the levels of evidence and their clinicalplications. Such understanding is extremely helpful,t only from a research design standpoint but also to

readers in understanding the importance of a par-ular studys conclusions.From a design standpoint, in order for a researchtocol to maximize the best possible level of evi-

nce, it is important for the orthopaedic sports med-ne researcher to consider levels of evidence whentlining the primary research question. Furthermore,is important to recognize that performing a Level Irgical study is extremely difficult, because it re-ires a significant amount of preparation, time, andancial investment to allocate resources. Level II,, and IV studies have their own worth and merit andespecially useful in the circumstances where Level

tudies would not be feasible. When observationaldies are being performed, the STROBE recommen-tions will assist the investigator in maintainingthodologic transparency and also assist the readercomprehensively evaluating the quality of the

dy.From a readers standpoint, if a study is assignedvel I evidence, and a grade A recommendation, then

reader can feel confident that the results of thedy have the highest level of validity. In this situa-n the reader/surgeon may choose to change clinicalctice based on those recommendations, thus shap-and directing future sports medicine care. Ulti-

tely, it is this endpoint, the best care for patients,t is our highest goal.

Aaron J. Krych, M.D.Bruce A. Levy, M.D.

Mario Ferretti, M.D., Ph.D.

SECTION 4

d TC

Tdeareanpeidewe

theEfpaporanplawoabinvcoticpra

thicaind(ouallentiofordifupintgrocharacteristics (e.g., inclusion and exclusion criteria)and unknown characteristics (those present bychance). Provided that an appropriate sample size iscachtwide

effime

beterranblilowsioresevthe

opsursomu

cotioim

Th

cliqutiecliun

circosavtreother words, the treatment effect is so large, and theconsequences so grave, that it is not necessary tocompare the surgical procedure with existing treat-

S15RESEARCH METHODS HANDBOOKlculated and recruitment is achieved, the unknownaracteristics are likely to be equally distributed be-een groups. Therefore, if a difference in outcome isntified, the findings can be directly attributed to the

ment. The same can be said for parachute use com-pared with placebo.41 All problems are not amenableto an RCT.42 Until the rules of surgical engagementchange to be similar to those related to medical ther-Study Designs: RandomizeCONSORT

he randomized clinical/controlled trial (RCT)represents the highest level of evidence or study

sign. In orthopaedic surgery and sport medicine, therea multitude of questions requiring evidence-based

swers. It will become increasingly more important torform RCTs to address these questions. This chapterntifies the problems encountered by the surgeon, asll as the strategies and how to address these concerns.

WHY PERFORM RCTS?

An RCT is the most valid study design to evaluateefficacy or effectiveness of surgical treatments.

ficacy refers to the ideal situation with optimaltient selection, well-controlled surgical technique,stoperative compliance, and so on. This type ofdomized trial is sometimes referred to as an ex-natory trial.39 Effectiveness refers to a more real-rld situation, where the patients have more vari-

ility in their disease state, multiple surgeons may beolved, and the postsurgical course is less well-

ntrolled and more typical of most surgeons prac-es. This type of RCT is sometimes referred to as agmatic trial.30

The RCT is prospective by definition, and therefores term is redundant. All RCTs are prospective be-use the primary research question being addressed,ependent variable (treatment), dependent variabletcome), and inclusion and exclusion criteria shouldbe determined a priori. Patient recruitment and

rollment, consenting, randomization, data collec-n, and analysis are subsequently performed in award direction. Patients are randomly allocated toferent treatment groups and are typically followedin an identical manner with the main outcome of

erest measured at a specific period of time. Theups of patients are similar with respect to knownrials, Level I Evidence,hecklist

cacy or effectiveness of the specific surgical treat-nt.

Therefore the RCT is less likely to introduce biascause treatment group assignment is randomly de-mined. Other biases can occur, however, despite thedomized design. These would include a lack ofnding of the patients or assessor, different fol--up times, loss to follow-up, differential exclu-

ns, expertise-based bias, and early reporting ofults before the full sample size is achieved.40 What-er bias is introduced must first be recognized, andn it may be accounted for.

So, if we truly want to determine the benefits of oneerative technique over another, surgical versus non-gical treatment, different rehabilitation protocols, andon, the RCT is the best possible study design. West be cognizant of the fact that the design is only one

mponent of conducting valuable research. Randomiza-n does not compensate for poor adherence to all otherportant methodologic issues.

REASONS FOR NOT PERFORMINGAN RCT

e Problem

Randomized clinical trials are only necessary if thenical problem/surgical treatment is common; if theestion is a significant issue to clinicians and pa-nts; and most importantly, if the answer to thenical question is clearly not known. It would benecessary to perform a randomized clinical trial incumstances where observational studies are sompelling and/or outcomes are dramatic and life-ing (e.g., amputation compared with antibiotic

atment for clostridial gas gangrene of the foot). In

apy it is unlikely that surgical RCTs will become theno

Th

whfripheagaspdre

wooufortal

theofbelattalSo

affsurintreonThpa

Th

triaregicfrocomiforto

nomeHisivlen

e

sent and with little or no regulation, but if we want topeob

Th

regthaopchnififacstathesisvoatthethamathebethrThsamstrrieofbeRCthece

Th

pefinincWthrgodoproI awaexresass

reqeff

S16 J. KARLSSON ET AL.rm rather than in the minority.43

e Patient

Patients present with preconceived notions aboutat treatment is best. Whether they have talked to a

end or family member, someone in health care, aysician, or even another surgeon, there is a bias thatch patient may have. Patients will typically investi-te their problem on the Internet. They will identifyecific operations and other surgeons who have ad-ssed their problem with a particular procedure.

There are different cultural expectations around therld. At the same time that an RCT comparingtpatient with inpatient ACL surgery was being per-med in Canada, patients were staying in the hospi-for 2 weeks or more in Europe and Japan.44

Patients may simply want to know what proceduresy will be undergoing, and any doubt leads to a lackconfidence. Some patients will consent to a trial

cause it is the only chance that they can undergo theest procedure (i.e., being included in the experimen-group rather than undergoing the usual technique).me patients feel the exact opposite sentiment.There is a permanency regarding surgery. This canect a patients decision, such as in a trial comparinggical with nonsurgical treatment. If a patient ends upthe nonsurgical arm, there is a perception that if theatment fails, then surgery may be an option. However,ce a procedure is performed, there is no going back.ese patient-related concerns influence whether eligibletients are open to recruitment into a trial.

e Performer (Surgeon)The surgeon may be the greatest barrier to surgicalals! Compared with medical therapies where there

strict regulations on how a drug is released, sur-al innovation can occur with little or no restraintm regulating agencies, hospitals, or local ethicsmmittees. This is definitely the case when there is anor variation in technique or implant used. There-e there is no incentive whatsoever to perform a trialdetermine the efficacy of a particular procedure.Surgeons are innovators. Arthroscopic surgery mayt have become the gold standard if we had the require-nt of randomized clinical trials to show its benefit.storically, arthroscopy of the knee was more expen-e, took longer to perform compared with the equiva-t open procedure, and was fraught with complications.The irony of being a surgeon is that we can performxperimental surgery on our patients with their con-rform an experiment (i.e., an RCT), then we musttain both scientific and ethical approval.45

e Procedure

There are many barriers to performing an RCTarding the surgical procedure. It is well-recognizedt there is a learning curve with respect to anyeration. One could argue that, if it is a minimalange, then the learning curve is shallow but a sig-cant departure from what is usually done may int have a very steep curve. A certain minimumndardization is required in any trial, particularly ifre is more than 1 surgeon involved to ensure con-tency between surgeons. If only 1 surgeon is in-lved, then it may be the case that he or she is better1 procedure compared with another. We can takeanalogy from the sport of tennis. It is well known

t only a few players in history have been able toster the game on hard courts, clay, and grass all insame year. Why would we expect surgeons toable to perform different procedures (e.g., ar-

oscopic compared with open) equally as well?46erefore, if more surgeons are required (to increaseple size), then strategies such as training manuals,

atification by surgeon, and matching surgeon expe-nce are techniques that can alleviate the variabilitythe procedure. One recently applied method has

en called the expertise-based design.40 In thisT the patient is randomized to the procedure, andsurgeon with the expertise for that particular pro-

dure carries out the surgery.40,47

e Process

This is probably what scares most surgeons away fromrforming a randomized trial: the process from start toish is overwhelming to most surgeons. Statementslude the following: It is going to take too much time!e do not have the resources! We will never get itough ethics! We do not have enough patients! I getod results with the technique I am familiar with! Wenot need to do a trial; I can do a bunch of these newcedures and compare to what I have done in the past!m too busy taking care of patients to do research! Int to do the latest techniques; it is what my patientspect! It is not my responsibility to do research; let theearchers figure this out! I do not have a researchistant! It takes too much money!There is no doubt that conducting a randomized trialuires significant infrastructure support, time, andort. The process is daunting at first and difficult to

implement but eventually routine. Until there arestrsuap

HA

thidifwhhetumideiningica

Cl

ovThmumemoprohaiswhRCeva mtoisan

cepethequtheranthrpathiimmawhdoinfl

trum. They may desire to understand in great depth thetrebiganthe

aticopracewitribemoifsh

ofsibcorectocosuNoideevingrecatecThHoofACamanfultiscatora cgepabapaicaofunamboran

S17RESEARCH METHODS HANDBOOKicter rules regarding the use of new procedures,rgeons will not be compelled to be involved inpropriate trials.45

OW DO WE SOLVE THESE PROBLEMSND WHAT ARE THE PREREQUISITES?

It is much easier to identify reasons not to do some-ng. It is easier for a medical student to answer aficult question with a negative response. For example,en asked about the differential diagnosis of an acute

marthrosis in the knee, he or she may say, It is not aor. Although this statement is correct and easy to

ntify on the differential, it is not a very useful answerevaluating the patient. The solution lies in the follow-concepts, the 3 Cs of successful trial research: clin-

l equipoise, commitment, and collaboration.

inical Equipoise

Clinical equipoise is defined as genuine uncertaintyer whether one treatment or another is beneficial.is equipoise should involve the expert clinical com-nity where there is honest professional disagree-nt among . . . clinicians.48 Therefore, if one orre treatment options are available for a particularblem and the best option is not known, then we

ve reason to consider a randomized clinical trial. Itnecessary to build a case for clinical equipoise,ich essentially is the essence of the rationale for theT. A surgeon should first review the available

idence in a systematic way, analyze the results (witheta-analysis if possible), and determine an answer

his or her question. If this answer is clear, then thereno equipoise and, therefore, no need to performother trial.However, clinical equipoise relates to not only un-rtainty regarding treatment options but the ethics ofrforming a trial from all perspectives: the patients,

surgeons, and societys. It is necessary to ask theestion, Who is uncertain, the individual surgeon or

community of surgeons? A surgeon may considerdomizing his or her patients to a particular ar-oscopic fixation technique such as absorbable com-red with nonabsorbable suture anchors. Whereass trial may be easy to perform because there is littlepact on the patients decision making, it may nottter to the community of surgeons or society as aole. Therefore, is it really worth performing a ran-mized clinical trial? Patients just like surgeons areuenced by their position on the equipoise spec-atment options, they may want to appreciate theger-picture perspective of helping medical science

d therefore the surgical community perspective, ory may in fact simply trust the surgeon.

Clinical equipoise requires not only the consider-on of the individual surgeons perspective but themmunity of surgeons that establishes standards ofctice. Most surgeons have difficulty with this con-

pt, and therefore failure of consensus of evidencethin the clinical community is the usual driver for aal. Ultimately, the ethical surgeon must do what isst for his or her individual patient. Uncertainty is aral prerequisite for being involved in an RCT, butwe know the best and correct treatment, then weould perform it.45,48,49The following example should illustrate the conceptclinical equipoise and the moral or ethical respon-ility of the surgeon. A randomized clinical trial wasnducted to compare 3 surgical techniques for ACLonstruction.50 A meta-analysis had been conducteddetermine whether an autograft patellar tendon

mpared with autograft hamstring reconstruction re-lted in improved outcomes for patients at 2 years.t only was the meta-analysis inconclusive but itntified many concerns with respect to the available

idence.51 At the same time, surgeons were advocat-the so-called double-bundle technique for ACL

onstruction. Therefore it seemed logical to conductrandomized clinical trial comparing the existinghniques with the newer double-bundle procedure.is would represent established clinical equipoise.wever, there was a clinically identifiable subgrouppatients who have the prerequisite diagnosis of anL-deficient knee but whose knees on careful ex-ination had minimal documentable translational

d rotational instability. These patients, also on care-arthroscopic examination, had identifiable ACL

sue that was both biologically viable and mechani-lly supportive. Whether this represents a partiallyn ACL, a single-bundle ACL failure, or healing ofomplete ACL tear is debatable. However, the sur-

on believed on moral and ethical grounds that thesetients should be excluded from the trial. This wassed on the principle of biological preservation of thetients own tissue and, most importantly, the empir-l evidence from his own practice that this subgrouppatients had a better outcome than those who haddergone reconstruction in the usual way. This ex-ple demonstrates the difficulty with addressingth clinical equipoise and the ethics of performing adomized clinical trial.

Commitment

durelrolqutictouachforimThistenwipaenqusuaretre

thecepamipropromaDuwamapro

a wexfactakrun

areoftimcliin

Co

isto

clinical practice to conduct a meaningful randomizedcli

hecotopeindbiotrifinaproingitie

clisamterhatheesstriarectriaceninc

C

imtiome

thedealoappro

(hotiemaideatoevdutivcli

S18 J. KARLSSON ET AL.Probably the most important prerequisite for con-cting an RCT is ones commitment. Commitmentates to not only being involved with respect to onese but more importantly being committed to theestion rather than finding the answer. This is par-ularly difficult for surgeons because we are drivensolve patients problems through our own individ-l surgical skills.30,42,43,45,52-56 Surgeons are typicallyaracterized as innovative rather than reflective, per-mers rather than observers, and are interested inmediate gratification rather than long-term rewards.e RCT requires a different type of commitment thatreflective before and after the fact, requires persis-ce, and may lead to an answer that is not consistentth what is expected. For example, in a trial com-ring surgical with nonsurgical treatment, it is inher-tly difficult for a surgeon to be committed to theestion unless there is a perceived problem with thergical option. Our training, experience, and rewards

derived from the outcomes of the patients surgicalatment.As innovators, surgeons become readily aware of

latest technique or improvements on previous pro-dures. It might take 2 to 5 years to recruit enoughtients into a trial and then the requirement of animum 2-year follow-up. The process of writing theposal, obtaining funding, and obtaining ethical ap-val may take at least 1 year. A meaningful trialy take anywhere from 5 to 10 years to complete.ring this time, the surgical world has moved for-rd, techniques have been modified, case reportsy have suggested complications with a particularcedure, and so on.57

The surgeon must therefore act and be committed inay that is somewhat foreign to his or her normal

istence. This commitment is compounded by thet that in every respect conducting a surgical triales more time and effort than what is necessary toa clinical practice. Successful surgical trialistsjust as passionate about the research as any aspect

their clinical practice. They likely spent additionale learning how to perform research in the fields of

nical epidemiology, methodology, or public healthaddition to their clinical fellowships.

llaboration

We are not aware of any surgeon in the world whoa methodologist and biostatistician, has extra timedevote to clinical research, and also a large enoughnical trial without help.The collaborative infrastructure support is not onlylpful but necessary. One solution is to pay forllaborative support by hiring a research organizationconduct the trial and therefore enter patients and

rform the surgery. Another approach is to identifyividuals with the expertise in methodology andstatistics who will be partners/co-authors in the

al and therefore will provide their expertise withoutancial compensation. There will always need to beresearch coordinator or assistant. This individualvides day-to-day support and keeps the trial mov-forward, addressing all of the details and complex-s of conducting an RCT.

Collaboration may take the form of including othernicians and surgeons. These individuals may be at the

e institution or could be based out of multiple cen-s. In these circumstances the clinicians will need tove the same requisite ethical and clinical equipoise to

primary surgeon and the time and commitment nec-ary for success. It is well-recognized in multicenterls that the host site is usually more successful inruiting patients and conducting all aspects of thel.58,59 The exception to this is when the trial is fundedtrally and the collaborating centers have financialentives to recruit and follow up the patients.

ONDUCTING A RANDOMIZED CLINICALTRIAL?

Once the prerequisites have been addressed (i.e., anportant clinical concern, commitment to the ques-n, and collaboration), the trial is ready to be imple-nted.

However, implicit within these prerequisites is thatresearch question has been carefully refined and a

tailed proposal drafted, reviewed, and rewritten,ng with an application for funding and ethical

proval.58,60 The implementation starts once ap-val and funding have been achieved.

It is necessary to engage all people and settingsspital wards [i.e., emergency, inpatient, and outpa-

nt], clinics, operating rooms, and so on) that may ory not be impacted by the trial. This process shouldally occur during the proposal stage but is an oblig-ry part of implementation. Informing and engaging

eryone must occur before, during, and after con-cting the trial if it is to be successful. Simple incen-es such as providing refreshments to the hospital ornic staff or giving gift vouchers to referring physi-

cians have proven to be cost-effective ways to facili-tat

verouan

hopamicriingtimwicahasuahig

anresinv2 ssu

iswhbeproRCindofizaan

spincgedolopniqidebaizape

copa(i.op

the primary outcome, the measured treatment effect(i.tremepeknfrothisperetrelespesmvacodequouare

recsizpatiestringus(asincenavansp

caneanplefamsm

shfole-m

bea btri

seq

S19RESEARCH METHODS HANDBOOKe this critical engagement.Informing the medical community of the trial is alsory important at the start. This includes presenting atnds and business meetings, advertising the trial,

d registering the trial in an international database.Within the written proposal are specific criteria onw the patient population is to be sampled. Whentients are seen, they need to be screened to deter-ne whether they fit the inclusion and exclusionteria. Assuming the patient is eligible, the consent-process can proceed. Informed consent is a criticale-consuming activity. There may be ethical issues

th respect to who obtains consent, and this is typi-lly regulated through each individual institution. Its been our experience with surgical trials that argeon is the person best suited to obtain consent forsurgical randomized clinical trial. This leads toher recruitment of patients.

The process of randomization can take many forms,d there are different types of randomization. Withpect to surgical trials with more than 1 surgeonolved, stratification by surgeon is necessary unlessurgeons are matched for all known characteristics

ch as experience, location, and so on.One technique to help the process of randomizationcalled expertise-based randomization.40 This isere the patient is randomized to the procedure

fore going to the operating room. This techniquevides the surgeon the ability to participate in anT but still retain his or her independence andividual preference to perform his or her procedurechoice. We have used this expertise-based random-tion technique successfully when comparing opend arthroscopic procedures in the shoulder.46Irrespective of the type of randomization, there areecific requirements that must be adhered to. Theselude allocation concealment and adequate sequence

neration, i.e., typically, computer-generated ran-m-number sequencing.39 Although opaque enve-es are considered an appropriate concealment tech-ue, they can be tampered with and the sequencentified. Current standards would involve a Web-

sed remotely accessed computer-generated random-tion process that is performed by someone inde-ndent of the surgeon or primary investigator.39The randomized trial should include a primary out-me (i.e., the dependent variable), such as a validatedtient-reported outcome, and the defined interventione., the independent variable), such as the standarderation compared with the new surgical procedure.The sample size for the RCT is directly related toe., the expected clinical difference between the 2atment groups), and the variability of the outcomeasured in the standard deviation. Ideally, the ex-

cted difference and variability of the outcome areown values based on previous pilot data or datam similar populations of patients.39,53,54 Withouts information, the sample size calculation becomeseculative, and therefore the trial may be underpow-d to show a meaningful clinical difference betweenatment groups. In general, the more precise (i.e.,s variability) the outcome and the greater the ex-cted differences between treatment groups, thealler the sample size. In addition, those dependentriables that are measured on a scale that allows forrrect statistical analysis with means and standardviations (i.e., parametric statistics) are likely to re-ire a smaller sample size. Trials where the primarytcome is a probability (i.e., nonparametric statistics)

more likely to require a greater sample size.The greatest barrier to conducting a surgical trial isruitment and therefore meeting the a priori samplee.59 Surgeons typically overestimate the number oftients who would be eligible, and the eligible pa-nts do not always consent to the trial.57 Some of theategies to improve recruitment include collaborat-

with more surgeons, involving multiple centers,ing baseline data to recalculate the sample sizesuming that there is less variability), providingentives to include patients, continual strategies to

gage people, ensuring regular patient contact tooid loss to follow-up, and modification of inclusiond exclusion criteria to be more inclusive with re-ect to eligibility.Once all of the details of the trial are organized,rrying out the trial is arguably the easiest part. Itcessitates the help of the coordinator and assistants,d it requires a time commitment; however, as peo-

in the clinics, wards, and operating rooms becomeiliar with the process, the trial should move ahead

oothly.Every strategy to maintain contact with the patientsould be used. This may include regularly scheduledlow-up visits, phone communication, and use ofail or social media.

Once the data are collected and the patients haveen followed up, the analysis will occur. The help ofiostatistician is usually necessary for randomized

als.The results will be interpreted, presented, and sub-uently published.

staofforresthadesolstasiochgugrathephexcrian

SORT

S20 J. KARLSSON ET AL.CHECKLIST

The randomized clinical trial represented the goldndard for evaluating interventions, but the accuracysuch trials reporting was not consistent and there-e bias could be introduced. A worldwide group ofearchers, methodologists, and clinicians, concernedt the reporting of trials lacked lucid and complete

scriptions of the critical information, created the Con-idated Standards of Reporting Trials (CONSORT)tement (1996).61,62 This has undergone recent revi-n, in 2010.63,64 The CONSORT 2010 statement,ecklist, and flow diagram provide authors withidance on how to report their trials. The flow dia-m (Fig 3) illustrates the progress of the trial fromstart and includes the following: (1) the enrollment

ase with numbers of eligible patients and thosecluded for reasons such as not meeting inclusionteria or declining to participate or for other reasons,d the number of patients randomized; (2) the allo-

FIGURE 3. CONtients who were allocated to the treatment groups,ether they received the allocated treatment, and ift, why not; (3) the follow-up phase, which includes

numbers lost to follow-up and the reasons why;d (4) the analysis phase, which includes those pa-nts in each group who were analyzed and any whore excluded and for what reasons.63,64The checklist (Table 6) represents a much moretailed list of characteristics of the trial that need toreported.63,64 The list includes the title and struc-ed abstract, an introduction, the methods, the re-lts, a discussion, and a section for other information.e checklist requires the authors to identify withinir manuscript the page number where the appropri-information is written. Important concepts includebackground and objectives, the trial design, the

tients, detailed descriptions of the interventions,ether the outcomes were completely defined andspecified, sample size determination, blinding of

flowchart.REPORTING RCTS: THE CONSORT cation phase, which includes the exact numbers ofpawhno

thean

tiewe

debetursu

Ththeatethepawhpre

TABLE 6. CONSORT 2010 Checklist of Information to Include When Reporting a Randomized Trial*

S21RESEARCH METHODS HANDBOOK

Section/TopicItem No Checklist item

Reported on page No

Title and abstract1a Identification as a randomised trial in the title1b Structured summary of trial design, methods, results, and conclusions (for specific guidance see CONSORT for abstracts)

IntroductionBackground and objectives

2a Scientific background and explanation of rationale2b Specific objectives or hypotheses

MethodsTrial design 3a Description of trial design (such as parallel, factorial) including allocation ratio

3b Important changes to methods after trial commencement (such as eligibility criteria), with reasonsPa