The Effect of Electronic Prescribing on Medication Errors and Adverse Drug Events: A Systematic Review

Journal of the American Medical Informatics Association Volume 15 Number 5 September / October 2008 585

http://jamia.oxfordjournals.or

Dow

nloaded from

JAMIAThe Practice of Informatics

Review Paper �

The Effect of Electronic Prescribing on Medication Errors andAdverse Drug Events: A Systematic Review

ELSKE AMMENWERTH, PHD, PETRA SCHNELL-INDERST, PHD, CHRISTOF MACHAN, MSC,UWE SIEBERT, PHD

A b s t r a c t The objective of this systematic review is to analyse the relative risk reduction on medicationerror and adverse drug events (ADE) by computerized physician order entry systems (CPOE). We includedcontrolled field studies and pretest-posttest studies, evaluating all types of CPOE systems, drugs and clinicalsettings. We present the results in evidence tables, calculate the risk ratio with 95% confidence interval andperform subgroup analyses for categorical factors, such as the level of care, patient group, type of drug, type ofsystem, functionality of the system, comparison group type, study design, and the method for detecting errors. Ofthe 25 studies that analysed the effects on the medication error rate, 23 showed a significant relative risk reductionof 13% to 99%. Six of the nine studies that analysed the effects on potential ADEs showed a significant relativerisk reduction of 35% to 98%. Four of the seven studies that analysed the effect on ADEs showed a significantrelative risk reduction of 30% to 84%. Reporting quality and study quality was often insufficient to exclude majorsources of bias. Studies on home-grown systems, studies comparing electronic prescribing to handwritingprescribing, and studies using manual chart review to detect errors seem to show a higher relative risk reductionthan other studies. Concluding, it seems that electronic prescribing can reduce the risk for medication errors andADE. However, studies differ substantially in their setting, design, quality, and results. To further improve theevidence-base of health informatics, more randomized controlled trials (RCTs) are needed, especially to cover awider range of clinical and geographic settings. In addition, reporting quality of health informatics evaluationstudies has to be substantially improved.

� J Am Med Inform Assoc. 2008;15:585– 600. DOI 10.1197/jamia.M2667. by guest on May 5, 2016

g/

IntroductionMedical errors are an important factor that influences thequality of patient care. According to Barach et al., nearly100,000 individuals per year in the US die of preventablemedical errors.1

Affiliations of the authors: UMIT–University for Health Sciences,Medical Informatics and Technology Tyrol, Institute for HealthInformation Systems (EA, CM), Tyrol, Austria; Department ofPublic Health, Medical Decision Making and HTA (US, PS-I), Tyrol,Austria; Alfried Krupp von Bohlen und Halbach Foundation, Chairfor Medical Management, University of Duisburg-Essen (PS-I),Essen, Germany; Institute for Technology Assessment and Depart-ment of Radiology, Massachusetts General Hospital, Harvard Med-ical School (US), Boston, MA; Program in Health Decision Science,Department of Health Policy and Management, Harvard School ofPublic Health (US), Boston, MA.

The authors thank all reviewers for their helpful comments onearlier versions of this paper.

Correspondence: Elske Ammenwerth, Professor, Institute forHealth Information Systems, UMIT—University for Health Sci-ences, Medical Informatics and Technology Tyrol, Eduard Wall-nöfer Zentrum 1, A-6060 Hall in Tyrol, Austria; e-mail: �[email protected]�.

Received for review: 11/19/07; accepted for publication: 06/06/08.

Medication errors have been identified as a major type ofmedical errors. The Council of Europe2 and the BritishDepartment of Health3 defined medication errors as “anypreventable event that may cause or lead to inappropriatemedication use or patient harm . . . .” The Institute of Med-icine reports that a hospital patient can expect on average tobe subjected to more than one medication error per day.4,5

Medication errors can lead to adverse drug events (ADEs)that are defined as “any response to a drug that is noxiousand unintended. . . .”3 A report from the Institute of Medi-cine that was published in 1999 stated that annually in theUS 7,000 deaths can be associated with medication errors.6

In light of these figures, it is not surprising that the BritishDepartment of Health recommends the wider use of elec-tronic prescribing to reduce the risk of medication errors.3

Electronic prescribing is supported by Computerized Phy-sician Order Entry (CPOE) systems. The term CPOE refers toa variety of computer-based systems for ordering medica-tions, which share the common features of automating themedication ordering process.7 The CPOE systems can rangefrom systems that only provide a list of possible medicationsthat the physician can then choose from, to systems provid-ing varying levels of decision support, including checks ofdrug-drug interactions, drug-allergy contraindications, or
checks of prescriptions concerning the patient’s recent lab-
mailto:[email protected]

mailto:[email protected]

http://jamia.oxfordjournals.org/

586 Ammenwerth et al., Effect of Electronic Prescribing: A Systematic Review

by guest on May 5, 2016


Dow

nloaded from

Table 2 y Characteristics of 27 Studies on Electronic Prescribing

PaperTime ofstudy Name of hospital Clinical setting

Level ofcare

Patientsincludedin study

Drugsincludedin study

Bates (1998) 1993–1995 Brigham and Women’s Hospital,Boston, USA

Six intensive, medical andsurgical units in a700-bed academictertiary-care hospital

Normal care No restrictions No restrictions

Bates (1999) 1992–1997 Brigham and Women’s’ Hospital,Boston, USA

One intensive and twogeneral medical careunits in a 700-bedacademic tertiary-carehospital


Bizovi(2002)

1999–2000 Oregon Health & ScienceUniversity, USA

Emergency department ofa tertiary care teachinghospital

Normal care Emergencycare patients

No restrictions

Chertow(2001)

1997–1998 Brigham and Women’s Hospital,Boston, USA

Medical, surgical,neurology, andobstetrics andgynaecology services of720-bed urban tertiarycare teaching hospital

Normal care Patients withrenalinsufficiency

Renally cleared ornephrotoxicmedications

Colpaert(2006)

2004 Ghent University Hospital,Belgium

22-bed intensive care unitat tertiary care hospital

Intensivecare

Intensive carepatients

No restrictions

Cordero(2004)

2001–2002 Ohio State University MedicalCenter, USA

Neonatal intensive careunits

Intensivecare

Very-low-birth-weightinfants

Caffeine andGentamicin

Evans(1998)

1992–1995 LDS Hospital, Salt Lake City,Utah, USA

12 bed shock traumarespiratory intensivecare unit

Intensivecare


Antiinfectives

Feldstein(2006)

2000–2004 Kaiser Permanente Northwest, ahealth maintenanceorganization, USA

8 primary care clinicswith 126 primary careclinicians

Normal care No restrictions Warfarin

Fraenkel(2003)

1995–1998 Royal Brisbane Hospital,Queensland, Australia

12-bed adult generalintensive care unit in a790-bed tertiary referralteaching hospital

Intensivecare


No restrictions

Galanter(2005)

2002–2003 University of Illinois Hospitaland Medical Center, USA

Not further described Normal care Patients withrenalinsufficiency

Renally cleared ornephrotoxicmedications





Dow

nloaded from

Table 2 y continued.

Name of system Type of systemElectronic prescribingintroduced since when Description of functionality

Type offunctionality Control situation

BICS Home grown Electronic prescribingintroduced as part ofBICS since 1993 withincreasing functionality

CPOE system with default doses,display of lab results,consequent orders, limiteddrug-allerg, dru-druginteraction checking anddrug-laboratory checking

AdvancedDSS

Orders were handwrittenin order books


(Period 3): CPOE system withdisplay of lab values,drug-allergy checking,redundant medicationchecking, drug-druginteraction checking, drug-labchecking, order sets

AdvancedDSS

Orders were handwrittenin order books

EmSTAT,CyberPlusCorporation(in use since1997)

Commercial Electronic prescribing wasintroduced in theemergency departmentin 2000

Prescription writing module withdisplay of patient name, age,sex, patient’s weight

No DSS Handwrittenprescriptions


Decision-support system thatcalculates estimated creatinineclearance and proposes arecommended dose andfrequency for the ordered drug

Limited DSS CPOE without specialadvice for patientswith renalinsufficiency withregard to dose andfrequency

CentricityCritical CareClinisoft, GEHealthcareEurope

Commercial Electronic prescribing wasimplemented in theintensive care unit 10months beforebeginning of study

ICU with incorporated CPOE:Protocol-basedrecommendation on drug, doseand frequency, alerts toimportant drug-druginteractions and drug-relatedcomplications, highlighting ofpatient allergy status

AdvancedDSS

Paper-based ordering

Invision 24,SiemensMedicalSolutions

Commercial Electronic prescribing wasintroduced to selectiveservices in February2000 and introduced tothe study units inMarch 2002

CPOE with best-practice basedorder sets, drug-allergy checks,drug-drug interactions, orderduplication, corollary orders,weight-based dosage,maximum dosage, drug-routerestriction

AdvancedDSS

All order were enteredmanually in the chartkept at the patientbedside

HELP Home grown The antiinfectivesmanagement programwas implemented in thestudy unit in 1994

Computerized antiinfectives-management program withadvice on antiinfectiveregiment (drug, dosing, dosinginterval) based on clinical dataand available evidence,drug-allergy checks, drug-druginteractions, information ontoxicity and costs

AdvancedDSS

No antiinfectives-management programwas used

EpicCare EMR,Epic Systems*(in use since1996)

Commercial Warfarin alerts wereimplemented in January2003

CPOE system with alerts on fivewarfarin-interacting drugs

AdvancedDSS

CPOE without alerts

CareVue,PhilipsMedicalSystems

Commercial Intensive care systemwith electronicprescribing wasintroduced on studyunit in November 1996

Medication administration recordmodule with pre-defined doseranges, drug scheduling anddrug preparation notes,prompt for drugadministration

No DSS Paper-baseddocumentation

Millenium andDiscernExpert,Cerner

Commercial Alerts were implementedduring study period

CPOE system that calculates thecreatinine clearance and checksfor potentially contraindicateddrugs

AdvancedDSS

CPOE without alert onpotentiallycontraindicated drugsfor patients with renal

Corporation insufficiency





Dow

nloaded from


PaperTime ofstudy Name of hospital Clinical setting

Level ofcare

Includedpatients Included drugs

Gandhi(2005)

1999–2000 Four Boston adult primary carepractices affiliated with anacademic medical center, USA

4 primary care practices Normal care Outpatientsover Age 18

No restrictions

Igboechi(2003)

1999–2002 Metropolitan Hospital Center,New York, USA

Inpatient and outpatientservices of 350-bedacute tertiary carehospital


Kim (2006) 2001–2004 John Hopkins Children’s Center,Baltimore, USA

Paediatric oncology Normal care Children withneoplasticdisorders

Chemotherapeutica

King (2003) 1993–1999 Children’s Hospital of EasternOntario, Canada

Three medical and twosurgical wards of atertiary care paediatrichospital

Normal care Children No restrictions

Kirk (2005) 2003 National University Hospital,Singapore

Paediatric unit in auniversity teachinghospital

Normal care Paediatricpatients(� 16 yearsof age)

Paracetamol andpromethazine

Koide(2000)

1994–1996 University of Tokyo Hospital,Japan

1040-bed tertiary care andteaching hospital

Normal care Patients withpsoriasis

Etretinate, amedicationindicated forpsoriasis

Maurer(2003)

1999–2000 Centre Hospitalier Le Raincy-Montfermell, Montfermell,France

General Pneumology Unitwith 60 beds

Normal care PneumologyPatients

No restrictions

Mitchell(2004)

2002 Southmead Hospital, Bristol, UK General surgery Normal care No restrictions No restrictions

Mullett(2001)

1998–1999 Primary Children’s MedicalCenter, University of Utah,USA

26-bed paedriatricintensive care unit of a232-bed primarychildren’s medicalcenter

Intensivecare

Children Antiinfective

Oliven(2005)

2001* Bnai-Zion Medical Center, Haifa,Israel

Two 44-beds generalmedical wards in anacute care universityhospital






Dow

nloaded from


Name of system Type of systemElectronic prescribingintroduced since when Description of functionality

Type offunctionality Control situation

Onecommercial,one home-grown system

Mixed No further informationprovided

Computerized prescribing withprinted prescriptions, requiredfields and non-mandatorydefault doses, without anyautomatic checks

No DSS Hand-writtenprescriptions

Ulticare SystemDatabase, PerSeTechnologies

Commercial Electronic prescribing wasintroduced in June 2001after recent upgrade toa new clinical system

CPOE system with drug-allergyalerts, drug interaction alerts,dosage range checks, labvalues, and drug informationresource

AdvancedDSS

Handwritten orders

RxTFCPharmacyInformationSystems, GEMedicalSystemsInformationTechnologies

Commercial Electronic prescribing wasintroduced as part ofthe available pharmacysystem in 2002

CPOE system with dosingsupport and drug-allergychecking*

AdvancedDSS

Paper-based order entry

Sunrise ClinicalManager,Eclyps

Commercial Electronic prescribing wasintroduced, withperiodic upgrades, onthe two interventionwards in 1996

CPOE without decision support(no further informationprovided)

No DSS Handwritten order entry

Not specified Home grown Existing clinical decisionsupport system (in usesince 2000) system wasmodified in March 2003to also supportelectronic prescribingfor paediatricdepartments

Computer calculated dosingbased on weight

Limited DSS Traditional computer-based prescription, nodosing advice

Medicalinformationsystem (inoperationsince 1973)

Unknown Electronic prescribing inoperation since 1988,alerts for etretinateadded in 1995

Alert system monitors ALTvalues and AST values

Limited DSS CPOE without alerts

Excel-basedapplication

Home grown Electronic prescribingintroduced in January2000

Prescription sheet with drug listand recommended dosingbased on weight and renalfunction

Limited DSS Handwrittenprescriptions

ClinicalManager3.0A, iSoftUK PLC

Commercial Electronic prescribingintroduced in generalsurgery as 3-months-pilot in 2002

Comparing unit with electronicmedication administrationrecord and electronicprescribing with areas ofhand-written prescriptions

No DSS Paper-based ordering

HELP Home grown Pediatric antiinfectivesdecision support toolwas implemented onintensive care units inJanuary 1999

Computerized antiinfectives-management program withdosing recommendationsbased on evidence and clinicaldata

Limited DSS Handwritten orders

Unknown Home grown Electronic prescribing wasintroduced three yearsbefore the study

CPOE system withadministration reminders,reminders for blood-drug-leveltest or further monitoring,extended drug information,treatment guidelines, drug-drug interaction, drug-allergy,drug-disease, drug-lab alerts,

AdvancedDSS

Handwritten drug orders

access to blood test results





Dow

nloaded from

oratory results. All those checks lead to alerts and remindersgiven to the ordering physician in case problems are de-tected.

In the scientific literature, many studies investigate theeffects of electronic prescribing on the quality of patient care.Many studies showed a reduction in medication errors (suchas Bates et al., 1999),8 while others described negative effectssuch as the facilitation of medication error risks9 or anincrease in mortality.10 The available reviews on this topicseem to not come to an unambiguous conclusion on theoverall effects of electronic prescribing, concluding thatpublished evidence is often insufficient.11–16 None of thepublished reviews has—to our knowledge—tried to quanti-tatively summarize the available evidence on the effects, andto systematically investigate the factors contributing to those


PaperTime ofstudy Name of hospital

Pestotnik(1996)

1988–1994 LDS Hospital, Salt Lake City,Utah, USA

520-bhos

Peterson(2005)

2001–2002 Brigham and Women’s’ Hospital,Boston, USA

Medineugynterthos

Potts (2004) 2001–2002 Vanderbilt Children’s Hospital,Nashville, USA

20-becarcar

Shulman(2005)

2001–2002 University College HospitalsLondon, Middlesex hospital,London, UK

Inten

Spencer(2005)

2002–2003 University of North Carolina,Chapel Hill, USA

Twoflooaca

Tamblyn(2003)

1997–1998 General Practicioners in Quebec,Canada

107 pphy

Teich (2000) 1991–1995* Brigham and Women’s’ Hospital,Boston, USA

720-bme

*this information was provided by the author upon request.

effects.

Therefore, we performed a systematic and quantitative reviewto determine the effect of electronic prescribing on the risk ofmedication errors and adverse drug events (ADEs), and ana-lyzed factors for those effects. This review should inform thehealth informatician of the effects that can be expected ofelectronic prescribing, and on the factors that influence thesuccess of such systems. We see this review as a contribution tosupport evidence-based health informatics.17

MethodsIdentification of TrialsWe searched MEDLINE (1966 to April 2006) and EMBASE(1976 to April 2006) for studies on electronic prescribing thatevaluated its effect on medication errors and ADEs. Wecombined MeSH terms, such as “Medical Order Entry

l settingLevel of

careIncludedpatients Included drugs

te-care referral Normal care No restrictions Surgical antibiotics

gical,, andgy services ofre academic

Normal care Elderlypatients (65years andolder)

Psychotropicmedication

iatric criticalin a tertiary-ren’s hospital

Normal care Children No restrictions

re Unit Intensivecare


No restrictions

l medicine688-bed

hospital


care Normal care Elderlypatients (66years andolder)

No restrictions

an academicnter

Normal care Adults No restrictions

Clinica

ed acupital

cal, surrologyaecoloiary capital

d paede unite child

sive Ca

generars of a

demic

rimarysicians

ed urbdical ce

Systems,” “prescriptions, drug,” “drug therapy, computer-





Dow

nloaded from

assisted” with general search terms such as “order entry,”“CPOE,” “POE,” “order communication,” “prescription sys-tem,” “drug prescription,” “prescribing,” “ordering,” and“computerized reminders.” Those search terms were com-bined with terms that searched for evaluation studies thatwere taken from an earlier literature survey.18 The completequery is available upon request.

We also searched the Cochrane Database of SystematicReviews and examined the reference lists of the relevantreviews in order to find further studies. Finally, we performeda hand search of the Journal of the American Medical Infor-matics Association (1994–2006), of the International Journal ofMedical Informatics (1997–2006), and of the Methods of Infor-mation in Medicine (1990–2006). We completed this by hand-searching the references of retrieved study papers. We did not


Name of system Type of systemElectronic prescribingintroduced since when

HELP Home grown Surgical antibioticsmanagement moduleswere developed since1985

Anwb

BICS Home grown Decision-support forpsychotropicmedication wasdeveloped andimplemented for thestudy

CPgra

WizOrder(developed in1994)

Home grown CPOE was implementedin critical care unit inDecember 2001

CPsdipld

QS 5.6 ClinicalInformationSystem, GEHealthcare

Commercial QS with electronicprescribing wasintroduced in April2002

CPd

SiemensMedicalSolutions,Malvern, PA

Commercial Siemens CPOE systemwas implemented ontwo study units inOctober 2002 resp.January 2003

Patp

Unknown Commercial Electronic prescribing wasintroduced tointervention groupphysicians two monthsafter randomization

Hedpdi


CPrddacr

restrict the search to any single language.

Inclusion Criteria and the Selection of StudiesIntervention: We included studies wherein the interven-tion was electronic prescribing. As an intervention, weincluded all the computer-based application systems toorder drugs that are used at the point of care. We includedelectronic prescribing systems independent of the level ofdecision support that they provided (e.g., with or withoutalerts on drug-drug interaction), and for all types ofdrugs. We excluded the studies that used electronicprescribing only for ordering diagnostic tests or therapeu-tic procedures.

Control: We only included studies wherein either an elec-tronic prescribing system was compared with handwrittenordering, or where an electronic prescribing system with a

ption of functionalityType of

functionality Control situation

management programtient-specific guideline-linical decision support

AdvancedDSS

No antibioticmanagement programavailable,computerized orderingmainly related tonursing orders andsupply orders*

th evidence-basedc-specificendations for dosingstitution

Limited DSS CPOE with dosingsuggestions andrecommendations forsubstitutions

th decision supportdrug allergy alerts,ecking, drugion alerts, clinicalys-based order sets,drug monographs,

b alerts

AdvancedDSS

No CPOE

ponent withoutsupport

No DSS Hand-written drugcharts

lergy screening wasn use*

AdvancedDSS

No CPOE

cord software whichnted medicationed, offering drug-contraindications andtion on previous drugs

AdvancedDSS

No computerizeddecision support

tem with dosingendation, drug-allergy,ug interactions,te medications, possibleive medication,ent orderendation

AdvancedDSS

Paper-based ordering

Descri

tibioticith pa

ased c

OE wieriatriecommnd sub

OE wiuch asose ch

nteractathwa

inks torug-la

OE comecision

ient alartly i

alth-reocumerescribisease

nformaOE sysecommrug-druplicalternatonsequecomm

more sophisticated functionality (e.g., a drug-drug interac-





Dow

nloaded from

tion alert) was compared with another less sophisticatedsystem.

Population and setting: We included studies where physi-cians were the primary users of the electronic prescribingsystem. We excluded studies where other groups (e.g.,nurses, pharmacists) were the primary users. We includedall clinical settings such as outpatient care, inpatient care,and intensive care. We included all patient groups.

Study design: We included randomized controlled trials,non-randomized controlled trials, before-after trials, as wellas time-series analysis with multiple measurements. Weonly included field studies and excluded all lab studies andsimulation studies.

Outcomes: We included studies that evaluated the effect ofelectronic prescribing on medication errors, potential ADEs,and ADEs. We defined medication error as all errors in theprocess of ordering, transcribing, dispensing, administering,and monitoring medication. This included an inappropriatedrug, dosing, frequency, route, or timing (when related topatient safety), involving problems such as illegible orunsigned orders, and problems related to drug-allergy,drug-drug, drug-lab, and other interactions. Potential ADEswere defined as a medication error with significant potentialto harm a patient that may or may not actually reach apatient. Adverse drug events (ADE) were defined as patientinjuries resulting from drug use. We excluded studies wheremedication errors or ADEs were not the primary focus of thestudy, and studies that were still ongoing. We excludedpapers if groups were definitely not comparable. If the datareporting was unclear, we contacted the authors and re-quested further information.

Data Extraction and Study Quality AssessmentWe extracted data from the text, tables, and graphs of theoriginal publications. Two reviewers (EA and CM) exam-ined the data and reached consensus after discussion. Inaddition, one reviewer (PSI) independently reviewed all theextracted data. All of the cases of discordant data wereresolved by discussion.

To detect medication error rates and ADE rates, we used thedefinition that is provided in each paper (please see Table 3,available online at www.jamia.org, for the definition ofmedication error and ADE for each respective paper).

When no absolute numbers were provided for medicationerror or ADE, we calculated these numbers based on thegiven data (e.g., if the frequency of ADEs was only given per1.000 patient-days, the absolute ADE number could becalculated from the number of patient-days). To determinethe study size, we used the number of orders as an obser-vation unit of the analysis. If the number of orders was notavailable, the number of patients or patient-days was used.If multiple data were reported in a study (e.g., in time-seriesanalysis), we used the data of the last reported measure-ment.

We classified the functionality of the CPOE system either as

• no decision support: selection of drugs from a list,information on available doses and on costs, access todrug monographs, no further decision-support;

• limited decision support: evidence-based patient-specific
recommendation of a drug, dosing, frequency etc.; or
• advanced decision support: at least some drug-allergy,drug-drug interaction, drug-lab, or other patient-specificalerts.

All results were reported in systematic evidence tables. Thestudy quality was assessed by using a checklist (please seeTable 1, available online at www.jamia.org), which wasdeveloped based on a 16-item assessment tool by the Ger-man Scientific Working Group Technology Assessment forHealth Care that was applied independently by two review-ers (PSI, EA). Differences in judgment were then solved bydiscussion.

Statistical AnalysisFor each study, the risk ratio (RR) with its 95% confidenceinterval (CI) was calculated by comparing medication errorrates, potential ADE rates, and ADE rates between theintervention and comparison group. If available, the numberof orders was used as the denominator. Otherwise, thenumber of patient-days or the number of patients was usedas the denominator.

We used a graphical approach based on forest plots toperform sub-group analysis and arranged studies by in-creasing risk ratios within subgroups. Subgroups were apriori defined as potentially relevant such as the clinicalsetting (inpatient, outpatient, or intensive care), patientgroup, type of drugs, type of system (home-grown orcommercial), functionality (no, limited, or advanced deci-sion support), study design, and method to detect errors.

All the analyses were performed with the software packageSTATA 9.2 (StataCorp, College Station, Texas, USA).

ResultsOverall, we identified 172 evaluation studies and 15 system-atic reviews. From the 172 evaluation studies, 27 studies metall the inclusion criteria and were, therefore, included in adetailed review.8,19–44 For details, see Figure 1, availableonline at www.jamia.org. We obtained additional data fromeleven authors.

Characteristics of the TrialsTable 2 shows the general characteristics of all the analysed27 studies.

The majority of studies were conducted in the United Statesand in normal inpatient care units. Most studies wereunspecific concerning the included patients and concerningthe ordered drug. Half of the studies evaluated a commer-cial system, and half evaluated a home-grown, self-devel-oped system. All of the systems offered a display of basicpatient data (name, age), a list of drugs from which aprovider can choose, and a list of potential doses for achosen drug. Half of the studies used a system with ad-vanced decision support, the others used systems with no orlimited decision support. In most trials, electronic prescrib-ing was compared to handwritten ordering.

Table 3, available online at www.jamia.org, shows thefurther study characteristics of the 27 studies, showing thatmost of them were before-after studies (including time-series analysis), with only two studies being randomizedcontrolled trials.

Of those 27 studies, 15 only evaluated medication errors,
two were only on ADE, and 10 reported both on medication
http://www.jamia.org





f elect




Dow

nloaded from

errors and ADEs. Table 4, available online at www.jamia.org, shows the evidence table for outcomes.

Effect on Medication ErrorsTwenty-five studies reported on the risk of medicationerrors (Figure 2). Twenty-three of these studies showed asignificant relative risk reduction, with a risk ratio between0.01 and 0.87. This indicates a relative risk reduction formedication errors of 13% to 99%. One study reported astatistically significant increase of 26% for the risk of medi-

F i g u r e 3. Risk ratios of nine studies analyzing the effect

F i g u r e 2. Risk ratios of 25 studies analyzing the effect o

of electronic prescribing on potential ADEs.

cation errors (Spencer 2005).42 In this study, there was noconnection between the CPOE system and the pharmacysystem, so that orders had to be entered twice, in turnleading to transcription errors in the intervention group. Inaddition, the study authors only analysed those errors thatwere reported by the staff members themselves (voluntary

F i g u r e 4. Risk ratios of seven studies analyzing the effectof electronic prescribing on ADEs. Shulman was excluded,because there was no event in either the intervention group

ronic prescribing on medication errors.

or the comparision group.







Dow

nloaded from

reporting system)—this constitutes a potential detectionbias. The one study (Mitchell 2004)35 showing inconclusiveresults (RR � 1.02, 95%-CI: 0.88; 1.19) was one of the smalleststudies in this review, with only 320 orders analysed.

Effect on Potential Adverse Drug EventsNine studies reported on the risk of potential ADE (Figure3). Six of these studies showed significant relative riskreduction, with a risk ratio between 0.02 and 0.65. Thisindicates a relative risk reduction for potential ADEs of 35%to 98%.

Two studies showed inconclusive effects (Bizovi 2002 andGandhi 2005).20,28 Both of these studies were rather smallstudies. The remaining also very small study by Mitchell2004 showed a significant increase in the risk for potentialADEs.35

Effect on Adverse Drug EventsSeven studies reported the risk of ADE (Figure 4). In one ofthem (Shulman 2005),41 no events occurred in the interven-tion or in the comparison group and, therefore, this studywas excluded from further analysis.

Four of the six remaining studies showed a significant

F i g u r e 5. Sub-group analysis of the effect of home-gromedication errors (n�25).

relative risk reduction for ADEs, with a risk ratio between

0.16 and 0.70. This indicates a relative risk reduction forADEs of 30% to 84%. One study (Mullett 2001)36 showed anot statistically significant relative risk reduction of 13%,and the remaining study (Bates 1998)19 showed a small, notstatistically significant increase of 9%.

Sub-group AnalysisA graphical sub-group analysis was only performed forthose 25 studies that focused on medication errors.

The sub-group analysis comparing medication errorreduction by commercial systems versus home-grownsystems indicated a higher relative risk reduction byhome-grown systems (Figure 5).

The sub-group analysis for those seven studies that comparedadvanced electronic prescribing with limited electronic pre-scribing and those 18 studies that compared electronic pre-scribing with handwritten ordering showed a higher relativerisk reduction for the comparison made with handwrittenordering (Figure 6).

The 14 studies with an advanced decision-support reporteda higher relative risk reduction than the 11 studies withlimited or no decision-support (Figure 7). Most studies with

rsus commercial systems on the relative risk reduction of
wn ve
limited support, however, were compared to computer-


group




Dow

nloaded from

based ordering, whereas most studies with advanced deci-sion-support or without decision-support were compared tohandwritten ordering.

The sub-group analysis comparing different methods todetect errors showed higher relative risk reductions with themanual chart review of prescriptions (11 studies) than withthe automatic database analysis of prescriptions (6 studies)(please see Figure 8, available online at www.jamia.org).

The other graphical sub-group analysis did not indicatedifferences between groups: The risk ratios betweengroups seem to be similar for the level of care (normalcare versus intensive care), patient groups included (el-derly, children, unspecific), type of drugs included (spe-cific or unspecific), or study design (before-after trial,controlled trial, RCT).

Study and Reporting QualityThe reporting quality was poor for most of the studies. Thedetails are shown in Table 5, available online at www.jamia.org. For example, many studies did not clearly specify theinclusion or exclusion criteria of the participating institu-tions or patients, did not report on the baseline characteris-

F i g u r e 6. Sub-group analysis of the type of comparison

tics of participants or as to whether the comparison and

intervention groups were treated similarly (with the excep-tion of the intervention), or did not comment on missingvalues or drop-outs.

In addition, only two studies were randomized trials, andfew studies had comparison groups recruited over the sameperiod. Only in half of the studies, the outcome seemed to bemeasured validly and reliably, however, the measurementwas mostly not blinded. Nearly all the studies used ade-quate hypothesis tests and reported on the statistical preci-sion of the main outcomes, but only less than half of thestudies attempted to adjust for confounding or clustering.Most studies were single centre studies, so that the general-isability of the results to other centres, especially of adifferent type, is unclear.

DiscussionMain FindingsThwenty-three of the 25 studies that analysed the effects onthe medication error rate showed a significant relative riskreduction of 13% to 99% for medication errors. Six of thenine studies that analysed the effects on potential ADEs

on the relative risk reduction of medication errors (n�25).

showed a significant relative risk reduction for potential








Dow

nloaded from

ADEs of 35% to 98%. Four of the seven studies that analysedthe effect of electronic prescribing on ADEs showed asignificant relative risk reduction for ADEs of 30% to 84%.These findings, that are in line with results from otherreviews, indicate that electronic prescribing can substan-tially reduce the risk for medication errors, potential ADEs,and ADEs.12,13,16,45–47 The low number of studies focussingon ADE may be due to the fact that medication errors can bedetected more easily than ADEs.16

From a clinical perspective, the observed effect sizes (e.g., upto 99% reduction of medication errors, up to 84% reductionof ADEs) seems to be rather large. However, the medicationerrors and ADEs are just surrogate outcomes, and are notnecessarily directly related to changes in the patient-relevantmedical outcomes.48 The actual improvements in medicaloutcomes (e.g., reduction in mortality rates or hospitalisa-tion days) have not yet been sufficiently analysed by quan-titative, controlled trials.15,45,48 One exception is the study byHan et al.10 that observed an increase in mortality after theintroduction of a CPOE system (these heavily discussed

F i g u r e 7. Sub-group analysis of the effect of the level of(n�25).

findings could not be reproduced later on by others such as

Del Baccaro et al.49). Altogether, more systematic trialslooking at patient-relevant medical outcomes should beconducted in the future. Furthermore, more studies shouldalso be conducted to analyse the costs for any benefit thatmay be obtained. For this review, we concentrated onmedication errors and ADEs, as a sufficient amount ofpublished evidence is available.

Results of the Sub-group AnalysisThe included studies showed substantial heterogeneity (seetheir detailed description in Table 2 as well as in Table 3,available online at www.jamia.org). This reflects the diver-sity of electronic prescribing systems and their respectiveuse.

The sub-group analysis indicated that the level of care, aswell as the included patient groups and drugs did not affectthe overall risk ratios. The graphical presentation, however,indicated that home-grown systems show a higher relativerisk reduction than commercial systems. This is not surpris-ing, as home-grown systems can be modified and adapted

ionality on the relative risk reduction of medication errors
funct
easier and more quickly to the local needs. Rigby50 calls this






Dow

nloaded from

“alpha sites” and argues that those sites are atypical con-cerning larger technical, emotional, and financial support.

Also, studies comparing handwritten prescribing with elec-tronic prescribing seem to show a higher relative riskreduction than those studies comparing groups with adifferent level of electronic ordering. This is also not surpris-ing, as certain error types such as illegible orders, that areoften present in handwritten prescribing, are completelyremoved by electronic prescribing.

Electronic prescribing systems with advanced decision sup-port seemed to show a higher relative risk reduction com-pared to those with limited or no decision support. It is herethat the patient-specific alerts seem to best support thequality of prescribing.

Finally, different methods for detecting medication errorsseem to show a different relative risk reduction. Using amanual chart review, relative risk reduction seems to behighest. However, chart review can hardly be blinded, andreviewers may be biased towards CPOE.

Study QualityOverall, the reporting quality and study quality of theincluded studies were not always satisfactory.

One major problem was that many studies did not providesufficient information in order to adequately assess thecomparability of the intervention and comparison groups(e.g., by not reporting baseline characteristics, by not report-ing about adjunct initiatives such as training sessions orabout drop-outs etc.). This made it difficult to analyse as towhether the differences found between the study groupsreally originated from electronic prescribing, or from otherfactors. Subsequently, about two-thirds of the studies didnot attempt to adjust for possible confounding factors. Then,many studies also used designs of lower level of evidencesuch as before-after trials, here it is unclear whether anycontext such as staffing or workflow of the study depart-ments may have changed over time, in turn influencing theobserved effects. The same is true for any non-randomizedallocation of clinicians and/or patient to study groups. Thisall affects the validity of the analysed studies. We dealt withthis by describing in detail the individual study quality (seeTable 5, available online at www.jamia.org), and by exclud-ing those studies that seem to have non-comparablegroups.51,52

There were other limitations of the included studies: Fourstudies analysed the data from voluntary error reportingsystems (e.g., nurses and physicians voluntary reported onerrors), which may underestimate the real number of errorsand ADE in those studies (detection bias). Other studiesretrospectively analysed routinely collected information onmedication errors and ADEs. Here, it is partly unclear as tohow they assessed the quality of this routine data. Overall,only a few studies provided information on the objectivityand reliability of the instruments that were used to (manu-ally or automatically) detect errors.

Summarizing, the reporting and study quality were notalways adequate. This must be taken into account wheninterpreting the results of the studies. Efforts should bemade to improve the reporting quality and study design andanalyses of evaluation studies. Publication guidelines that
are comparable to CONSORT53 or STROBE54 should be
developed, taking into account the often quasi-experimentaldesign of the health informatics evaluation studies. Oneexample for such publication guidelines for health informat-ics evaluation studies are the STARE-HI guidelines (Stan-dards for Reporting on Evaluation Studies in Health Infor-matics) that will shortly be published.55

Limitations of the ReviewOur review may present some limitations.

First, the published trials on electronic prescribing differsubstantially in their respective terminology. We used manydifferent Keywords to search for trials; searched in severaldatabases as well as for non-English papers; and appliedhand-searching. Nevertheless, we may have overlookedsome individual studies. In addition, there is the possibilityof publication bias, as we have only included publishedstudies.

Second, the definition of the measured outcome variablediffered between the studies. For example, some studiesincluded the legibility of orders in the definition of medica-tion, and others did not. This can affect the measuredmedication error rate.

Third, when several end-points were reported (e.g., in atime-series analysis), we only included the final endpoint.This may overestimate the positive effects of electronicprescribing, as the intermediate results partly show less (oreven negative) effects during an early introduction period.In addition, our analysis relied on the assumption that oneorder could have maximal one error. However, this was notexplicitly reported by several studies.

Fourth, our categories that were used to stratify the sub-groups might have been too crude. For example, the studydesign was very heterogeneous and the diversity might nothave been represented adequately by the categories wechose. A multivariate sub-group analysis controlling for themultiple factors simultaneously was not performed becauseof the limited number of studies. Then, further factors suchas the organisation of medication processes in the varioussettings, age and computer knowledge of users, manage-ment support, or the quality and usability of the CPOEsystem may further affect the effects. Information on thoseitems was not included in most studies and could, therefore,not be used for sub-group analysis.

Fifth, the rather large percentage of home-grown systems inour review may have biased this review towards morepositive results. Moreover, 8 of the studies included camefrom just two sites: the Brigham and Women’s Hospital,Boston, MA with the BICS system, and the LDS Hospital,Salt Lake City, Utah, with the HELP systems. Therefore, wecannot exclude a selection bias, with the observed effects notbeing representative for general routine clinical settings.Some studies on BICS and HELP may not be completelyindependent of each other (we therefore excluded Evans etal., 1994),56 but when they used different outcome criteria oraddressed different patient populations, we decided toinclude them.

Sixth, we were unable to use meta-analysis to pool the effectsizes, given the large heterogeneity among the studies.Meta-analysis would help to quantify the overall effect, andhas already been performed e.g., for computer-assisted
management of anticoagulant therapy57 or for clinical re-





Dow

nloaded from

minder systems for preventive care,58 but needs a sufficientnumber of homogeneous studies. We therefore conducted agraphical sub-group analysis—even this, however, shouldbe viewed with care, given the tremendous variability instudy design and in the methodological rigor of the includedstudies.

Seventh, most of the evaluated systems have only beenimplemented shortly before or during the study, and some-times only as pilots in selected areas. This may have led toerroneous estimations concerning the long-term effects ofelectronic prescribing. More long-term studies seem to beneeded.

Finally, researchers such as Berger15 note that electronicprescribing systems could increase the number of medica-tion errors and ADEs when not appropriately designed andimplemented. There is, in fact, increasing evidence from(mostly qualitative) research pointing to risks with elec-tronic prescribing by CPOE systems. Koppel et al.,9 e.g.,described an increase in medication errors risks by CPOE.Campbell et al.59 identified categories of unintended conse-quences of CPOE. This (qualitative) evidence is, however,not (yet) reflected in the results of quantitative controlledevaluation studies and therefore also not in our review. Thediscussion of the reasons for this (such as a possible publi-cation bias) is outside the scope of the present paper. For amore detailed discussion see Ammenwerth et al.60

Implication and ConclusionAfter having been envisioned by Morris Collen in 1970(“Physicians should enter their medical orders directly intothe computer”),61 electronic prescribing systems have nowbeen in routine use for approximately 15 years. Our reviewis, to our knowledge, the first that quantitatively calculatesand compares the risk ratios for each evaluation study. Wealso conducted a sub-group analysis, hereby analysing fac-tors that influence the effects of electronic prescribing; andwe analysed and discussed carefully the heterogeneous andoften weak quality of published studies.

Our results indicate that electronic prescribing seems to be auseful intervention for reducing the risk of medicationerrors and ADEs. We found that there is especially goodevidence for a positive effect of electronic prescribing offer-ing advanced decision support functionality in hospitalsettings. Less evidence is available for systems outside theU.S., for primary care settings, and for specific commercialsystems. Therefore, more studies are necessary.

The study quality was often weak, with many before-aftertrials and an unclear comparability of the study groups.More randomized controlled trials from more sites areneeded to further improve the evidence and to identify thesetting that those systems are most useful in. Researchersplanning comparable studies are advised to focus early onthe issue of study quality and study validity, to avoid asmany sources of bias as possible (see also Brender, 200262). Alarger number of high-quality studies for certain settingsand patient groups would also allow for the use of meta-analysis to pool the effect sizes. In addition, the reportingquality of health informatics evaluation studies has to besubstantially improved; here, the initiatives for publicationguidelines such as STARE-HI55 are urgently needed. All this
should help to increase our knowledge on electronic pre-
scribing and globally increase the evidence-base of healthinformatics.17

Systematic reviews of quantitative trials can indicate whicheffect can be obtained from an intervention. They do notshow, however, how to implement electronic prescribing. Itseems obvious that the quality of the implementation pro-cess has an impact on the success of a CPOE implementa-tion.63 Insufficient implementation planning, or systems thatare not integrated into the general information systems, maylead to negative effects on the process and even the outcomeof care.64–67 Negative effects of electronic ordering wereidentified only in two studies in our review, even whenincreasing evidence from qualitative research points to pos-sible adverse effects of electronic prescribing. Each imple-mentation should be carefully monitored concerning anincrease in medication errors and ADEs, and more qualita-tive research should be conducted to assess the reasons forthe respective success or failure, and to guide furthersuccessful implementation processes.

References y

1. Barach P, Small SD. Reporting and preventing medical mishaps:lessons from non-medical near miss reporting systems. BMJ2000;320(7237):759–63.

2. Council of Europe. Committee of Experts on Management ofSafety and Quality in Health Care (SP-SQS) - Expert Group onSafe Medication Practices: Glossary of terms related to patientand medication safety. 2005; Available at: http://www.who.int/patientsafety/highlights/COE_patient_and_medication_safety_gl.pdf. Accessed on April 1st, 2008

3. Smith J. Building a Safer National Health System for Patients: Improv-ing Medication Safety. London: Department of Health; 2004. Avail-able at: http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_4071443. Ac-cessed on April 1st, 2008.

4. Institute of Medicine. Preventing Medication Errors: QualityChasm Series. Washington DC: Institute of Medicine; 2006.

5. Reinertsen JL. Let’s talk about error. BMJ 2000;320(7237):730.6. Kohn L, Corrigan J, Donaldson M, editors. To err is Human:

Building a Safer Health System. Washington: National Acad-emy Press; 2000.

7. AHRQ. Making Health Care Safer - A Critical Analysis ofPatient Safety Practices. Chapter 6: Computerized PhysicianOrder Entry (CPOE) with Clinical Decision Support Systems(CDSSs). Evidence Report/Technology Assessment, No. 43:Agency for Healthcare Research and Quality. Available at:http://www.ahrq.gov/clinic/ptsafety. Accessed on April 1st,2008.

8. Bates D, Teich J, Lee J, et al. The impact of computerizedphysician order entry on medication error prevention. JAMIA1999;6:313–21.

9. Koppel R, Metlay J, Cohen A, et al. Role of computerizedphysician order entry systems in facilitating medication errors.JAMA 2005;293(10):1197–2003.

10. Han YY, Carcillo JA, Venkataraman ST, et al. Unexpectedincreased mortality after implementation of a commercially soldcomputerized physician order entry system. Pediatrics 2005;116(6):1506–12.

11. Kaushal R, Shojania KG, Bates DW. Effects of computerizedphysician order entry and clinical decision support systems onmedication safety: a systematic review. Arch Intern Med 2003;163(12):1409–16.

12. Rothschild J. Computerized physician order entry in the criticalcare and general inpatient setting: a narrative review. J Crit Care
2004;19(4):271–8.
http://www.who.int/patientsafety/highlights/COE_patient_and_medication_safety_gl.pdf



http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_4071443

http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_4071443

http://www.ahrq.gov/clinic/ptsafety





Dow

nloaded from

13. Kuperman GJ, Gibson RF. Computer physician order entry:benefits, costs, and issues. Ann Intern Med 2003;139(1):31–9.

14. Johnston M, Langton K, Haynes R, Mathieu A. Effects ofComputer-based Clinical Decision Support Systems on Clini-cian Performance and Patient Outcome—A Critical Appraisal ofResearch. Ann Intern Med 1994;120:135–42.

15. Berger RG, Kichak JP. Computerized physician order entry:helpful or harmful? J Am Med Inform Assoc 2004;11(2):100–3.

16. Eslami S, Abu-Hanna A, de Keizer NF. Evaluation of OutpatientComputerized Physician Medication Order Entry Systems: ASystematic Review. J Am Med Inform Assoc 2007;14(4):400–6.

17. Ammenwerth E, De Keizer N. The evidence-based in healthinformatics: An investigation on publication and non-publica-tion in health care informatics. J Am Med Inform Assoc 2007;14(3):368–71.

18. Ammenwerth E, de Keizer N. An inventory of evaluationstudies of information technology in health care: Trends inevaluation research 1982–2002. Methods Inf Med 2005;44:44–56.

19. Bates DW, Leape LL, Cullen DJ, et al. Effect of computerizedphysician order entry and a team intervention on prevention ofserious medication errors. JAMA 1998;280(15):1311–6.

20. Bizovi KE, Beckley BE, McDade MC, et al. The effect ofcomputer-assisted prescription writing on emergency depart-ment prescription errors. Acad Emerg Med 2002;9(11):1168–75.

21. Chertow GM, Lee J, Kuperman GJ, et al. Guided medicationdosing for inpatients with renal insufficiency. JAMA 2001;286(22):2839–44.

22. Colpaert K, Claus B, Somers A, Vandewoude K, Robays H,Decruyenaere J. Impact of computerized physician order entryon medication prescription errors in the intensive care unit: acontrolled cross-sectional trial. Crit Care 2006;10(1):R21.

23. Cordero L, Kuehn L, Kumar RR, Mekhjian HS. Impact ofcomputerized physician order entry on clinical practice in anewborn intensive care unit. J Perinatol 2004;24(2):88–93.

24. Evans R, Pestotnik S, Classen D, et al. A Computer-AssistedManagement Program for Antibiotics and Other AntiinfectiveAgents. NEJM 1998;338(4):232–260.

25. Feldstein AC, Smith DH, Perrin N, et al. Reducing warfarinmedication interactions: an interrupted time series evaluation.Arch Intern Med 2006;166(9):1009–15.

26. Fraenkel DJ, Cowie M, Daley P. Quality benefits of an intensivecare clinical information system. Crit Care Med 2003;31(1):120–5.

27. Galanter WL, Didomenico RJ, Polikaitis A. A trial of automateddecision support alerts for contraindicated medications usingcomputerized physician order entry. J Am Med Inform Assoc2005;12(3):269–74.

28. Gandhi TK, Weingart SN, Seger AC, et al. Outpatient prescrib-ing errors and the impact of computerized prescribing. J GenIntern Med 2005;20(9):837–41.

29. Igboechi C, Ng C, Yang C, Buckner A. Impact of computerizedprescriber order entry on medication errors at an acute tertiarycare hospital. Hosp Pharm 2003;38:277–231.

30. Kim GR, Chen AR, Arceci RJ, et al. Error reduction in pediatricchemotherapy: computerized order entry and failure modesand effects analysis. Arch Pediatr Adolesc Med 2006;160(5):495–8.

31. King WJ, Paice N, Rangrej J, Forestell GJ, Swartz R. The effect ofcomputerized physician order entry on medication errors andadverse drug events in pediatric inpatients. Pediatrics 2003;112(3 Pt 1):506–9.

32. Kirk RC, Li-Meng Goh D, Packia J, Min Kam H, Ong BK.Computer calculated dose in paediatric prescribing. Drug Saf2005;28(9):817–24.

33. Koide D, Ohe K, Ross-Degnan D, Kaihara S. Computerizedreminders to monitor liver function to improve the use of
etretinate. Int J Med Inform 2000;57(1):11–9.
34. Maurer C, Lecointre K, Cachin N, et al. [Impact of medicalprescription computerisation on the incidence of adverse drugeffects]. Rev Mal Respir 2003;20(3 Pt 1):355–63.

35. Mitchell D, Usher J, Gray S, et al. Evaluation and audit of a pilotof electronic prescribing and drug administration. J Inform TechHealthcare 2004;2(1):19–29.

36. Mullett CJ, Evans RS, Christenson JC, Dean JM. Developmentand impact of a computerized pediatric antiinfective decisionsupport program. Pediatrics 2001;108(4):E75.

37. Oliven A, Michalake I, Zalman D, Dorman E, Yeshurun D, OdehM. Prevention of prescription errors by computerized, on-linesurveillance of drug order entry. Int J Med Inform 2005;74(5):377–86.

38. Pestotnik SL, Classen DC, Evans RS, Burke JP. Implementingantibiotic practice guidelines through computer-assisted deci-sion support: clinical and financial outcomes. Ann Intern Med1996;124(10):884–90.

39. Peterson JF, Kuperman GJ, Shek C, Patel M, Avorn J, Bates DW.Guided prescription of psychotropic medications for geriatricinpatients. Arch Intern Med 2005;165(7):802–7.

40. Potts AL, Barr FE, Gregory DF, Wright L, Patel NR. Computer-ized physician order entry and medication errors in a pediatriccritical care unit. Pediatrics 2004;113(1 Pt 1):59–63.

41. Shulman R, Singer M, Goldstone J, Bellingan G. Medicationerrors: a prospective cohort study of hand-written and comput-erised physician order entry in the intensive care unit. Crit Care2005;9(5):R516–21.

42. Spencer DC, Leininger A, Daniels R, Granko RP, Coeytaux RR.Effect of a computerized prescriber-order-entry system on re-ported medication errors. Am J Health Syst Pharm 2005;62(4):416–9.

43. Tamblyn R, Huang A, Perreault R, et al. The medical office ofthe 21st century (MOXXI): effectiveness of computerized deci-sion-making support in reducing inappropriate prescribing inprimary care. CMAJ 2003;169(6):549–56.

44. Teich JM, Merchia PR, Schmiz JL, Kuperman GJ, Spurr CD,Bates DW. Effects of computerized physician order entry onprescribing practices. Arch Intern Med 2000;160(18):2741–7.

45. Georgiou A, Williamson M, Westbrook JI, Ray S. The impact ofcomputerised physician order entry systems on pathology ser-vices: A systematic review. Int J Med Inform 2006;76(7):514–29.

46. Wolfstadt JI, Gurwitz JH, Field TS, et al. The effect of comput-erized physician order entry with clinical decision support onthe rates of adverse drug events: a systematic review. J GenIntern Med 2008;23(4):451–8.

47. Eslami S, Keizer NF, Abu-Hanna A. The impact of computer-ized physician medication order entry in hospitalized pa-tients-A systematic review. Int J Med Inform 2008;77(6):365–76.

48. Conroy S, Sweis D, Planner C, et al. Interventions to reducedosing errors in children: a systematic review of the literature.Drug Saf 2007;30(12):1111–25.

49. Del Beccaro MA, Jeffries HE, Eisenberg MA, Harry ED. Com-puterized provider order entry implementation: no associationwith increased mortality rates in an intensive care unit. Pediat-rics 2006;118(1):290–5.

50. Rigby M. Essential prerequisites to the safe and effective wide-spread roll-out of e-working in healthcare. Int J Med Inform2006;75(2):138–47.

51. Fontan JE, Maneglier V, Nguyen VX, Loirat C, Brion F. Medi-cation errors in hospitals: computerized unit dose drug dispens-ing system versus ward stock distribution system. Pharm WorldSci 2003;25(3):112–7.

52. Upperman JS, Staley P, Friend K, et al. The impact of hospital-wide computerized physician order entry on medical errors in apediatric hospital. J Pediatr Surg 2005;40(1):57–9.

53. Begg C, Cho M, Eastwood S, et al. Improving the Quality ofReporting of Randomized controlled Trials - The CONSORT
Statement. JAMA 1996;276(8):637–9.


Dow

nl

54. Strobe. STROBE Statement - Strengthening the Reporting ofObservational studies in Epidemiology (Version 3). 2005; Avail-able at: http://www.strobe-statement.org/Checklist.html. Ac-cessed on April 1st, 2008.

55. Talmon J, Ammenwerth A, Brender J, de Keizer N, Nykänen P,Rigby M. STARE-HI—Statement on the Reporting of EvaluationStudies in Health Informatics. Int J Med Reform 2008; to bepublished. See also http://iig.umit.at/efmi/starehi.htm. Ac-cessed on April 1st, 2008.

56. Evans RS, Classen DC, Pestotnik SL, Lundsgaarde HP, Burke JP.Improving empiric antibiotic selection using computer decisionsupport. Arch Intern Med 1994;154(8):878–84.

57. Chatellier G, Colombet I, Degoulet P. An overview of the effectof computer-assisted management of anticoagulant therapy onthe quality of anticoagulation. Int J Med Inform 1998;49(3):311–20.

58. Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16randomized controlled trials to evaluate computer-based clini-cal reminder systems for preventive care in the ambulatorysetting. J Am Med Inform Assoc 1996;3(6):399–409.

59. Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Typesof unintended consequences related to computerized providerorder entry. J Am Med Inform Assoc 2006;13(5):547–56.

60. Ammenwerth E, Schnell-Inderst P, Siebert U. Vision and Chal-
lenges of Evidence-Based Health Informatics: The case of a
CPOE meta-analysis. Invited submission to Int J Med Inform2008; under review.

61. Collen MF. General Requirements for a Medical InformationSystem (MIS). Comput Biomed Res 1970;3(5):393–406.

62. Brender J. Methodological and Methodical Perils and Pitfallswithin Assessment Studies Performed on IT-based solutions inHealthcare. Report. Aalborg: Virtual Centre for Health Infor-matics; 2002 August 2002. Report No.: ISSN 1397–9507.

63. Kuperman G, Bobb A, Payne T, et al. Medication-related clinicaldecision support in computerized provider order entry systems:a review. J Am Med Inform Assoc 2007;14(1):29–40.

64. Beuscart-Zephir M, Pelayo S, Anceaux F, Meaux J, Degroisse M,Degoulet P. Impact of CPOE on doctor-nurse cooperation forthe medication ordering and administration process. Int J MedInform 2005;74:629–41.

65. Scanlon M. Computer physician order entry and the real world:we’re only humans. Jt Comm J Qual Saf 2004;30(6):342–6.

66. Ammenwerth E, Talmon J, Ash JS, et al. Impact of CPOE onmortality rates–contradictory findings, important messages.Methods Inf Med 2006;45(6):586–93.

67. Eslami S, Abu-Hanna A, de Keizer NF, de Jonge E. Errorsassociated with applying decision support by suggesting de-
fault doses for aminoglycosides. Drug Saf 2006;29(9):803–9.


oaded from

http://www.strobe-statement.org/Checklist.html

http://iig.umit.at/efmi/starehi.htm


Documents

The Effect of Electronic Prescribing on Medication Errors and Adverse Drug Events: A Systematic Review