Quantitative Evaluation of Drug Name Safety Using Close-to-Reality Simulated

Pharmacy Practice

Sean Hennessy, PharmD, PhDAssistant Professor of Epidemiology & Pharmacology

Center for Clinical Epidemiology and BiostatisticsUniversity of Pennsylvania School of Medicine

CCEB

Outline• Big-picture view of drug name

evaluation• Improving the process by making

it quantitative• Model for measurement in mock

pharmacy setting• Research agenda

NameQualitative Evaluation

Process

Outcome•accept•reject

A Big-Picture View of Drug Name Evaluation

Outcome•accept•reject

A Big-Picture View of Drug Name Evaluation

Qualitative Evaluation

Process

NameQualitative Evaluation

Process

Outcome•accept•reject

A Big-Picture View of Drug Name Evaluation

Quantitative

Outline• Big-picture view of drug name

evaluation• Improving the process by making

it quantitative• Model for measurement in mock

pharmacy setting• Research agenda

Advantages of Quantitative Approach

• Explicit and systematic

• Uses fuller range of information

• Transparency of data & assumptions

• Acknowledges uncertainty

• Identifies knowledge gaps

NameQuantitative Evaluation

Process

Outcome•accept•reject

A Big-Picture View of Drug Name Evaluation

What underlies this binary

(yes/no) decision?

Safe name Unsafe name

Rating

NameQuantitative Evaluation

Process

Rating• probability of error

Outcome•accept•reject

A Big-Picture View of Drug Name Evaluation

Is this enough?

Are All Medication Errors Created Equal?

99%

1%

No observable ADE

Observable ADE

Bates DW. Drug Safety 1996;15:303-10.

Are These Equally Bad?

• erythromycin clarithromycin

• chloramphenicol chlorambucil

NameQuantitative Evaluation

Process

Rating• probability of error• consequences of

error•probability of AE

Outcome•accept•reject

A Big-Picture View of Drug Name Evaluation

Probability of Adverse Event

• Includes adverse outcomes from not getting intended drug

– From placebo-controlled trials

• ADE depends on identity of drug that is mistakenly substituted

– Measured empirically, as discussed later

• Frequency of ADEs in recipients of mistakenly substituted drug

– From pharmacoepidemiologic studies

NameQuantitative Evaluation

Process

Rating• probability of error• consequences of

error•probability of AE•disutility of AE

Outcome•accept•reject

A Big-Picture View of Drug Name Evaluation

Disutility

•The value of avoiding a particular health state, usually expressed on a scale from 0 to 1

•Measured empirically by asking patients standardized questions

Disutility of Outcomes for Occult Bacteremia

• Blood draw 0.0026

• Hospitalization 0.0079

• Meningitis recovery 0.0232

• Deafness 0.1379

• Minor brain damage 0.2607

• Severe brain damage 0.6097

• Death 0.9823

Benett JE, et al. Arch Ped & Adoles Med 2000;154:43-48.

A Possible Quantitative Rating

Perror Consequenceserror

=Perror PAEerror Disutility of AE

Probability of error

Co

nse

qu

ence

s o

f er

ror Rating

NameQuantitative Evaluation

Process

Rating• probability of error• consequences of

error•probability of AE•disutility of AE

Outcome•accept•reject

A Big-Picture View of Drug Name Evaluation

What settings?• outpatient pharmacy• inpatient pharmacy• physician office• inpatient unit• nursing administration • patient home administration• etc.

Outline• Big-picture view of drug name

evaluation• Improving the process by making

it quantitative• Model for measurement in mock

pharmacy setting• Research agenda

Potential Model for Name Evaluation: Mock Pharmacy Practice

NameQuantitative Evaluation

Process

Rating• probability of error• consequences of

error•probability of AE•disutility of AE

Outcome•accept•reject

A Big-Picture View of Drug Name Evaluation

Close-to-RealitySimulated Pharmacy Practice

• New or existing simulated pharmacies

• Use per diem practicing pharmacists or late-year pharmacy students

– Cost vs. realism

• List test drugs in computerized drug info source

• List test drugs in prescription entry program

• Put test drugs on pharmacy shelf

Pharmacy Practice Lab for Testing Drug Names

• Simulate pharmacy practice by presenting Rx’s (phone, hand-written, computer-generated) for real and test drug

• Add Rx volume, noise, interruptions, 3rd party reimbursement issues, Muzak, etc.

• Pharmacist enters and fills prescription• Measure the rate of name mix-ups at all stages

of filling process, and which drug was mistakenly substituted

Getting from EvaluationRating

• For probability of error, use point estimate or upper confidence limit (CL)?

Maximum value statistically compatible with data; function of measured rate & sample size

Getting from EvaluationRating

• For probability of error, use point estimate or upper confidence limit (CL)?–Using upper CL encourages bigger studies

–What coverage for CLs? (95%? 90%? 80%?)

»Base on what seems reasonable using real data?

Potential Advantages vs. Expert Opinion

• Yields empiric estimates of error rate, and of which drugs are mistakenly substituted

• Better face validity• Validity can be tested by examining

known “bad” names• Makes knowledge & assumptions

explicit

Obstacles & Limitations-1• Hawthorne effect?

–Initial improvement in a process of production caused by the obtrusive observation of that process

• Technical challenges

Obstacles & Limitations-2• Need large sample sizes

• Use routinely, or just to validate qualitative approaches?

• Worth the added cost?

Outline• Big-picture view of drug name

evaluation• Improving the process by making

it quantitative• Model for measurement in mock

pharmacy setting• Research agenda

Research Agenda

• Feasibility

• Cost

• Reliability

• Validity

• Utility