Upload
richard-boyce-phd
View
294
Download
1
Embed Size (px)
DESCRIPTION
A talk on methods for representing potential drug-drug interactions (PDDIs) so that they can be more useful for clinical decision support. The talk has three sections: 1) a novel "evidential" approach to evaluating the validity of PDDIs, 2) a discussion of what information is necessary to make a judgement on the clinical relevance of a PDDI exposure, and 3) a proposal to use predictive risk models to actively monitor of the risk of specific adverse events during a PDDI exposure. This talk was given at the AHRQ-sponsored "Generating, Evaluating, and Implementing Evidence for Drug-Drug Interactions in Health Information Technology to Improve Patient Safety: A Multi-Stakeholder Conference"
Citation preview
Page 1
Effective communication of Drug-drug interaction (DDI) knowledge
Richard Boyce, PhDPostdoctoral Fellow in Biomedical Informatics
University of Pittsburgh
Page 2
Objectives
1. Present a new approach to representing and makingpredictions with drug mechanism knowledge
2. Identify core knowledge elements for DDI decision support and suggest the possibility of a common model for representing and sharing DDI knowledge
3. Suggest how further research on clinical trigger systems could lead to reduced DDI alert fatigue while improving patient safety
Page 3
Objective 1: Present a new approach to representing and makingpredictions with drug mechanism knowledge
Page 4
A large number of potential interactions can be inferred based on mechanisms
pre-clinical and pre-market post-market in vitro and in vivo
For example, metabolic interactions:
Page 5
Three informatics challenges to reasoning with drug mechanisms1
Drug mechanism knowledge is dynamic, sometimes missing, and often uncertain
Hypothesis:A computable model of drug-mechanism evidence will enable powerful methods for addressing these issues.
[1] Richard Boyce, Carol Collins, John Horn, and Ira Kalet. Modeling Drug Mechanism Knowledge Using Evidence and Truth Maintenance. IEEE Transactions on Information Technology in Biomedicine, 11(4):386-397, 2007.
Page 6
Case study: The Drug Interaction Knowledge Base (DIKB)
The DIKB:infers drug-mechanism knowledge from a computable representation of evidencemakes interaction and non-interaction predictions with measurable performance characteristics
Current focus:DDIs that occur by metabolic inhibition
Page 7
The DIKB architecture
Page 8
The reasoning system's current DDI prediction model1
Uses facts in the knowledge-base that meet explicitbelief criteria
Provides a "ball-park" magnitude estimatebased on the importance of the affected metabolic pathway to the object drug
Predicts enzyme-specific non-interactionsenables the suggestion of alternative drugs
[1] Richard Boyce. An Evidential Knowledge Representation for Drug-mechanisms and its Application to Drug Safety. PhD thesis, University of Washington, August 2008.
Page 9
The DIKB is an evidential system
All assertions are linked to their evidence for or against:
Evidence againstEvidence for
(FLUCONAZOLE inhibits CYP2C9)
helps assess credibility, draw attention to errors, and establish confidence
Page 10
All assumptions are linked to evidence
Enables the system to identify when assumptions are no longer validHelps identify poor uses of evidence
Page 11
All evidence is classified using a study taxonomy
A DDI clinical trial...randomized ...non-randomized
...A non-traceable statement
...in drug product labelling...
A metabolic enzyme inhibition experiment...focusing on CYP450 enzymes
...done in human liver microsomes...
Oriented toward user confidence assignment36 types over several sub-hierarchies
Page 12
Basic quality standards are defined for every evidence type
Example: Pharmacokinetic clinical trialsadequate duration and magnitude of dosing patient genotype/phenotype is noted if the target enzyme is polymorphic
Example: in vitro enzyme metabolism identificationhuman hepatocyte or recombinant CYP450 'selective' inhibitors and 'probe' substrates
Page 13
This DIKB's architecture helps identify evidence use strategies
What evidence is sufficient to justify that a drug possesses certain mechanistic properties in vivo?
What evidence-use strategy enables the system to make the best predictions?
How do these predictions compare with the most rigorous treatment of evidence possible?
Page 14
An experiment focusing on statins1
Study overview:Collect evidence on statins and a sub-set of co-
prescribed drugs (16 drugs, 19 metabolites)Make predictions using a “belief criteria strategy”
designed by two drug expertsExplore computer-generated strategies to find the ones that were “best-performing”Examine the feasibility of novel DDI predictions
[1] R. Boyce, C. Collins, J. Horn, and I. Kalet. Computing with evidence part II: An evidential approach to predicting metabolic drug-drug interactions. J Biomed Inform,2009. (2009) DOI:10.1016/j.jbi.2009.05.010.
Page 15
An experiment focusing on statins1
Study overview:
[1] R. Boyce, C. Collins, J. Horn, and I. Kalet. Computing with evidence part II: An evidential approach to predicting metabolic drug-drug interactions. J Biomed Inform,2009. (2009) DOI:10.1016/j.jbi.2009.05.010.
(16 drugs, 19 metabolites)
36,000 strategies
Page 16
Metricssensitivity, specificity, positive predictive value (PPV) accuracy = (true pos + true neg) / (all predictions)kappa
Page 17
The validation set
'unknown': a pair for which no interaction or non- interaction can be confirmed
drug pairs 586interactions 41 (7%)non-interactions 7 (1.2%)unknown 538 (91.8%)
Page 18
Test 1 – the DIKB's most rigorous evidence- use strategy possible
Sample of the two most critical assertions:D inhibits ENZ :
FOR: clinical trials with D and another drug known to selectively inhibit ENZ in vivoAGAINST: metabolic inhibition studies with probe substrates
D substrate-of ENZ
FOR/AGAINST: drug metabolism studies involving participants with known CYP450 phenotypes or genotypes
Page 19
Test 1 – results using the most rigorous criteria
Statistic Valuenumber of predictions 17True positive 14True negative 0novel 3Positive predictive value 1.00Sensitivity 1.00Specificity undefinedCoverage of validation set 14/49 = 29%
Page 20
Test 2 – computer-derived 'best' strategyLOE inhibits substrate-ofHigh in vitro study AND PK authoritative MedLow
Page 21
Test 2 – computer-derived 'best' strategyLOE inhibits substrate-ofHigh authoritative MedLow
authoritative or in vitro
Page 22
Test 2 – computer-derived 'best' strategyLOE inhibits substrate-ofHigh authoritative MedLow in vitro study
Page 23
Test 2 – computer-derived 'best' strategyLOE inhibits substrate-ofHigh in vitro study AND PKMed in vitro studyLow
Page 24
Test 2 – computer-derived 'best' strategyLOE inhibits substrate-ofHighMed in vitro studyLow
authoritative or in vitro
Page 25
Test 2 – computer-derived 'best' strategyLOE inhibits substrate-ofHighMed in vitro studyLow in vitro study
Page 26
Test 2 – better-performing strategies
8,351 (23%) of 36,000 tested strategiesEqual or better sensitivity, positive predictive value, and agreement than the expert-designed strategy
1,152 (3%) performed at the top level
Statistic Value Statistic Valuetrue positives 34 false positives 0true negatives 0 false negatives 0sensitivity 1.00 specificity n/a
PPV 1.00 kappa 0.52
Page 27
Test 2 – relaxing belief criteria often improved performance
more stringent less stringent
Page 28
Test 2 – relaxing belief criteria often improved performance
more stringent less stringent
n = 8
n=~1kn=~17k
n=~18k
Page 29
Test 2 Novel Predictions31 novel interaction predictions
13 matched to 15 published case reports 18 other non-refuted interaction predictions
Two interaction predictions supported by case reports that were not present in product labelling:
Interaction prediction Level Evidencediltiazem – atorvastatin > 2-foldfluconazole – simvastatin > 2-fold
2 case reports – possible1 case report – possible
Page 30
Conclusions from the pilot experiment
The new approach to representing drug mechanism knowledge
can suggest the best use of available evidence for metabolic DDI predictionis less subjective than ranking evidence and selecting the most rigorous
This is interesting because drug mechanism evidence base varies dramatically across marketed drugs
Page 31
Objective 2: Identify core knowledge elements for DDI decision support and suggest the possibility of a common model for representing and sharing DDI knowledge
Page 32
Example - A possible observed DDI
Two case reports reporting on five individuals who developed symptoms of myopathy or rhabdomyolysis1,2
All cases provide some evidence that an adverse event (AE) was caused by this DDI
Interaction prediction Level Evidencediltiazem – atorvastatin > 2-fold 2 case reports – possible
[1] P. Gladding, H.
Pilmore, and C.
Edwards. Potentially fatal interaction between
diltiazem and statins. Ann Intern Med, 140(8):W31, 2004.
[2] J. J. Lewin 3rd, J. M. Nappi, and M. H. Taylor. Rhabdomyolysis with concurrent atorvastatin and diltiazem. Ann Pharmacother, 36(10):1546-1549, 2002.
Page 33
Assessing the example DDI
The DDI:is reasonable could lead to a serious or fatal adverse event
...but, we don't know:patient-specific risk factors prevalence of co-prescribing and various outcomes
Interaction prediction Level Evidencediltiazem – atorvastatin > 2-fold 2 case reports – possible
Page 34
Structured DDI assessmentA structured assessment scores evidence and potential severity1
[1] E. N. van Roon, S.
Flikweert, M.
le
Comte, P.
N. Langendijk, W.
J. Kwee-
Zuiderwijk, P. Smits, and J.
R. Brouwers. Clinical relevance of drug-drug interactions :
a structured assessment procedure. Drug Saf, 28(12):1131-1139, 2005.
Page 35
Evidence for/against a DDI
pre- and post-market studies
in vitro experimentsknown or theoretical mechanismscase reports and case seriespharmacovigilance
Page 36
Risk factors
patient characteristics X potential adverse eventpatient characteristics X DDI mechanismdrug characteristics
route of administration, dose, timing, sequence
Page 37
Risk factors depend on evidence
Page 38
Incidence
prevalence of co-prescriptionprevalence of AEincidence of AE in exposed and non-exposed
Page 39
Incidence and evidence strengthen each- other
Page 40
Seriousness of the AE
Classified by specific clinical outcome...but, can any seriousness ranking be generally accepted?
no effect death?
Page 41
Re-assessing the example DDI
drug pair diltiazem – atorvastatinpotential AE rhabdomyolysis
risk factors
evidence
prevalance of pair commongeneral incidence of AE rareincidence in exposed ...incidence in non-exposed ...
other HMG-CoA, inhibitors, phenotype, ...five possible cases, established mechanism
Page 42
Structured assessments vary
focus and content methods for ranking severityacross compendia, vendors1, and implementations2
[1] F.D. Min, B. Smyth, N. Berry, H. Lee, and B.C. Knollmann. Critical evaluation of hand-held electronic prescribing guides for physicians. In American Society for Clinical Pharmacology and Therapeutics, volume 75. 2004.
[2] Thomas Hazlet, Todd A. Lee, Phillip Hansten, and John R. Horn. Performance of community pharmacy drug interaction software. J Am Pharm Assoc, 41(2):200-204, 2001.
Page 43
Agreement on common elements might be possible
drug pairpotential AErisk factorsevidenceprevalance of pairgeneral incidence of AEincidence in exposedincidence in non-exposed
...
...and could form the basis for a sharing DDI knowledge across resources
Page 44
Objective 3: Suggest how further research on clinical trigger
systems might lead to reduced DDI alert fatigue while improving patient safety
Page 45
Results from a recent review on medication alerting1
“Adverse events were observed in 2.3%, 2.5%, and 6% of the overridden alerts, respectively, in studies with override rates of 57%, 90%, and 80%.”“The most important reason for overriding was alert fatigue caused by poor signal-to-noise ratio”Only one study looked at error reductions for DDI alerts –
the results were statistically non-significant
[1] H. van
der Sijs, J.
Aarts, A.
Vulto, and M.
Berg. Overriding of drug safety alerts
in computerized physician order entry. J Am Med Inform Assoc, 13(2):138-147, 2006.
Page 46
What does the literature suggest?
find a balance between push vs. pull alerts tier DDI alerts by severity give users the ability to set preferences for some types of alerts provide value e.g. changing meds or correcting the medical record from the alert make alert systems more intelligent
Page 47
A potential complementary approach
Clinical event monitor -a system that identifies and flags clinical data indicative of a potentially risky patient state
Page 48
Example – UPMC MARS-AiDE
Page 49
Example triggers from UPMC MARS-AiDE1
signal alerts ADRs PPV
2 1 0.5
Vitamin K given; taking warfarin 4 4 1
2 2 1
5 5 1
Na+ polystyrene given; taking a drug that may cause hyperkalemia
Clostridium difficile toxin positive; taking a drug that may cause pseudomembranous colitisElevated alanine aminotransferase concentration; taking a drug that may cause/worsen hepatocellular toxicity
[1] S. M. Handler, J. T. Hanlon, S. Perera, M. I. Saul, D. B. Fridsma, S. Visweswaran, S. A. Studenski, Y. F. Roumani, N. G. Castle, D. A. Nace, and M. J. Becich. Assessing the performance characteristics of signals used by a clinical event monitor to detect adverse drug reactions in the nursing home. AMIA Annu Symp Proc, pages 278-282, 2008.
Page 50
DDI-aware clinical event monitoring
Page 51
Potential benefits and risks of DDI-aware clinical event monitoring
Benefitsautomatic consideration of patient-specific risk factorsa possible safety net for some 'potential DDI' alerts may provide implicit management options (e.g. stop/change interacting drug)
Riskspotential for additional alert burdenis it ethical to not alert prescribers?...
Page 52
Conclusions
We've looked briefly at three areas of research that aim to make more effective use of DDI knowledge in clinical care:
representing and using evidence for predicting DDIsDDI knowledge sharing integrating DDI knowledge with clinical event
monitoring
Page 53
AcknowledgementsThanks to AHRQ for sponsoring this conferenceAdvisors/mentors: Steve Handler, Ira Kalet, Carol
Collins, John Horn, Tom Hazlet, Joe Hanlon, Roger Day
Funding:University of Pittsburgh Department of Biomedical InformaticsNIH grant T15 LM07442Elmer M. Plein
Endowment Research Fund from the
UW School of Pharmacy University of Pittsburgh Institute on Aging