Upload
cuthbert-griffith
View
217
Download
1
Tags:
Embed Size (px)
Citation preview
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Ida Sim, MD, PhD
February 28, 2006
Division of General Internal Medicine, and Program in Biological and Medical Informatics
UCSF
Electronic Health Records for Clinical Research (cont.)
Copyright Ida Sim, 2006. All federal and state rights reserved for all original material presented in this course through any medium, including lecture or print.
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
EHR for Research Summary
• An EHR is not automatically going to help clinical research– if all unstructured free text, won’t help much at all
• the more structured it is (ie more defined fields), the better
– if just coded sporadically in ICD-9• problem with gamed codes• poor coverage of many clinical concepts
– if coded in SNOMED• some clinical concepts still not well covered
• EHR better than chart review; can we do even better?
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Capturing Clinical Data: Now
Recode Patient Data
Financial Database
Collect Patient Data
Clinical Repository
Collect Patient Data
Research Database
ICD CDEFree text
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Capturing Clinical Data: Wish
Recode Patient Data
Financial Database
Collect Patient Data
Clinical Repository
Collect Patient Data
Research Database
SNOMED-CT
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
MICU
FinanceResearch
QA
Clinical Repository
Internet
ADT Chem EHR XRay PMB Claims
• Integrated historical data common to entire enterprise
What is a Clinical Data Repository?
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Types of Queries
• Clinical care• What was Mr. Smith’s last
potassium?• Does he have an old CXR
for comparison?• What antihypertensives
has he been on before?• What did the neurology
consult say about his epilepsy?
• Research• What % of diabetics with
AMI admissions were discharged on -blockers?
• What was the average Medicine length of stay in 2004 compared to 2000?
• What is the trend in use of head CTs in patients with migraine?
• Is admission creatinine independent predictor of bacteremia outcomes?
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Types of Data Repositories
• A data repository (aka data warehouse) is just a collection of data from other databases– is itself just a database
• Two somewhat distinct types– clinical data repository
• collects data from day-to-day clinical care, admin data, etc.• for quality improvement, outcomes research, business decision
making…– research data repository
• collects data from multiple research projects• may also collect data from day-to-day clinical care, admin data, etc.
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Data Repositories: Hope• Touted for
– business decision making– health care quality improvement– outcomes research– geno-pheno correlations for translational research
• UCSF planned “Research Data Infrastructure and Services” – goal: a single clinical and research data repository
• care data from STOR, radiology, UCare etc. • research data from all UCSF research projects
– to enable correlation of clinical, genomic, imaging, etc data across data sets
– one query across all systems -- great!
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
• Garbage in = garbage out• Incompatible data in = garbage out
– query in one system would return meaningful results from another (interoperable)
– requires• interoperable data schemas
– type (e.g., relational)– data modeling (i.e., column names)
• common naming of data items– eg., “PNA” vs. “pneumonia”
Or Hype?
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
UCSF “RDIS” Example
• Standard clinical vocabulary? data representation model?• Are queries mostly within or across projects? ongoing or completed
projects or both?• Need administrative data (e.g., insurance)?
(in SNOMED-CT) xrays/CT/MRImicroarray data(in MAGE-OM) (in DICOM)
•Breast CA (not DCIS)•Menopause
•Osteoporosis (Heel US)•Menopause
Project 1
DB 1
Project 2
DB 2
Project 3
DB 3
Project 4
DB 4
•Osteoporosis (DXA)•Menopause
•Breast CA (DCIS ok)•Alzheimers (path)
RDIS
Data mining/Display ToolsRadiologySTOR
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Repurposing for QI and Research?
Financial Database
Collect Patient Data
Clinical Repository
Research Database
Clinical Quality Improvement
?
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Data Repository Summary
• Enterprise viewpoint more appropriate for research than patient viewpoint of EHR
• Integrates data from multiple sources– need standardization of codes, definitions, and data
schemas• Querying and processing occurs “offline”
– little impact on real-time clinical care• Repository must be designed for anticipated uses
– can single repository serve clinical and research needs?
Viewpoint Time Queries
EHR Patient Real-Time ClinicalData Repository Enterprise Historical Ad Hoc
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
• EHR does not always = easier clinical research• Structure and coding is critical
– structure: e.g., relational schema, designed to support intended queries
– standard formats needed for genomic, imaging, etc. data– coding: standardized, coded data trumps free text
• especially important for research• but most controlled vocabularies have insufficient clinical coverage
– controlled vocabularies (e.g., SNOMED-CT) are hard to design and hard to use
• Clinical/Research data repositories must be designed “correctly” with high-quality, cross-compatible data
Take-Home Points
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Decision Support Systems
Ida Sim, MD, PhD
February 28, 2006
Division of General Internal Medicine, and the Program in Biological and Medical Informatics
UCSF
Copyright Ida Sim, 2006. All federal and state rights reserved for all original material presented in this course through any medium, including lecture or print.
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Outline
• Decision support systems– background, definition– clinical versus research decision support
• How decision support systems “think”• Effectiveness
– improving quality– reducing errors
• Implications
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Decision Support for Quality
• Endemic quality problems– “Health care today harms too frequently and routinely
fails to deliver its potential benefits… Between the health care we have and the care we could have lies not just a gap, but a chasm.” (IOM, 2001)
• Evidence-based practice is means to quality– practice based on currently best available evidence from
clinical research– hardest way to practice, logistically impossible
• > 1,000 guidelines in National Guideline Clearinghouse• > 4,600 journals indexed in Medline
– over 10,000 RCTs per year– over 2700 systematic reviews per year
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Informatics to the Rescue
• Point-of-care decision support– Medline on every desktop
• and “beyond Medline”…
– reminders (e.g., preventive care)– guideline-based recommendations
• e.g., when to prescribe antibiotics for sore throat
• Diagnostic support– how not to miss anthrax
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Clinical Decision Support
• Clinical decision support system (CDSS)– software that is designed to be a direct aid to clinical
decision-making – in which the characteristics of an individual patient are
matched to a computerized clinical knowledge base– and patient-specific assessments or recommendations
are then presented to the clinician and/or the patient for a decision (Sim et al, JAMIA, 2001)
• Examples of clinical decisions to be supported?
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Major Target Tasks of CDSSs• Diagnostic support
– DxPlain, QMR• Drug dosing
– aminoglycoside, theophylline, warfarin• Preventive care
– reminders for vaccinations, mammograms• Disease management
– diabetes, hypertension, AIDS, asthma• Test ordering, drug prescription
– reducing daily CBCs in hospital, drug allergy checking• Utilization
– referral management, clinic followup
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
What Isn’t a CDSS
• Medline• UpToDate• Static guideline repositories
– www.guideline.gov (National Guideline Clearinghouse)
• Online laboratory data, test results, chart notes• Retrospective quality improvement reports
– how your vaccination rates compare to your colleagues’
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Research Decision Support
• Research decision support system– software that is designed to be a direct aid to ???
decision-making – in which the characteristics of an individual ??? are
matched to a computerized ??? l knowledge base– and ??? -specific assessments or recommendations are
then presented to the ??? for a decision
• Examples of research decisions to be supported?– determining eligibility– what to do when (if WBC<2 then hold Drug)– reporting adverse events, etc.
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Similarities/Differences
• customized to patient• identify applicable
guidelines, evidence• variable presentations and
contexts• wide clinical coverage• may include diagnostic
support• involves many team
members• one locale
• uniform treatment • identify applicable patients• narrower range of
presentations/contexts• narrower clinical coverage• more procedural, less
diagnostic support• smaller defined, more
uniform target staff• could be in multiple sites• more controlled
circumstances, regulatory overlay
Research Decision SupportClinical Decision Support
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Outline
• Decision support systems– background, definition– clinical versus research decision support
• How decision support systems “think”• Effectiveness
– improving quality– reducing errors
• Implications
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Basic Decision Support Task
• Decision– action that consumes resources in the real world
• Decision support– given starting conditions and a defined set of action
choices, recommend or rank action choices for user
• Requires some “thinking” to recommend or rank– strictly deterministic thinking– thinking with fuzziness and probabilistic features
• in starting data or reasoning procedure• outcomes (e.g. prob. of adverse reaction)
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Decision Support “Thinking”• Strictly deterministic
– first-order logic rule-based systems– adhoc rule-based systems (non-mathemetical reasoning about
probability)• e.g., if high WBC AND cough AND fever AND abn. CXR then
likelihood of pneumonia is 4 out of 5
• Probabilistic/fuzzy– bayesian networks
• formal probabilistic reasoning, extension of decision analysis
– neural networks– fuzzy logic, genetic algorithms, case-based reasoning, etc., or
hybrids of these
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Rule-Based Decision Support (1)
• Forward chaining/reasoning (data-driven)– start with data, execute applicable rules, see if new
conclusions trigger other rules, and so on– example
• if HIGH-WBC and COUGH and FEVER and ABN-CXR => PNEUMONIA
• if PNEUMONIA => GIVE-ANTIBIOTICS• if GIVE-ANTIBIOTICS => CHECK-ALLERGIES• if PNEUMONIA and GIVE-ANTIBIOTICS and NOT
(ALLERGIC-DOXYCYCLINE) => GIVE-DOXYCYCLINE
– use if sparse data
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Rule-Based Decision Support (2)
• Backward chaining/reasoning (goal-driven) – start with “goal rule,” determine whether goal rule is
true by evaluating the truth of each necessary premise– example
• patient with lots of findings and symptoms• is this SLE? => are 4 or more ACR criteria satisfied?
– malar rash?– discoid rash?– skin photosensitivity? etc
• if SLE => ...– use if lots of data
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Problems with Rule-Based DSSs• Combinatorial explosion of rules
– need rule for each contingency• if MOD-WBC and COUGH and FEVER and ABN-CXR =>
PNEUMONIA
• Rules may be contradictory– if COUGH and ABN-CXR => INTERSTITIAL-LUNG-DZ
• Rules may be circular• Need knowledge engineering and clinical
expertise to build and maintain the knowledge base over time– need to keep rules up-to-date with latest evidence
• Could rules be centrally defined and then shared?
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Medical Logic Modules (MLMs)
• help_amp_for_pneumonia - Ampicillin for Pneumonia
• maintenance:– title: Ampicillin for
Pneumonia;;– filename:
help_amp_for_pneumonia;; – version: 1.00;; – institution: LDS Hospital;; – author: Peter Haug, M.D.;
George Hripcsak, M.D.;; – specialist: ;; – date: 1991-05-28;;
• validation: testing;; • library:
– purpose: Recommend the use of ampicillin for pneumonia.;;
– explanation: If the patient has pneumonia, then suggest treatment with ampicillin unless there is a penicillin allergy.;;
• keywords: pneumonia; penicillin; ampicillin;;
• citations: 1. HELP Frame Manual, version 1.6. LDS
Hospital, August 1989, p.81.;;
• For sharing forward chaining rules • Expressed in Arden Syntax (an international ASTM standard)
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Sharing of MLMs: No Success• Work of reuse often greater than building from
scratch– rules are often outdated: need to check evidence base– context is under-specified
• is pneumonia rule inpatient or outpatient? in HIV patients?
– can be wrong for local context• resistance patterns vary in different locales
– definitional problems• your “pneumonia” is not my “pneumonia”
– curly braces problem• if {K+} > 5.5 => alert MD• how to access the value of K+ automatically? requires interfacing
to lab system which differs from place to place
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Rule-based DSSs for Research
• More promising than for clinical decision support– usually narrow context (e.g., single study)
• small, explicitly defined, stable, uncontroversial rule base
– definitional problems resolved as part of study design– clinical context standardized or taken into account
• studies on inpatient vs. outpatient pneumonia
– trained, dedicated staff sharing same objective• e.g., is clinical guideline for saving $ or saving lives?
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Decision Support “Thinking”• Strictly deterministic
– rule-based systems– adhoc (non-mathemetical reasoning about probability)
• e.g., if high WBC AND cough AND fever AND abn. CXR then likelihood of pneumonia is 4 out of 5
• Probabilistic/fuzzy– bayesian networks
• formal probabilistic reasoning, extension of decision analysis
– neural networks– fuzzy logic, genetic algorithms, case-based
reasoning, etc., or hybrids of these
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Neural Networks• Finds a non-linear relationship between input parameters and
output state• Structure of network
– usually input, output, and 1-2 hidden fully connected layers
– each connection has a “weight”
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Neural Network for MI Diagnosis
• Inputs (e.g., all patient characteristics in the EHR) • EKG findings (ST elevation, old Q’s)• rales (Yes, No)• JVD (in cm)
• Outputs are the set of possible outcomes/diagnoses
EKG findings
Rales
JVD
Response to TNG
Acute MI
No Acute MI
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Training the Neural Network• Network gets “trained”
– feed network many examples of known patients and their diagnoses
• can handle missing data
– system iteratively adjusts the weights of the connections to find the network “pattern” that associates sets of input variables (patients) with the right output state (MI or not)
• Test network’s accuracy on another set of patients• In Baxt’s MI neural network
– training set: 130 pts with MI, 120 without– test set: 1070 UCSD ER patients with anterior chest pain
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Baxt’s Acute MI Neural Net• Evaluation results: prevalence of MI 7% (Lancet, 1996)
• Results were driven by non-standard predictors– rales, jugular venous distention
• Why isn’t this neural network used more widely?– “black box” nature limits explanatory ability and lessens
acceptance– users have to input the variables manually
• interfacing to EHRs would increase adoption of DSSs
Sensitivity Specificity
Physicians 73.3% (63.3-83.3) 81.1% (78.7 – 83.5)Neural Net 96.0% (91.2 – 100) 96.0% (94.8 – 97.2)
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Probabilistic Decision Support for Research
• Mostly used for eligibility determination– more adaptable than rule-based systems
• variable clinical presentations require many rules to handle contingencies
• what if data not available? fire a rule or not?
• Bayesian networks, neural networks, etc.– can handle missing data– adaptable to unanticipated presentations
• Several prototype Bayesian eligibility systems in the works…
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Decision Support Methods Summary
• General limitations to clinical decision support– lack of formal, reproducible methods for making
clinical decisions – how to represent qualitative data (e.g., “looks sick”)
• Vast majority of clinically-used DSSs are rule-based systems, limited by– combinatorial explosion of rules
• Probabilistic approaches more “forgiving”, more “realistic”– can be computationally intractable
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Outline
• Decision support systems– background, definition– clinical versus research decision support
• How decision support systems “think”• Effectiveness
– improving quality– reducing errors
• Implications
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Systematic Review of CDSS RCTs
• Only occasional modest benefit found (Hunt, JAMA 1998; updated RB Haynes 2000)– diagnosis: 1/5 studies beneficial– drug dosing: 9/15– preventive care reminders: 19/26
few studies looked at patient outcomes• 6 of 14 showed benefit
• Counted the number of systems in each category (e.g., drug dosing) that were “effective” (p>0.05)– but CDSS not all the same (apples and oranges)
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Heterogeneity of CDSSs
• Preventive care reminder CDSS– A clerk routinely abstracts preventive care interventions
from paper chart into a database. Before each clinic session, nurse runs the CDSS for patients coming in that day. Guideline-based recommendations are printed out on paper and attached to front of chart. Doctor orders preventive care in usual way using paper-based methods.
• Hypertension treatment CDSS– Clinic has an EMR. During patient visit, CDSS notes that BP
and trend is too high. It checks patient’s Cr, diabetes status, cardiac status, current meds and allergies and recommends drug therapy change according to JNC VI guidelines. Presents e-prescription for MD to verify. If verified, order is sent directly to pharmacy and medication list updated.
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
CDSS Characteristics: Highlights• Reviewed and classified 42 RCT-evaluated systems• Tremendous variation in decision-maker/context,
how recommendation delivered, staff needed to make system run, complexity of recommended actions– 45% targeted to clinician, 55% patient, 5% both– 62% based on national guidelines or literature– 69% “pushed” recommendation to decision maker– 43% collected data directly from the EHR
• 45% required data input intermediary (11% MD), 58% of time requiring clinical knowledge
– 26% required an output intermediary
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
CDSS for Quality Summary• Current data suggests CDSSs can improve the
process of care and perhaps clinical outcomes– most effective at preventive care reminders– modest at best for drug dosing and active care– generally not helpful for improving diagnosis except
with trainees• Findings limited by
– methodological problems and design type of studies– insufficient appreciation of workflow component of
CDSSs– insufficient appreciation of heterogeneity of systems
• Bottom line: equivocal evidence, limited use
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Outline
• Decision support systems– background, definition– clinical versus research decision support
• How decision support systems “think”• Effectiveness
– improving quality– reducing errors
• Implications
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Fundamental Barrier
• Better quality care <-- better decision support• Better decision support <-- “smarter” systems
– “know” more about the patient, evidence, context
• “Smarter” systems <-- more richly coded data– for EHR: SNOMED, standard EMR structure– for knowledge: coding, structures for guidelines, RCTs…
• Coded data <-- Coding of data entry• Coding of data entry <-- Greater physician time• Greater physician time --> no play --> no gain
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Implications for Quality Improvement
• Clear trade-off between physician coding effort and “smarter” decision support
• Don’t expect more decision support than coding supports– generally successful decision support
• preventive care: age, last mammogram, etc.• allergies: Yes/No on specific drugs• drug dosing: weight, height, creatinine, age
– generally unsuccessful decision support• diagnostic assistance• complicated therapies (e.g., management of hypertension)
• Unrealistic to think of CDSSs as improving evidence-based practice in general
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Implications for Research
• DSS possible with paper-based “charts”– narrow focus, well-defined rules, could help
standardize treatment by protocols– BUT requires double-data entry, workflow hassles
• DSS with EHR– ideal to use routinely collected EHR data and/or a
module “plugged in” to EHR– BUT requires interoperability of DSS with the EHR
• e.g., trial randomizing pts. to metformin or pioglitazone• exclusion rule #3: history of congestive heart failure
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
DSS Interoperability with EHR
itsa
coordcenter with DSS
ucsf.edu
LAN
Site 1/GE EHR Site 2/Epic EHR
1. Is C. Jones eligible for this trial?
2. …Exclusion Rule #3: Does C. Jones have a history of congestive heart failure?
3. Return Yes if “congestive heart failure” is in Past Medical History
HL-7 communications protocol
4. If Yes to history of CHF, C. Jones is not eligible
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Summary on Decision Support
• Clinical care and research “decisions” similar• Different methods useful for different types of
decisions– using deterministic versus “fuzzy/probabilistic” reasoning
• Equivocal evidence for improving quality– limited by methodological and other shortcomings– workflow and organizational inputs generally
underappreciated• Fundamental trade-off between
– effort of coded data and quality (e.g., nuance) of decision support, and
– acceptability, effectiveness
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Barriers• Near-term for clinical care
– knowing which decisions to support (e.g., preventive care reminders)
– data exchange among legacy systems– integrating decision support smoothly into EHR, incentives for
quality, etc. (3/15 class)• Near-term for research
– paper-based systems, requiring double-data entry• Longer term
– standard clinical vocabulary with adequate semantic coverage– much wider use of EHRs– efficient coding: of what, by whom, when, why– interoperation of EHRs with decision support systems– more explicit decision-making strategies for clinical care
February 28, 2006: I. Sim Decision Support SystemsMedical Informatics – Epi 206
Teaching Points• CDSSs have great promise for improving
quality/reducing error– but promise essentially not yet realized
• Much can be done today but only in limited settings
• “HAL”-like artificial intelligence not the main barrier
• Greater decision support for care and research requires– wider EHR use– richer, standard clinical vocabulary – better interoperability