Using the Explorys Platform for Clinical Effectiveness Research (CER) with De-identified, Population Level Data
David C Kaelber, MD, PhD, MPH, FAAP, FACPAssociate Professor of Internal Medicine, Pediatrics, Epidemiology, and Biostatistics
Director of the Center for Clinical Informatics Research and EducationChief Medical Informatics Officer
The MetroHealth SystemCase Clinical and Translational Science Center (CTSC)
Case Western Reserve University
Disclosures
• I receive no compensation from Epic, although tens of millions of dollars of institutional funds and my academic career are committed to Epic .
• I have no financial relationship with Explorys, Inc. The MetroHealth System was one of the first Explorys, Inc. partners and contributes all of its electronic health record data in exchange for use of the Explorys Explore tool. And Explorys, Inc. seems to be helping my academic career .
Case 1
• Relationship between weight, height, and blood clots (venous thromboembolic events)
• Not CER Example
Patient Characteristics association with Venous Thromboembolic Events (VTEs) – A Cohort Study using Pooled Electronic Health Record (EHR) data
Kaelber, et al, JAMIA, e-published 3 July 2012
• 959,030 patients (vs 26,714 -> ~40 times more)• 21,210 VTE patients (vs 451 -> ~50 times more)• 12 year retrospective study (vs 14 years)• ~2 months from idea to submission (vs 18 years)
Similar results with much higher power!Not human studies research (No PHI; No IRB)!
Kaelber, et al, JAMIA, e-published 3 July 2012
Case 2
• Post-market surveillance of Azathioprine– Anti tumor necrosis factor medication
• CER Example
Azathioprine - A case study using pooled electronic health record data and co-morbidity networks for
post-market drug surveillance
Manuscript submitted and under review
Study Design
• Design: A “prospective” cohort study (from a retrospective cohort).
• Setting: Explorys network of ~11 million patients (at the time of the study).
• Patients: All patients in the Explorys network who were prescribed Azathioprine (AZA) and/or similar medication(s).
• Main Outcome Measures: Side effects from AZA (and how side effects compare to other similar drugs).
Side Effects Investigates
Side Effect Lab Value Abnormal RangeAnemia Hemoglobin (Hgb) <11 g/dLCell lysis Lactate dehydrogenase (LDH) >190 IU/LFever Temperature >37.8oFHepatotoxicity AST, ALT AST>40 IU/L and ALT>40 IU/LHepatotoxicity Total bilirubin (Bili) >1 mg/dLHypertension Blood pressure (BP) Systolic >140 mm Hg
or Diastolic>90 mm HgNephrotoxicity Creatinine (Cr) >1.5 mg/dLNeutropenia Neutrophil count Count<57% or <2.5 cells/µlNeutrophilia Neutrophil count Count>70%
ResultsControl cohort administered one of 12 anti-rheumatic drugs. Overlap is evident between the cohorts since controlling the
AZA cohort for the absence of the other 12 drug. Drug Name (RxCUI) Control Cohort AZA Cohort
Abatacept (614391) 140 (0.1%) 60 (0.4%)Adalimumab (327361) 2660 (2.1%) 650 (4.7%)
Azathioprine (1256) 3610 (2.8%) 13890 (100.0%)Clioquinol (5942) 110 (0.1%) 0 (0.0%)
Etanercept (214555) 2490 (1.9%) 250 (1.8%)Homatropine (27084) 66170 (51.1%) 680 (4.9%)
Hydroxychloroquine (5521) 22900 (17.7%) 2000 (14.4%)Infliximab (191831) 2880 (2.2%) 1200 (8.6%)
Iodoquinol (3435) 7350 (5.7%) 80 (0.6%)Leflunomide (27169) 1460 (1.1%) 480 (3.5%)Methotrexate (6851) 17710 (13.7%) 1750 (12.6%)Oxyquinoline (110) 220 (0.2%) 0 (0.0%)
Sulfasalazine (9524) 5320 (4.1%) 570 (4.1%)Total 129560 13890
Results% of patients with comorbidities induced by AZA. Diagonal represents proportion of patients experiencing single side
effect. Cell color indicates relative risk of developing a comorbidity (compared to other drug in class).
Cr AST/
ALT Bili Neutro-
penia Neutro-philia
Temp BP Hgb LDH
Cr 11% 24% 18% 12% 29% 41% 47% 65% 24%
AST/ALT 20% 14% 35% 10% 25% 30% 15% 50% 20%
Bili 15% 35% 14% 5% 50% 25% 30% 45% 20%
Neutropenia 2% 2% 1% 25% 0% 2% 7% 6% 0%
Neutrophilia 4% 4% 8% 0% 45% 6% 12% 18% 8%
Temp 19% 16% 14% 5% 22% 12% 59% 54% 5%
BP 6% 2% 5% 5% 13% 17% 29% 18% 3%
Hgb 22% 20% 18% 10% 46% 40% 46% 28% 22%
LDH 29% 29% 29% 0% 71% 14% 29% 79% 61%
1.0 1.5 2.0 2.5 3.0 3.5 4.0
R e l a t i v e R i s k
1° effect
2 ° effect
ResultsAZA-induced comorbidity network showing links with significantly
increased risk relative to other anti-rheumatic drugs. Lab measurements in green have an increased risk for occurrence in
patients taking AZA; grey nodes have a decreased or non-significant risk. Size of a node corresponds to proportion
patients experiencing that side effect.
Study Conclusions
• 1st study of confirm anecdotal case reports in large cohort.
• Able to compare AZA to other drugs in class (CER).
• Identified temporal relationships among side effects.
• Identified possible mechanisms to screen for impending renal dysfunction (anemia and increasing LDH predict/preceed renal side effects).
• Study performed by 3rd year Case Medical School student as part of 4 week informatics rotation.
Discussion
De-identified Population Data
• Advantages– Not human studies research (no IRB)– No HIPAA issues (no security issues)
• Disadvantages– Limited data analytic (statistical) tools– Limited research questions
Keys to Using EHR Data
• Understanding Data Sources
• Corroborating Data/Findings– Internal versus external corroboration
• Clinical Data versus Research Data
Understand your data sources, corroborate your data/findings, and realize that the data
represents clinical practice.
EHR Data Quality
Type of data Relative Quality
Demographic (age, gender, race/ethnicity) Very High
Lab Results Very High
Prescriptions Very High1
Vital Signs High
Diagnoses (ICD-9 codes) Medium (variable)
Family/PMH/Social History Low
Other ???
1- for prescriptions written; up to ~40% of prescriptions are never filled
Lots of information desired for research is not stored in the electronic health record as digital data during
routine clinical care.
Clinical Research Paradigm
Characteristic Old Paradigm New Paradigm
Data siloed aggregated
Infrastructure Resources significant none/minimal
Queries/Analysis days/weeks/months
real-time/near-real time
Self-Service minimal high
Researchers want quick, easy, access to “all” data themselves!
Clinical Research Implications
Characteristic Old Paradigm New Paradigm
Data Separate Research Database
Shared Research and Clinic Database (EHR)
Time 1000+ hours 100+ hours
Money 100,000-1,000,000+ 0-10,000+
People Many Few
Order of magnitude less time and money with electronic health records.
EHR data and clinical research informatics tools are creating a paradigm shift in CER.
THE FUTURE IS NOW!