23
Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1 , Gregory W. Hruby 1 , Sunmoo Yoon 2 , Alla Babina 1 , Feng Liu 1 , Jianhua Li 1 , David Hanauer 3 1 Department of Biomedical Informatics, Columbia University, New York, NY 2 School of Nursing, Columbia University, New York, NY 3 Department of Pediatrics, University of Michigan, Ann Arbor, MI

Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

Embed Size (px)

Citation preview

Page 1: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

Get the Data You Need Via Cross-Disciplinary Conversation Alignment

Chunhua Weng1, Gregory W. Hruby1, Sunmoo Yoon2, Alla Babina1, Feng Liu1, Jianhua Li1, David Hanauer3

1Department of Biomedical Informatics, Columbia University, New York, NY2School of Nursing, Columbia University, New York, NY

3Department of Pediatrics, University of Michigan, Ann Arbor, MI

Page 2: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

2

Big Health Data

Page 3: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

3

Data Access for Life Scientists

Data

• Patient Privacy• Data Governance• Knowledge Gaps

Page 4: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

4

Example Data Request FormsForm Title Institution

DISCOVERY Committee Data/Report Request Columbia UniversityPRIMARY CARE INFORMATION PROJECT DATA REQUEST FORM NYC-Department of HealthDATA Request Statement of Work NYC-Department of HealthInternal Hub Data Request Form NYC-Department of HealthEnterprise Data Warehouse Northwestern UniversityHonest Broker Office Michigan UniversityCTRI Service Request Form UCSDUCSF MEDICAL CENTER CANCER REGISTRY UCSFClinical Data Access Request Online Form Boston UniversityData Request Form Kansas Univ. Medical CenterEnterprise Data Warehouse VanderbuiltMetting Request with ICTR BMI University of Wisconsin UCH RSS New Procotol Application University of Colorado… …others

Page 5: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

5

Data Access for Life Scientists…Hard!

• Vocabulary Differences• Conceptualization Discrepancies• Tacit Knowledge Gaps

Page 6: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

6

Related Work• Hruby GW, Cimino JJ, Patel V, Weng C, Toward a Cognitive Task Analysis

for Biomedical Query Mediation, Proc of 2014 AMIA Joint Summits on Translational Science, 7-11 April 2014, San Francisco, CA, in press.

• Hruby GW, Boland MR, Cimino JJ, Gao J, Wilcox AB, Hirschberg J, Weng C, Characterization of the Biomedical Query Mediation Process, Proc of 2013 AMIA Joint Summits on Translational Science, 18-22 March 2013, San Francisco, CA, 89-93.

Page 7: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

7

How do researchers and query analysts negotiate data needs via email?

Page 8: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

8

Methods

• Participants: query analysts from CUMC– A: > 10 years of experience, no MD– B: novice analyst with a non-US MD– C: 2 years of experience, no MD

• Eight user-reported critical incidents, including 150 email messages

• Message de-identification• Thematic analysis of dialog acts

Page 9: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

9

Results

Page 10: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

10

Themes in Dialogue ActsRow Labels M->Q M->R Q->M Q->R R->M R->Q R->R TotalNeed Clarification 1 15 3 33 52Data transfer 17 17Set up meetings 1 8 2 5 16Follow-up 1 1 1 7 10Confirmation 5 2 1 8Initial contact 2 6 8Progress Reporting 2 1 1 1 5Request problem specification 1 1 1 2 5Middle-point data verification 4 4Request sample patient list 3 3Revise query 2 1 3Send a starting patient list 3 3Guide data interpretation 2 2Request help with data interpretation 2 2Request help with query formulation 1 1 2Results Transfering 2 2Data reformatting request 1 1Guide data need clarification 1 1Help with data interpretation 1 1Progress Inquiry 1 1Request help with data interpretation 1 1Send a starting patient list; Setting up meetings 1 1Send related information 1 1Suggest Query Revision 1 1Grand Total 5 2 2 65 8 65 3 150

Page 11: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

11

Top Themes

Row Labels Q->R R->Q other TotalNeed Clarification 15 33 … 52Data transfer 17 … 17Set up meetings 8 5 … 16Follow-up 1 7 … 10Confirmation 5 2 … 8Initial contact 2 6 … 8

Q: Query Analyst; R: Researcher; M: Manager

Page 12: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

12

Example Communication Patterns

Page 13: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

13

Aggregated Activity Transition Graph

Page 14: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

14

Need Clarification Example 1Hi Alice,

Thank you very much for your help. I just want to make sure I get it right. About the code 531.9 from your list - there are fifth digits with 531.9 code (0 and 1) and non of them match your description in the list. Please look at the icd9 codes below and let me know the right ones. 531.90 GASTRIC ULCER 531.30 ACUTE GASTRIC ULCER 531.00 ACUTE GASTRIC ULCER WITH HEMORRHAGE, WITHOUT MENTION OF OBSTRUCTION 531.10 ACUTE GASTRIC ULCER WITH PERFORATION, WITHOUT MENTION OF OBSTRUCTION 531.20 ACUTE GASTRIC ULCER WITH HEMORRHAGE AND PERFORATION 531.70 CHRONIC GASTRIC ULCER 531.40 CHRONIC GASTRIC ULCER WITH HEMORRHAGE 531.50 CHRONIC GASTRIC ULCER WITH PERFORATION 531.60 CHRONIC GASTRIC ULCER WITH HEMORRHAGE AND PERFORATION 531.91 GASTRIC ULCER WITH OBSTRUCTION 531.31 ACUTE GASTRIC ULCER WITH OBSTRUCTION 531.01 AC STOMAC ULC W HEM-OBST 531.11 AC STOM ULC W PERF-OBST 531.21 AC STOM ULC HEM/PERF-OBS 531.71 CHRONIC GASTRIC ULCER WITH OBSTRUCTION 531.41 CHRONIC OR UNSPECIFIED GASTRIC ULCER WITH HEMORRHAGE, WITH OBSTRUCTION 531.51 CHRONIC OR UNSPECIFIED GASTRIC ULCER WITH PERFORATION, WITH OBSTRUCTION 531.61 CHRONIC GASTRIC ULCER WITH HEMORRHAGE, PERFORATION, AND OBSTRUCTION

Thank you again.A

Hi A………….……………….………………..……………….. Hi A,

These are relevant codes pasted below:ICD9 codes041.86 Helicobacter pylori151.0-151.9 Malignant neoplasm of stomach209.23 Malignant Carcinoid Tumor of the Stomach211.1 Benign neoplasm of stomach230.2 Carcinoma in situ of stomach235.2 Neoplasm of uncertain behavior of stomach, intestines, and rectum531.9 Gastric ulcer unspecified536.8 Unspecified functional disorder of stomach537.9 Unspecified disorder of stomach and duodenum These are the CPT codes we need: 43620 43621 43622 43631 43632 43633 43634 43999 44238 49329

Page 15: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

15

Need Clarification Example 2Hi,>> Age 18 is the current age or diagnosis age?>> Do you care about which diagnosis is made first?

Hi A………….……………….………………..………………..

I filed the discovery request today.I am requesting 2 lists of MRNs that meet the criteria below.LIST 1Population: adults aged 18 years or olderDate range: 1992-presentInclusion criteria:ICD 079.5, OR 079.53, OR 795.71, OR 042, OR 043, OR 044,ANDICD 155.0, OR 230Data elements requested:Date of birthAge sexrace/ethnicity

Page 16: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

16

Need Clarification Example 3I will double check. It is not 'primary', I will correct it.Then exclude V24.6 and <15.Then exclude ICU only patients, if I can find a way.

By the way, I used 'age<18' according to your request doc. Youprefer 15 instead?

Hi A………….……………….………………..………………..

Hi A,Thanks for the data. It does seem strange to me that the numbers increase so much in 2013. Can you confirm that the numbers are indeed correct?

Also, can you tell me if the diagnosis is the "primary” diagnosis or not? We are most interested in those with a primary admitting diagnosis that is one of these variables.

Please also confirm that those with concomitant V24.6 are excluded and those age <15 are excluded.

Thanks,Researcher

Page 17: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

17

Iterative Need Clarification

Sample patients

Data sample

Data Requests

….…..…..

Codes

Concept Clarification

Data Verification

Page 18: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

18

Length of Query Processing Time

Analyst MD Experience Case Start End Length Avg

A No > 10 Y 1 10/8/1310/25/1

3 17 days 26d2 11/28/13 1/6/14 40 days3 12/28/13 1/17/14 20 days

B Yes < 0.1 Y

4 10/28/13 11/1/13 3 days

15d5 10/22/13 11/2/13 10 days

6 10/1/1310/22/1

3 21 days7 1/6/14 1/23/14 17 days

C No ~ 2 Y 8 10/6/11 1/9/13>400 days >400d

Medical knowledge may be associated with more efficient query processing

Page 19: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

19

Discussion

Page 20: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

20

Conclusions

• Decision support should not be just about user augmentation, but enable user engagement and shared decision making

• Besides alerts, reminders, information filters or summarizers, we also need better dialog-based decision support to bridge the gap between clinical researchers and Big Data

Page 21: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

Towards Dialog-based Query Support

Page 22: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

22

Limitations

• One institution• Small sample• Communication outside emails

Page 23: Get the Data You Need Via Cross- Disciplinary Conversation Alignment Chunhua Weng 1, Gregory W. Hruby 1, Sunmoo Yoon 2, Alla Babina 1, Feng Liu 1, Jianhua

23

Thank you

• NLM R01LM009886 (Bridging the semantic gap between clinical research eligibility criteria and clinical data, PI: Weng)

[email protected] http://people.dbmi.columbia.edu/~chw7007/