TRUST, Stanford, November 10-11, 2010

Towards Understanding the Usage Pattern of Web-based Electronic

Medical Record System

Xiaowei Li

Vanderbilt University

TRUST, Stanford, November 10-11, 2010 2

Outline

Background System Description Usage Pattern Study

– System-wide characteristics– User behaviors– Patient record access

Summary

TRUST, Stanford, November 10-11, 2010 3

What is EMR System

A computerized system that maintains patient data, connected to other clinical components, e.g., laboratory, pharmacy, decision support, etc.

Benefits: Facilitate medical information sharing; Reduce documentation errors; Improve healthcare service delivery;

Research Issues: Patient data privacy, comply with HIPPA and other policies. Web portal security, e.g., web attacks, insider threats. Performance optimization, etc.

TRUST, Stanford, November 10-11, 2010 4

Understanding Usage Pattern

First essential step towards building robust, secure, efficient web-based EMR system– Performance: component deployment, load balancing is

related with system utilization, user request pattern, etc.– Security & Privacy: established system profiles, including

usage pattern, help build anomaly detection system to defend against intrusion and insider threats.

– Interface design: user navigation pattern.

A comprehensive study on the usage pattern of a large-scale web-based EMR system.

TRUST, Stanford, November 10-11, 2010 5

Outline

Background System Description

– StarPanel system– Web session model

Usage Pattern Study Summary

TRUST, Stanford, November 10-11, 2010 6

StarPanel System

An integrated, longitudinal EMR system, deployed at Vanderbilt University Medical Center for over a decade.

Aggregates a number of patient data sources across clinical domains, including diagnosis, lab tests, radiology reports, etc.

A variety of user groups, including residents, physicians, technicians, clinic support staff, etc.

TRUST, Stanford, November 10-11, 2010 7

Web Session Model

Model definition:– Raw trace entry: { timeStamp, ipAddress, userid,

actionModule (.cgi), parameters (patient record number)}– s(u, t): user u initiates session s at time t.– A: action set, R: record set.– Γs = (Γt1, Γt2,…, Γtn,): operation sequence in session s, where

Γ = (a, r), a A, r R {-}. Indicates an instance of clinical ∈ ∈ ∪workflow.

Session extraction: – Starts with a login action, ends with another login by the

same user.– Measure “active” session duration from login action to the

last action performed.

TRUST, Stanford, November 10-11, 2010 8

Outline

Background System Description Usage Pattern Study

– System-wide characteristics– User behaviors– Patient record access

Summary

TRUST, Stanford, November 10-11, 2010 9

System-wide Characteristic

overall system usage– Server workload, user

population, data access correlate with each other showing a strong weekly pattern;

– The system usage is highly consistent over the year, especially for the user population.

– Indicates effective DoS detection system can be established based on accurate load prediction.

TRUST, Stanford, November 10-11, 2010 10

User Behavior (1)

Users behave differently– Examine user behaviors in

terms of the number of sessions, the number of distinct actions, the number of patient record accesses. The variances across users are extremely large.

– User profiles should be established for individuals or groups of users, based on user role or department affiliation information.

TRUST, Stanford, November 10-11, 2010 11

User Behavior (2)

User actions are different across sessions– Actions in user session is encoded into action vector.– Use cosine similarity to examine “distances” between

consecutive sessions.

– Indicates individual sessions cannot be used for training user profile.

TRUST, Stanford, November 10-11, 2010 12

User Behavior (3)

User actions are consistent within certain session window.– Let be aggregated action vector, starting from user

session s with window size w

– A stable user profile can be built over a carefully selected time frame and updated with time.

TRUST, Stanford, November 10-11, 2010 13

Patient Record Access (1)

Sessions target at a small group of records– Most users are using the system to service specific patients

in each session.

User-Record pairs are sparse– Echoes the stable patient-caregiver structure. – Indicates fine-granularity access control policies can be

established.

TRUST, Stanford, November 10-11, 2010 14

Patient Record Access (2)

Records have quite different “popularity”– The variances between the “popularity” of records are large.– Deviates from Zipf distribution (web object popularity model)– There is no “interest hotspot”, since the record access is

based on the medical status and treatment of patients, rather than the “interest” of the caregiver.

TRUST, Stanford, November 10-11, 2010 15

Outline

Background System Description Usage Pattern Study

– System-wide characteristics– User behaviors– Patient record access

Summary

TRUST, Stanford, November 10-11, 2010 16

Summary

The workload of EMR system is highly consistent and predictable over time.

EMR users behave quite differently. For an individual user, his/her behavior exhibits fluctuation

across consecutive sessions. Yet, the aggregated behavior over certain time frame is consistent.

Patient records have different access patterns and “popularity”. Pairing between users and records is extremely sparse.

We observe that general web-based system performance optimization and anomaly detection system cannot fully utilize EMR-specific system features, thus less efficient and effective.

One of future directions can be incorporating EMR-specific behaviors into anomaly detection system.