4.1 - WANDA: An End-to-End Remote Health Monitoring and Analytics System for Heart Failure Patients

WANDA : An End-to-End Remote Health Monitoring and Analytics System for Heart Failure Patients

Mars Lan, Lauren Samy, Nabil Alshurafa, Myung-kyung Suh,Hassan Ghasemzadeh, Aurelia Macabasco-O'Connell and Majid Sarrafzadeh

Wireless Health Institute, UCLA, USASchool of Nursing, UCLA, USA

Outline

• Background• Related Work• WANDA system architecture• Clinical Trials• HF Symptom prediction & results• Conclusion

2Copyright: UCLA Wireless Health Institute

Background

• Heart Failure (HF) is one of the leading causes of death in US and around the world (Lloyd-Jones, 2010)

• 25% of patients treated for HF are re-hospitalized within 30 days. 50% readmitted within 6 months (Ross, 2010)• $17b/year in US. 20% of total Medicare expenditure in acute hospital

care

• Remote health monitoring offers a promising solution• Must be low cost, robust, and scalable• More than just collecting, storing, and viewing sensor data –

need to have the “smart” to analyze data automatically


D. Lloyd-Jones et al., "Executive summary: heart disease and stroke statistics--2010 update: a report from the American Heart Association," Circulation, vol. 121, p. 948, 2010.J. S. Ross et al., "Recent national trends in readmission rates after heart failure hospitalization," Circulation: Heart Failure, vol. 3, pp. 97-103, 2010.

Related Work

• Chaudhry’s telemonitoring study (Chaudhry, 2007)• Daily phone call to automated telemonitoring system over 6 months• Manually enter weight & responses to question using phone’s keypad• Unsuccessful in reducing risk of readmission or death for HF patients

• Soran’s HF Study (Soran, 2010)• Weight + daily questions• Manual data entry via phone• Failed to show improvement due to poor adherence

• Home or Hospital in Heart Failure study (Mortara, 2009)• Weight, heart rate, blood pressure, questionnaire, ECG• Manual data entry via phone and transmit only once every week• Higher compliance (81%) but still show no improvement


S. I. Chaudhry et al., "Randomized trial of telemonitoring to improve heart failure outcomes (Tele-HF)," J of cardiac failure, 13, 709-714, 2007.O. Z. Soran et al., "Cost of medical services in older patients with heart failure: those receiving enhanced monitoring using a computer-based telephonic monitoring system compared with those in usual care" Journal of cardiac failure, vol. 16, pp. 859-866, 2010.A. Mortara, et al., "Home telemonitoring in heart failure patients: the HHH study (Home or Hospital in Heart Failure)," European journal of heart failure, vol. 11, pp. 312-318, 2009.

Related Work

• Problems with previous studies (Desai, 2010)• Poor adherence: asking patients to perform too many tasks• Slow and unreliable transmission: phone line• Unreliable measurement: manual data entry• Not capturing the important markers: weight & questionnaire alone may not

be enough• Lack of advanced algorithm to predict adverse events

• How does WANDA address these issues?• Automate the monitoring process as much as possible, from measurement,

transmission, to notification• Support multiple sensors and gathering of a wide range of bodily

measurements• Provide data analytics engine for developing event prediction algorithms


A. S. Desai and L. W. Stevenson, "Connecting the circle from home to heart-failure disease management," New England Journal of Medicine, vol. 363, pp. 2364-2367, 2010.

WANDA System Architecture

• Data collection• Collect data from heterogeneous wireless sensors• Smartphone-based gateway & tablet-based portal

• Data storage & access• Scalable database & indexing• Easy-to-use access interface

• Analytics engine• Wealth of analytics algorithms• Data preprocessing, classification, clustering, rule/pattern mining• Support offline & online analysis

• Supporting tools• Administrative portal• Questionnaire• Social network app


WANDA System Architecture


Smartphone-based Data Collection

• Runs on most Android phones

• Collect data from Bluetooth-enabled sensors

• Transmit data to central repository via WiFi or cellular connections

• Built-in activity monitoring, fall detection, and pedometer

• Immediate feedback and data visualization

• Questionnaire and reminder


Caregiver Portal Tablet


Data Analytics

• Core strength of WANDA over other remote monitoring systems

• Provide a wealth of machine learning and data mining algorithms

• Offline analysis• Data downloaded and analyzed• Statistical models generated and validated

• Online analysis• Based on generated models to predict events from incoming data• Use temporary storage and PubSub interface

• Fully integrated with Weka framework


Storage and Access

• Amazon S3 for raw data storage• Reliable, secure, and scalable

• Amazon SimpleDB for data indexing• Schemaless: extensible and ideal for incomplete data (e.g. medical record)• High performance and scalable

• RESTful web interface• Decouple backend database technology from developers• Simple pseud object oriented access• HTTP lib widely available on most platforms

• Publisher-Subscriber (PubSub) interface• Real-time notification via email, SMS, or HTTP• Clients can subscribe to fine- or coarse-grained changes• Supports multiple publishers & subscribers• Ideal for events, alarms, and real-time analytics


Supporting Tools

• Administrative portal• Web-based tool for study & patient information management• Labeling, annotation, adverse event tracking• Rich data visualization, graphing, and searching

• Questionnaire system• Questions specified by clinicians and pushed to patient’s phone• Patients are reminded on daily basis• Reponses automatically stored in the database

• Social network• Facebook application• Encourage healthy behavior and compliances• Personal goals & competition


Current & Completed Clinical Trials

Type # of Subjects Institutes Status

Parolee study (questionnaires, remote coaching, SMS)

600 UCLA Nursing School Completed

Male contraceptive(questionnaires, compliance via SMS)

300 Harbor Medical Current

Congestive Heart Failure Monitoring(weight, BP, activity, questionnaires)

20 (pilot)20 (extension)1500

UCLA, UCSF, UCI, UC Davis

CompletedCompletedCurrent

Copyright: UCLA Wireless Health Institute 13

Current & Completed Clinical Trials

Type # of Subjects Institutes Status

Diabetes(blood glucose, activity)

50 UCLA Medical Center Current

Weight loss(activity, questionnaires, social networking)

20 UCLA Medical Center Completed

Cardiovascular Disease(activity, weight, BP, questionnaires)

60 UCLA Nursing School Current

Copyright: UCLA Wireless Health Institute 14

HF Symptom Prediction

• Daily weight change (DWC) is a commonly used predictor for worsening of HF symptoms• Recommended by American College of Cardiology/American Health

Association (Hunt, 2001)

• Several studies have raised doubt about DWC’s poor correlation with worsening of HF symptoms (Chaudhry, 2007; Lewin, 2005; Zhang, 2009)

• Experienced many false alarms based on DWC > 2lb during our clinical trials

• False-Positive (added load to the nurses)

• False-Negative (may eventually lead to readmission)


HF Symptom Prediction

• Example for “normal” patient• DWC > 2lb (false-positive)• Stable BP over 7-day period


Experiment Setup

• Real data from clinical trial over 3 months• 16 patients with daily weight, BP, questionnaires

• Patient self-reported worsening of HF symptom cross-checked with nurse’s phone log

• Total of 34 positive instances

• Nine features extracted• Daily change in weight, systolic & diastolic BP• 3-day standard deviation of weight, systolic & diastolic BP• 7-day standard deviation of weight, systolic & diastolic BP

• Ensuring fairness• Equal number of positive & negative instances• 10-fold validation to prevent overfitting• Average result over 10 runs to reduce effect of outliers


WANDA Algorithms

• Naïve Bayes (NBC)

• Nearest Neighbor (NN)

• Logistic Regression (LR)

• Voting Feature Interval (VFI)

• Ripple-Down Rule Learner (RIDOR)

• C4.5 Decision Tree (C4.5)

VS.

• Daily Weight Change thresholding (> 2lb)


Accuracy, Sensitivity, and Specificity

• Poor results for DWC• Accuracy: 51.9% cf. random

guess: 50% • Sensitivity: 15.9% (missed

many positives)• Specificity: 71.4% (mis-

classified many negatives)

• WANA algorithms• Accuracy: 70-74%• Sensitivity: At least 45%

better than DWC• Specificity: 70-87%• NBC, LR, and RIDOR

performs best overall


ROC Curve

• DWC is only slightly better than random guess with Area Under Curve (AUC) = 0.573

• All algorithms from WANDA performs significantly better with LR achieves largest AUC (0.817)

• NBC can be tuned to FPR = 0 with a respectable TPR = 0.652 20Copyright: UCLA Wireless Health Institute

Error Bound Analysis


• Is 74% accuracy good/bad? What’s the “optimal” accuracy?

• The error rate of any binary classifier is lower-bounded by the Bayes Error Rate:

• R1 & R2 are the regions where x is misclassified

• Class distributions w1 & w2 can only be estimated

• 2 x Bayes Error Rate ≥ 1-NN Error Rate• 1-NN Error Rate = 0.348 (based on empirical results), Bayes Error ≥ 0.174

• Theoretical max accuracy = 82.6%• Within 10% of WANDA’s results

Conclusion

• Design and implementation of WANDA• Smartphone-based gateway for data collection & transmission• Scalable data storage with RESTful web interface & PubSub real-time

notification• Analytics engine capable of prognostic prediction using machine learning &

data mining algorithms

• Prediction of worsening HF symptom using WANDA data analytics engine • Based on real data gathered from clinical trials• Identified 3-day & 7-day BP fluctuation as better predictors than DWC• Improved prediction accuracy by 20% & sensitivity by 45% over

thresholding algorithm based on DWC• 9% lower than the estimated upper bound


Documents

4.1 - WANDA: An End-to-End Remote Health Monitoring and Analytics System for Heart Failure Patients