Personalized Medicine Research at the University of Rochester

Henry KautzDepartment of Computer Science

Personalized Medicine

Smart SensingSmart Sensing

Intelligent Information

Management

Intelligent Information

Management

Effective InterfacesEffective

Interfaces

Putting the patient at the center of their health system

Family & Friends

HealthcareProviders

Web

Repositories

HealthcareInstitutions

Researchers

Smart Sensing• Non-invasive wearable sensors• Personal biosensors• Environmental sensors• New data streams + machine

learning = “New vital signs”

Invasive-Implant-Biopsy

Non-Invasive- BP, HR, …-Imaging-Smart materials

Ambient-Motion - Activity-Sound - Interaction

Intelligent Information Management

• Longitudinal data– Personal baseline– Detect trends & deviations from norm

• Personal health records– Privacy– Sharing– Anonymous aggregation

• Patient-centered decision support

Effective Interfaces

• Multimodal– GUI, touch, gesture, speech, …

• Mobile– Portable, networked, wearable

• Intuitive– Easy to learn, use, maintain

• Adaptive• Proactive

Computational Challenges

• Understanding human behavior from sensor data– Integrating vastly different kinds of data

• E.g.: RFID touch sensor, machine vision, EKG– Incorporating commonsense knowledge– Compute intensive methods for learning & inference

• Embedded, mobile, and distributed systems– Data transport in dynamic, heterogeneous environments

• E.g.: Data collected indoors, outdoors, laboratories, homes– Security and data sharing

• Patient, doctor, family, researchers, …– Data / annotation / interpretation streams

University of Rochester

• Center for Future Health– Interdisciplinary center for proactive healthcare

technology– Researchers from Strong Medical Center, UR

Electrical & Computer Engineering, UR Computer Science

• Laboratory for Assisted Cognition Environments (LACE)– New (2007) effort focuses on applying AI and

machine learning to technology to help cope with cognitive disabilities

8

System Concept

Personal Health Monitoring

PHASE Project• Create a prototype proactive personal

health monitoring system for cardiac patients

– Determine the value of prognostics in chronic care management

• Borrow from the field of machine health monitoring

– Identify the most minimally invasive ways to capture data

– Mine collected data to identify personal baselines, data defined models and track changes

– Identify patient preferences and create a system that gives a valuable user experience

– Identify effective ways to share data with health care providers

Measure• Cardiac function (ECG)• Respiration (Sound/ECG)• Activity (Accelerometry) Alivetec ECG & motionTouch screen

mobile phone

Personal Health Management Assistant• Provide effective, intuitive access to

information in Personal Health Record• True conversational interaction

– UR Computer Science leading center of research on dialog systems

– Not just canned responses: reasons about user model and dialog context

• Target population: Heart failure patients following self-care guidelines– Collect information relevant to condition – Interpret with respect to self-care guidelines– Suggest appropriate course of action– Facilitate information sharing with doctors &

family

Assisted Cognition• AI + pervasive computing + assistive technology• Potential users– Alzheimer’s disease– Traumatic brain injury– Autism

• Example applications– Maintaining a daily schedule– Indoor and outdoor navigation– Step-by-step task prompting– Behavior self-regulation

ACCESS• Help persons with cognitive disabilities

travel safely in their community and employ public transit– Huge issue for quality of life for millions of

people• GPS cell phone-based system– User carries phone during daily routine

• E.g. with job coach or family member– Automatically learns pattern of behavior

• Infers public transportation use– System notes breaks from ordinary

routine• Provides proactive help

Integrated Cueing & Sensing• PEAT: handheld-based activity cueing system

for persons with executive function impairment

• Problem: requires frequent input from user• Solution: use sensor to detect activities– Reduce user interaction– Reduce “learned dependency”– Enable context-dependent cues

• Video Clip: Compliance rule– “Use cane when leaving house”