The Hidden Stories Behind Missing Data

partially joint work with Henry Potts and Katarzyna Stawarz; partially with the Help4Mood Consortium

The Hidden Stories Behind Missing Data

Maria WoltersUniversity of Edinburgh

maria.wolters@ed.ac.uk@mariawolters

http://www.slideshare.net/mariawolters

Overview

❖ What is Missing Data?

❖ The Story of Appropriation: Help4Mood

❖ The Story of Monitoring Work: Activity Trackers

❖ The Story of Hidden Missing Data: Electronic Health Records

Missing Data

❖ informally: observations that we would like to be there, or that should be there, but that are not

❖ Statistical treatment differs according to whether missing data are

❖ completely random

❖ predictable from existing data

❖ not predictable from existing data

Goal❖ Tell the hidden stories behind missing data by

understanding and describing data generation processes

❖ qualitatively for deeper understanding

❖ quantitatively to feed into data analysis and visualisation

❖ Current stage: setting out problem space in order to pick appropriate models

The Appropriation of Help4Mood

depressioncomix.tumblr.com

Depression is a change

relative to an individual baseline

Help4Mood: Supporting People with Depression

• daily monitoring • of activity using

actigraph • of mood, thought

patterns & psycho-motor symptoms using talking head GUI

• weekly one-page reports to clinicians

Maria K. Wolters, Juan Martínez-Miranda, Soraya Estevez, Helen F. Hastie, Colin Matheson (2013). Managing Data in Help4Mood AMSYS ICST DOI: 10.4108/trans.amsys.01-06.2013.e2

One Help4Mood per Day❖ Users interact with Help4Mood once a day

❖ Sessions can be short, medium, or long

❖ Planning algorithm ensures

❖ different kinds of data are collected regularly, e.g.

❖ mood: every day

❖ speech: 1-3 times per week

❖ sessions are varied

Patient Focused Development1. Actigraphy

2. Mood Tracker +Actigraphy

3. Mood Tracker,Thought Patterns,Speech + Actigraphy

4. Mood Tracker,Thought Patterns,Behavioural Activation,Relaxation,Speech + Actigraphy

Case Studies

Pilot Randomised Controlled

Trial

People with history of depression

People with current depression

Pilot RCT Participants❖ Participants with Major Depressive Disorder (SCID

diagnosed)

❖ Use Help4Mood for 4 weeks

❖ Background measures include demographics and attitudes to computers

❖ (Pre/Post measures to establish change)

❖ Qualitative interviews at intake and debriefing for those randomized to Help4Mood

Participants and Usage Patterns

❖ 18 in Romania, 7 in Scotland, 2 in Spain (EU Project)

❖ 14 treatment as usual (age 42 years +/- 10), 13 Help4Mood (age 35 +/- 12)

❖ None formally tracked or measured their mood before, but some used introspection

❖ Half used it regularly, but that was not daily; instead, it was 2-3 times per week. Why? Appropriation

What is Going On With Me?

The monitoring part helped me understand some things [. . .] sometimes I did not realize how I felt that day, how happy I was or how active I was. The system helped me observe these things and also control them. (RO14, female, 20–29)

Help4Mood as Coping[. . .] when I felt very upset, very low, I preferred longer sessions [. . .] because the session is pretty much focused on analyzing the thinking patterns of depressed people, and those recovering from depression. It gave me space to reflect on my own thoughts and feelings and gave me alternative ways that I could resonate with. (RO17, female, 30–39)

The most important thing for me was those relaxation exercises and I think that it would have been very useful for the patient to have direct access to them. (RO03, female, 50–59)

It Doesn’t (Quite) Work This Way

http://imgarcade.com/1/depressed-stick-figure/

Well-DesignedMonitoring

Self-Help Internet-Based

Therapy

+

http://www.acog.org/About-ACOG/ACOG-Departments/Long-Acting-Reversible-Contraception

https://www.osneybuyside.com/forget-big-data-just-collect-smart-data/

Protected Peer Support

http://www.clipartsfree.net/small/3977-game-piece-group-clipart.html

It’s a complex adaptive system

http://www.thebolditalic.com/articles/3609-the-stick-figure-guide-to-kicking-depression

Individualised monitoring based on what person has & does

Coping and getting better:• Twitter, exercise, kindness• Friends • Medications• GP

Productive (!) self insightand reflection

The Story of Monitoring Work

The Chore of Monitoring

If at all possible, it would be good not to have the same questions every day; or even if the questions are the same, the phrasing should be different. At some point it gets boring—I think this could be changed. (RO15, female, 30–39)

Self-Reflection is Hard Work

“This wasn’t very pleasant. Because you don’t go to therapy every day. You wouldn’t go every day; you would go maybe once a week or two or three times maybe, but not every day. It’s a bit too much to use it every day.” (P01, Case Studies)

TRACKER

whatswimming

weightlifting

team sports

self-reporteffort

e.g., Rooksby et al., CHI 2014; Lazar et al., CHI 2015

TRACKER

whenworried well

motivated for change

not tracking during lazy days

whatswimming

weightlifting

team sports

stigmaself-report

effort if there is a need

e.g., Rooksby et al., CHI 2014; Lazar et al., CHI 2015

TRACKER

when

who job (e.g. nurses)

allergies

wrist anatomy

forgetting

to wear to bringto charge

worried well

techy

motivated for change

not tracking during lazy days

whatswimming

weightlifting

team sports

style / fashion

stigmaself-report

effort if there is a need

e.g., Rooksby et al., CHI 2014; Lazar et al., CHI 2015

http://whitemenwearinggoogleglass.tumblr.com/image/51710586195

http://www.cio.com/article/2452759/health/wearable-techs-dilemma-too-much-data-not-enough-insight.html

Apple Watch Hermès

TRACKER

when

who job (e.g. nurses)

allergies

wrist anatomy

forgetting

to wear to bringto charge

worried well

techy

motivated for change

not tracking during lazy days

device

breaks

no longer holds charge

lost / stolen

whatswimming

weightlifting

team sports

no Internetstyle / fashion

stigmaself-report

effort

not synchingproperly

if there is a need

e.g., Rooksby et al., CHI 2014; Lazar et al., CHI 2015

http://thoughtstipsandtales.com/2014/11/06/fitbit-fun-ten-months-later/

http://thoughtstipsandtales.com/2015/03/05/fabulous-fitbit-accessory-to-keep-the-clasp-from-opening/

TRACKER

when

who

forgetting is a function of- user characteristics- illness- external stress

fit with identity

device

what

detailed user modeldevice model

connectivity model

fit with activities

effort required to track activity

fit with ulterior need(why tracking?)

TRACKER

when

who

forgetting is a function of- user characteristics- illness- external stress

fit with identity

device

what

detailed user modeldevice model

connectivity model

fit with activities

effort required to track activity

fit with ulterior need(why tracking?)

All except for ulterior need and identity amenable to formal/ quantitative modelling from appropriate discipline.

When looking at studies:

People will do things for research, to benefit others, to be part of something bigger,

and maybe also to earn some money,

that they wouldn’t do for themselves.

The Story of Hidden Missing Data

Hiding in Electronic Health Records

❖ Data go dark because of quality issues in data entry and management (e.g., Chan et al., 2014, Medical Care Research and Review)

❖ People go to the doctor when worried about something, which increases likelihood of detection of other problems.

❖ So does diabetes really increase your cancer risk, or is your cancer more likely to be spotted in regular check ups? (e.g, Badrick and Renehan, 2014, Eur J Cancer)

We can only detect what is being observed. Does that mean regular screening and monitoring?

What is the human cost of a false positive?

What is the human cost of a false negative?

What are the numbers needed to treat?

Questions? maria.wolters@ed.ac.uk

http://www.thebolditalic.com/articles/3609-the-stick-figure-guide-to-kicking-depression

Individualised monitoring based on what person has & does

Coping and getting better:• Twitter, exercise, kindness• Friends • Medications• GP

Productive (!) self insightand reflection