“Validating the American Time Use Survey (ATUS): Does anybody really remember what they were doing yesterday?”

“Validating the American Time Use Survey (ATUS): Does anybody really remember what they were doing yesterday?”

Background & Design EMA Component Gallup ATUS Results for Location Results for Activities Results for Feelings

Background & Design EMA Component Gallup ATUS Results for Location Results for Activities Results for Feelings

Alan B. KruegerPrinceton University

American Time Use Survey• ATUS is a BLS survey conducted by the Census Bureau.

• ATUS began in 2003

• The sample consists of sampled individuals age 15 and older from households that have completed the final (8th) interview of the Current Population Survey

• One indiv idual per selected household in the CPS final rotation group is chosen to participate in the ATUS

• This person is interviewed once about his or her time use on the previous day.

• The ATUS collects data on how people spend their time; used to measure quality of life, work hours, home production, care giving, etc.

Study Design: Two Components• Real Time Data (EMA). Designed by Arthur Stone

- 168 workers carried a PDA that asked questions about their activities in real time for 3 consecutive days (Thursday, Friday and Saturday). - Convenience sample from Syracuse, NY and Stony Brook, NY. (Tried to conduct random sample but failed.) Data collected April –August, 2008

- Paid $120 for participation- Awake day divided into 6 evenly spaced intervals, and PDA beeps randomly within intervals. Approximately 168 x 6 beeps x 3 days ≈ 3,000 moments. - Called Ecological Momentary Assessment (EMA).

• Telephone Recall Survey. The same workers were given a survey that closely follows the ATUS on each day following EMA. This survey component was conducted by Gallup for Princeton and I call it P-ATUS.

- 478 interviews.

Compare P-ATUS to EMA in aggregate and overlapping moments.

Palm Zire 31

EMA: Designed by Arthur Stone w/Alan Krueger

►Training and practice take place on Wednesday ► Enter in sleep times► Beeps for 2 minutes► 5 minutes to respond after beep. Device turns off after then.► Recorded time is when first tap occurs.

NB: Can only check one.

Done

Done

Done

Additional EMA Questions

• Duration of activity (in intervals) ‘till beep• Feelings: happy, angry, tired, stressed,

interested, pain on 0 (not at all)-6 (very much) scale

• Are you at work? Workplace engagement• Confidence, control, competence• With supervisor?• Take medication since last prompt?• Caffeine, cigarettes, alcohol in last 30 mins?

Actiheart Device

The Actiheart is an innovative, remarkably compact device that records physical activity and heart rate with a high level of accuracy. Levels of caloric expenditure can be determined using the information acquired by the device.

Actiheart is not a peak detector, like all other heart rate detectors. It actually digitizes the ECG signal and calculates the heart rate from the true R-to-R interval. What this means is that Actiheart loses less than 1% of the data. Other detectors may lose up to 30% of the data.

Physical Measurements

• Actiheart Device: physical activity, caloric expenditure, and heart rate in continuous time

• Cortisol Measurements -- 6 times a day, at end of EMA questions

• Will use: (1) to validate self reports; (2) interest in activities and health pathways

Princeton ATUS: Start with ATUSPrinceton ATUS: Start with ATUS

Gallup modified ATUS Blaise program for Krueger already. Called PATS. Now call it P-ATUS

Collect all activities engaged in during the day with ATUS instrument

Princeton Module: Randomly select 3 awake episodes and ask about feelings after complete time diary

Interviewed on Friday, Saturday and Sunday about preceding days; on some occasions interviewed about two days previously (8.2%) or three days previously (0.6%)

Coded main and secondary activities Used Census/BLS materials to train interviewers

Gallup modified ATUS Blaise program for Krueger already. Called PATS. Now call it P-ATUS

Collect all activities engaged in during the day with ATUS instrument

Princeton Module: Randomly select 3 awake episodes and ask about feelings after complete time diary

Interviewed on Friday, Saturday and Sunday about preceding days; on some occasions interviewed about two days previously (8.2%) or three days previously (0.6%)

Coded main and secondary activities Used Census/BLS materials to train interviewers

Key Questions:Key Questions:

What did you do (next)?

How long did you do [ACTIVITY]?

Who was in the room/who accompanied you?

Where were you?

How did you feel at this time?

What did you do (next)?

How long did you do [ACTIVITY]?

Who was in the room/who accompanied you?

Where were you?

How did you feel at this time?

Sample CharacteristicsEMA/P-ATUS• 83% Female• Mean Age=43• Median Inc. $50-75k• 83% White• 9% Black• 4% Hispanic• 41% Some College• 53% College+• 61% Education Sector• 19% Healthcare Sector

ATUS (June-Aug, 2003-07)

• 50% Female• Mean Age = 41 • Median Inc. $50-75k• 83% White• 12% Black• 13% Hispanic• 29% Some College• 33% College+• 9% Education

Sector• 10% Healthcare Sector

Time Use in P-ATUS Looks ReasonableLocation: ATUS (2003-07) vs. P-ATUS

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Avg

. P

erc

en

t o

f A

wake T

ime

P-ATUS

ATUS

r = 0.995

Note: ATUS is April-Aug.

Is the Sample More Accurate at Reporting than Other Samples?

• Can look at consistency of reporting on Day 1 vs. Day 3 for same subjects (or Day 1 vs. Day 2 if Day 3 unavailable).

• Correlation of education 1-2 days apart is 0.81• Correlation of log household income 1-2 days

apart is 0.93

The sample does not seem unusually accurate despite higher education

Structure of the Data

EMA

Match Rate – Percent of moments when Activity or Location is same in EMA and P-ATUS. Can also expand window to +/- 15 mins.

P-ATUS

Comparison of Location in EMA and P-ATUS

73.7%

5.6%

20.8%Exact Match

Match within 15 m

Mismatch

Sample consists of 2,396 overlapping moments from EMA and P-ATUS.

79% match within 15 mins.

0%5%

10%15%20%25%30%35%40%45%

EMA

P-ATUS

Location in EMA and P-ATUS During Overlapping Moments

Sample is 2,396 moments.r=0.996

2.7 pointdifference

Next Turn to Activities

• Activities are trickier because people can do multiple activities at once.

• If any of the activities on EMA match any of the activities in P-ATUS in overlapping moments, we call it an exact match.

0.37% 0.26% 0.04% 0.04%4.4%6.3%

67.8%

20.8%

0%

10%

20%

30%

40%

50%

60%

70%

80%

0 1 2 3 4 5 6 7

Number of Activities

Per

cen

t o

f M

om

ents

Multi-Tasking: Number of Activities Done at a TimeEMA Data

► 26% of the time people are multitasking.► On average, 1.32 activities are done at any one moment (treating “none of the above” as one activity).

None ofthe above

Much Less Reporting of Multi-Tasking in P-ATUS Data

► For 97.2% of time, people indicated one activity.► For 2.6% of time, people indicated two activities.► For 0.1% of time, people indicated three activities.

Similar to ATUS 2006 (according to Jay Stewart)

NB: Non-sleeping episodes.

Average Number of Simultaneous ActivitiesEMA Data

1.3 1.4 1.5 1.5 1.6 1.6 1.6 1.8 1.9 1.9 2.1 2.2 2.2 2.3 2.4

1.0

00.5

11.5

22.5

3

NB: Because unit is activity, this chart over samples multi-tasked moments.

What Goes on during Work Activities?

EMA Data• 73% of time only checked work

• 9% of time on phone

• 6% of time eating/drinking

• 5% of time socializing

• 4% of time caring for someone

• 1% of time class related

NB: Not conditional on multi-tasking. Requires activity iswork.

62.5%8.3%

29.2%Exact Match

Match within 15 m

Mismatch

Comparison of Activities in EMA and P-ATUS

71% match within 15 mins.

Sample consists of 2,360 overlapping moments from EMA and P-ATUS.

0.0

1.0

2.0

3D

ensi

ty

0 20 40 60 80 100matchpct

Percent of Activities Matching within +/- 15 mins. by Subject

Mean is 70%, Median is 73%, Q1 is 63%, Q3 is 82%.

N=165 subjects

Subject-Level Analysis – Percent EMA-PATUS Matching Activity Within 15 Minutes

0%

5%

10%

15%

20%

25%

30%

35%

40%

Work

House

work

Eating

, drin

king

Sports

, exe

rcise

Social

izing

, leisu

re

Care

HH mem

ber

Care

nonHH m

embe

r

Telephon

e

Sleep

Gro

oming

Religio

us, s

piritua

l

Volun

teer

ing

Trave

l

Class

Get

ting

pers

onal s

erv

Gov

ernm

ent,

civic

EMA

P-ATUS

r=0.990

Difference is4.6 points

Activity in EMA and P-ATUS During Overlapping Moments

Note: If multiple activities, used EMA activity that matched P-ATUS; otherwise selected work, eating, sleep; otherwise randomly selected.

Reliability of Individual-Level Work Data

b = cov(y1,y2)/var(y1)

Note: plim of b is attenuation bias

y1 is EMA measure of percent of time spent working, and and y2 is P-ATUS measure. (Note: P-ATUS is full awake day and EMA is sampled.)

ei is measurement error

i is average over 3 days, i=1,…,165

yi1= a + b yi

2 + ei

y1= 7.42 + 0.69 y2 R-square=0.32 (2.55) (0.08)

Activity Match Rate (+/- 15 mins) by Day

75.0% 76.1%

60.6%

0%10%20%

30%40%50%60%

70%80%

Thursday Friday Saturday

NB: Drops on Saturday because work is low.

Activity Match Rate By Phone Interview Lag: 1 or 2/3 Days

Activity Match Rate (+/- 15 mins) by Day

70.5% 72.4%

0%

10%

20%

30%40%

50%

60%

70%

80%

1 Day 2 or 3 Days

Number of Days between EMA and P-ATUS

Note: Sample size is 2,131 for 1 day lag and 221 for 2/3 day lag.

Linear Probability Models Dependent Variable Equals 1 if Exact Match on Activity, 0 if Mismatch

Female

To do: Turn into a 15-minute data set and regress heart beat onactivity dummies, location, companions, time of day etc., and person fixed effects.

Pain Stressed

Average of Subjects’ Ratings

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Happy Sad Tired Stressed Interested Pain

EMA

P-ATUS

Note: Order of emotions was randomized in P-ATUS. Sample is 165 individuals. Except for happy, all differences are significant at .005 level in paired t-test.

Correlations between Affect EMA and P-ATUS at Person Level, Averaging Over Overlapping Days

Happy 0.75

Sad 0.81

Tired 0.79

Stressed 0.78

Interested 0.74

Pain 0.86

n=165

01

23

4p_

pain

0 1 2 3 4b_pain

Note: If one adjusts the data for sampling variance in the person averages, r rises to 0.92 for happy and 0.94 for pain.

Conclusions/Remaining ChallengesConclusions/Remaining Challenges Data indicate a reasonable match rate between real-time

reporting and P-ATUS – better for location than activity. Discrepancies might be due to mistakes in EMA or P-ATUS. Two day recall does as well as one day recall Bias. Work hours are overstated in P-ATUS. John Robinson

found that time diaries show lower work hours than CPS weekly hours recall question. This study reinforces – diary overstates actual work time. My interpretation: Less precise/longer recall questions about work blur work and nonwork activities. In ATUS people tend to report nonwork activities as work if they occur at the workplace.

Nonresponse in EMA (skipped beeps) can be studied. ATUS has a relatively low response rate for a government

survey – but EMA is even worse, and not an option. The module on feelings does a good job characterizing

individuals’ days compared with real-time reporting Can validate further with physiological data.

Data indicate a reasonable match rate between real-time reporting and P-ATUS – better for location than activity.

Discrepancies might be due to mistakes in EMA or P-ATUS. Two day recall does as well as one day recall Bias. Work hours are overstated in P-ATUS. John Robinson

found that time diaries show lower work hours than CPS weekly hours recall question. This study reinforces – diary overstates actual work time. My interpretation: Less precise/longer recall questions about work blur work and nonwork activities. In ATUS people tend to report nonwork activities as work if they occur at the workplace.

Nonresponse in EMA (skipped beeps) can be studied. ATUS has a relatively low response rate for a government

survey – but EMA is even worse, and not an option. The module on feelings does a good job characterizing

individuals’ days compared with real-time reporting Can validate further with physiological data.