Diary non-response and quality in 2 surveys

Image du produitDiary non-response and quality in 2 surveys

Delphine Roy, INSEE (France)

Diaries as a reliable data collection instrument Gershuny (2000)

compares diary information with retrospective questionnaire: memory effect, loss of information

What type of data? Lots of information, spread in time :

itineraries (National Transportation Survey) spending (Household Budget Survey) daily activities (Time Use Survey)

The point as regards quality is: prevent loss of information due to memory

= must be filled BY RESPONDENT, all over a time period - day(s), week(s), month(s)…

INTRODUCTION

Issues

Diary = quality

IFFilled over time, not in retrospectDoesn’t amplify non-response

Compliance? Problem: a written medium + respondents left alone,

autonomous. Difficult to control the data collection process as no one is present.

What can we learn from previous surveys?

INTRODUCTION

1. The data: 2 INSEE surveys

2. Diary non-response: household level

3. Diary non-response: individual level

4. Diary quality: when and by whom are diaries actually filled?

OUTLINE

1.1. The 1998 TUS

›Time Use Survey

– February 1998 to February 1999– 12 000 addresses in the sample– 1 diary for each household member 15– what is recorded = ACTIVITIES – 10-minute time slots– « natural language ». Ex: « washed the dishes »

1. Two diary-based surveys

The TUS diary: left page1. Two diary-based surveys

The TUS diary: left page - example1. Two diary-based surveys

The TUS diary: right page - example1. Two diary-based surveys

1.2. The 2006 HBS

›Household Budget Survey

– March 2005 to March 2006– 20 000 addresses in sample– 2 diaries for each household member 15 – 2 consecutive weeks – What is recorded: purchases– Receipts can be stapled in the diary but need

« commenting »– Purchases without receipt must be written down

by the household

1. Two diary-based surveys

The HBS diary1. Two diary-based surveys

The HBS diary - example1. Two diary-based surveys

Several steps when dropout can occur:

HH refuses to answer the survey from the start

HH doesn’t complete Interview n°1

HH reaches the end of ITW n°1 but refuses all diaries

≥ 1 diary is accepted but no ITW n°2 can be done

ITW n°2 is done but no diary is filled when the interviewer returns, and HH won’t fill them with the interviewer


TUS (1998) HBS (2006)

Sample size 12 045 20 000

Household contacted 8 861 14 736

Household reaches end of 1st ITW

8 186 11 513

At least 1 diary is left to the HH

7 995

(157 refusals)

10 839

(674 refusals)

Second ITW doneN/A

10 352

(487 avoid ITW n°2)

At least 1 diary is filled

7 949 10 243

2.1. Non-response figures2. Diary non-response: household level

Simple logistic regression : refusing the diary=1

Explanatory variables: only what is collected at the beginning of ITW1 to have information on a maximal number of households, even dropouts

3 different definitions of the outcome, for HBS:

1) Refusing to take even 1 diary

2) Eschewing ITW2

3) Not having filled a single diary when the interviewer returns

Then 2) + 3), and 1) + 2) + 3) together. Is it the same thing? Same type of HH in 3 behaviors?

2. Diary non-response: household level2.2. Methodology

TUS HBS

Refuse to take any diary

157 / 8186

(105 with covariates)

674 / 11531

Explanatory variables:

• Occupational category of HH head

• Education level

• Type of household (couple or not, with or without children)

• Age of HH head

• Region of residence

• Survey Wave ( = time of the year)

• Number of employed persons in the HH

• Subsidized housing

• Rural / urban + size of the urban area

• + for TUS: HH income (collected at the beginning of survey)

Several specifications were used: with / without income; OLS / logistic. Same results.

2. Diary non-response: household level2.2. Methodology

TUS HBS

HH = Farmer : + 1.3 pts HH = Self-employed (excl. farmers): + 1.5 pts

HH = Not in labor force, not retired : +2.5 pts

HH has no degree: +0.8 pts No sig. effect of HH education

Income : no sig. effect

Refusal to give income: +3.6 ptsN/A

No sig. effect of HH type Men living alone, no child: +2.7 pts

Subsidized housing: No effect

Survey wave n°1 : +1.3 pts

(16/02 to 29/03/1998)

No sig. effect of survey wave

> 75: +1.2 pts

45-55: +0.9 pts

15-25: - 3 pts

25-35: -1.8 pts

> 75 : + 3.7

Monotonously increasing in age

East, West , ‘Center-East’ : -

Paris urban area : + 2 pts

Same regions + ‘Central France’ : -

2. Diary non-response: household level2.3. Results on refusing to take diary

1) No ITW2 is done 2) ITW2 accepted, but no diary

Either 1) or 2)

N = 487 N = 109 N = 596

Couples

with or w/out children: - -

Couples

w/out children -

Couples

with or w/out children: - -

HH > 75: ++ HH > 75: ++

HH not in the LF: + HH not in the LF: +

Waves 2 and 3 (25/04/2005 to 28/08:2005: summer holidays): ++

Waves 2 and 3 (25/04/2005 to 28/08:2005: summer holidays): ++

Subsidized housing : + Subsidized housing : +

No LF participant in the H: - No LF participant in the H: -

Is there a difference between refusing a diary and not giving it back?

2. Diary non-response: household level2.3. Results: HBS details

Conclusion on households with no diary

› Elderly HH are much harder to convince

› The self-employed are more reluctant, esp. farmers for TUS and business owners for HBS

›Regional (dis)inclination towards NIS surveys

›Refusing to take a diary is different from not doing ITW2.

›No ITW2 = refusal (>75) + lack of availability– Vacations– Household w/out couple


What do we call « individual diary non response »?

(Little information was collected on the data collection process itself during the 1998 TUS)

Should receive a diary Do we have one ?

16462

yes : 15441

no: 1021

Among the 8186 HH who reached the end of the 1st ITW

3. Diary non-response: individual level3.1. TUS

Some individuals can write down their purchases on another household member’s diary: a kind of proxy (the relevant info is at the household level).

Mostly men and teenagers

They are considered as having filled their diary, even if we might think the quality is not as good as if they had had their own (loss of information caused by proxy)

3. Diary non-response: individual level3.2. A specificity of the HBS


Should receive a diary received one? Is it filled ?

Did report on someone else's diary?

yes : 572no: 363

20475

no: 935

Will report on someone else's diary ?

yes: 13313

yes: 6002no : 224

yes : 14249

no: 6226

+ Is there a difference between the 2 categories of non-respondents?

Among the 10240 HH for whom we have the individual variables



3.2. HBS: more detailed information

Assuming people with no diary did not want to fill it, who said so and who only didn’t give it back?

›Refuse from the start to take a diary (N=224)– More skilled– 15 to 25– Parisians– Above 55 and even more, above 75

›Do not give it back (N=363)– Self-employed– Less skilled


Is there a difference between the 2 categories of non-respondents?

3.2. Refusal or avoidance?

3.3 No diary: explanatory variables

Explanatory variables: Gender Age Lives in a couple Education Nationality (French by birth, became French, EU, non-EU) Occupation (6 categories + Student + Retired + Other Non LF participant) Region of residence


3.4. No diary : results

TUS HBS

Woman: – 2.7 Woman: – 1.4

Foreign non EU: + 4Foreign EU: +3.8Became French: not different from French by birth

Foreign non EU: + 1.6Foreign EU: + 2

Age 15 to 25: + 5.4Age >75: +1.7

Age 15 to 25: + 4Age 25 to 30: + 1Age>75: not significant

“Other non LF participant”: +1.9 (= not retired, excl. student)

“Other non LF participant”: + 1.3(= not retired, incl. students)

No degree: + 2.8 No degree: + 0.7

Lives in a couple: - 1.9 Lives in a couple: -1.6

Parisian region: +2.34East: – 1.8

North: +3“Center-East”: +1.5Brittany: +1

Are the 2 surveys’ diary non-response determined by the same factors?Dependent variable : no diary = 1


› 4 items, answered by the interviewer

– Away: 21.5%– Unable to (handicapped,

illiterate): 12%– Refusal: 42%– Other reasons: 10.5%– N/A: 14%

› Are more prone to refusal– 35 to 55 (60%)– No diploma (37%)

› Are away– 15 to 25 (45%)– Men

› Are unable to fill the diary– No diploma (31%)

› Citizenship has little, if any influence

› Variable on handicap not available

3. Diary non-response: individual level3.4. Cause of missing diary: TUS

› 5 items, answered by the interviewer at the end of ITW1:

– Away: 58%– Unable to (handicapped,

illiterate): 18%– Refusal: 12%– Other reasons:12%

› Are more prone to refusal– Men (15%)– Self-employed (25%)– 45 to 55 (40%)– Near zero refusal among

college graduates› Are away

– 15 to 35– College graduates

› Are unable to fill it– No diploma (43%)– Handicapped (60%)– Elderly >75 (68%)

Excluding the 30% of individuals who do not receive a diarybecause they will report on someone else’s

3. Diary non-response: individual level3.4. Cause of missing diary: HBS

Conclusion on individuals lacking a diary

› Very similar explanatory variables in the 2 surveys (except regions)

› Effect of educational level more important for TUS– easily understandable since it is much more of a

narrative exercise

› The possibility of a “proxy” (reporting on someone else’s diary) avoids losing too many men in the HBS

› Teenagers living with their parents are very difficult to catch


Every day 84%

Irregularly 11%

Just before or during ITW2 5%

HBS

TUSAll along the diary day 35%

In the evening, with notes 9.5%

In the evening, w/out notes 28.5%

The day after 14%

Other 13%

4. Indicators of diary quality4.1. When is the diary filled?

›Women are better compliers

– HBS: 85% every day VS 80% of men– TUS: 38% all along the day VS 31% of men

› The older the respondent, the later the diary is filled- HBS:

- TUS:

< 65 65-75 > 75

Just before / during ITW2

4 to 5% 7% 11%

< 65 65-75 > 75

« other » (~ ITW2)

10 to 15% 21% 33%


›Occupation: clear differences

–Respondents not in the LF (housewives, retirees) fill the TUS diary more often “all along the day”, as requested.

– Farmers and other self-employed fill the diary later: “in the evening without notes”, “the day after” (TUS) or “just before or during ITW2” (HBS)

– Tertiary sector wage earners (employees, managers, “intermediate occupations”) typically fill the diary “in the evening”, with or without notes (TUS), or every day (HBS)


› Education: no clear pattern except for those w/out diploma

– HBS: 76% of “no education” filled diary everyday, 15% irregularly, 9% just before or during ITW2

– TUS: 30% all along the day, 22% “other”

›Citizenship: language barrier

– HBS: 10% just before or during ITW2

– TUS: 20% “other” for non-French citizens

Again: TUS = more difficulties


›Region: Constants and variations

–TUS: West, East and ‘Center-East’ comply better (+ 4pts), Mediterranean much less (-7pts)

– HBS: East, West and ‘Center’ comply better (+ 2 to + 4 pts), South-West less, Mediterranean = OK


The respondent him/herself 89%

Another HH member 6.5%

The interviewer during ITW n°2 4%

Someone else 0.5%

4. Indicators of diary quality4.2. By whom is the diary filled? (HBS only)

› The interviewer fills the diary more often when the respondent is:

– Living alone (6.6%)– Without degree (9%)– Handicapped (10%)– A farmer (12%)– Elderly (15%)


› Someone else in the HH fills the diary (not incl. those with a « proxy diary ») more often when the respondent is :

– Without degree or less than high school (8%)– Living in the North of France (10%)– Self-employed or industrial worker (11%, VS. 4%

of white-collar workers, mainly women)– A man (11%, VS. 3% of women)– Under 25 (30%) or between 25-35 (10%)


›Quality is less good at the ends of age distribution:– Youth living with their parents– Elderly

›Men escape data reporting much more than women:– We know less on how they spend their money

and time› Self-employed are more difficult than wage-earners:

– Same as other surveys, they are a less well-known population

›Regional variations have some constants (East > Paris and South-East) as well as much unexplained variability

CONCLUSION

›Much happens during the second ITW› A lot of information is « rescued »

– Elderly– Less skilled– Handicapped– Non-native speakers

› Importance of the interviewer as a mediator when the information must be written down

› Survey design must take that into account and allow interviewer to spend time on those respondents

CONCLUSION

Documents

Diary non-response and quality in 2 surveys