Bill Blyth

Global Methods Director, TNS

Changing Respondent Pathways in the Eurobarometer LandscapeBetter EB for better understanding the citizens’ needs

2

• Too often, international studies are regarded as the sum of national studies conducted in different countries.

• The key principles of our philosophy:

Method and mode evaluative criteria

ConsistencyReplicability

Reliability TimelinessCost

effectiveness

Validity

The Quality Triangle

Quant. alone has its limits

Fitness

Process User

Design

Eurobarometer Surveys Mode Assessment

• Need explicit feasibility assumptions

• Consistent coverage definition

• Regular coverage benchmarking and updating

• Country by country evaluation

• Transparent ‘Fitness for Purpose’criteria

• Agreed decision-making process

CAPI availability- drivers and constraints

• Respondent independent• Internet dependent• Very high capital investment• Sophisticated IT infrastructure

need• Multi-country inter-operable

capabilities

The EB gold standard

CATI feasibility- drivers and constraints

• Respondent dependent

• Fixed line diminishing

• Mobile only growing rapidly in some countries

• Response declining

• Legislative issues (do not call etc…)

% Individuals with a phone of any sort

Source: Eurobarometer 2005 & 2008

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

CY DK LU EL FI FR UK AT BE DE IE IT SI CZ ES EE HU SK LV PT LT PL MT TR MK BG RO

2005 2008

% Individuals with a mobile phone

Source: Eurobarometer 2005 & 2008

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

FI LU CZ DK SI IE EE IT UK BE AT SK CY ES FR DE LT LV PT HU EL PL MT TR BG MK RO

2005 2008

% Individuals with mobile and no fixed line

Source: Eurobarometer 2005 & 2008

-20%

0%

20%

40%

60%

80%

100%

SE NL LU DE CY UK EL SI DK FR IE PL BE ES AT IT HU EE PT SK LV CZ FI LT

2008

2005

Individuals with a mobile but no fixed line

Penetration within ageSource: Eurobarometer 2005 & 2008

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

15-24 25-39 40-54 55+IT CZ ITITIT ELELELEL PTPTPTPT CZCZCZ

Mobile only challenges

• Inadequate/non-existent sampling frames

• Legislative/ethical barriers to RDD - e.g. cost to respondents

• Multiple chances of selection

• Questionnaire length restrictions

• Poor response levels

CAWI feasibility- drivers and constraints

• Respondent dependent

• Little defined best practice

• Coverage variability

• Sample frame availability

• Access panel approach the norm

• Results are design and process sensitive

% Individuals with internet access at homeSource: Eurobarometer 2005 & 2008

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

EL HU SK LT PT LV IT PL CY CZ ES FR IE AT EE DE SI UK BE LU FI DK SE NL

2008

2005

Individuals with internet at home

Penetration within ageSource: Eurobarometer 2005 & 2008

0%

10%

20%

30%

40%

50%

60%

70%

80%

15-24 25-39 40-54 55+ITITITIT ELELELEL PTPTPTPT CZCZCZCZ

Summary

• Fixed-line below 70% in a number of EU countries• Rate of mobile-only coverage growth can invalidate

national trend comparisons for fixed-line designs• Internet penetration under 60% in around 50% of EU

countries• Internet penetration is highly skewed• Internet skew by education suggests a minority will never

access mode• Fixed-line and Internet coverage gaps are different• Questionnaire effects compound coverage and frame

bias

A way forward?

• Fitness for purpose model provides optimal approach

• Re-visit and clearly articulate priorities for Flash- for example speed, validity and consistency

• Evaluate mobile-only coverage gap effects against priorities, by re-analysis of standard EB data

• Assess alternative single mode designs• Expert group to advise on mixed-mode design

implications

A way forward?

• Choose uni-mode or mixed-mode model• Expert group to produce detailed specification

and consensus view on budget requirement• Specification to include ongoing validation and

detailed quality control metrics supported by audit

• Tender evaluation using a ceiling price accompanied by quality base contract award as opposed to current price/quality hybrid

• Expert group provides independent ongoing performance and validation monitoring

18