19
1 1 A systematic and strategic system A systematic and strategic system for data collection management at for data collection management at Statistics Norway Statistics Norway Seminar on New Frontiers for Statistical Data Collection Geneva, Switzerland, 31 October-2 November 2012 Bengt Oscar Lagerstrøm, [email protected], Statistics Norway

A systematic and strategic system for data collection management at Statistics Norway

  • Upload
    vanida

  • View
    46

  • Download
    1

Embed Size (px)

DESCRIPTION

A systematic and strategic system for data collection management at Statistics Norway. Seminar on New Frontiers for Statistical Data Collection Geneva, Switzerland, 31 October-2 November 2012 Bengt Oscar Lagerstrøm, [email protected], Statistics Norway. Outline. Background What is a action plan? - PowerPoint PPT Presentation

Citation preview

Page 1: A systematic and strategic system for data collection management at Statistics Norway

1

1

A systematic and strategic system for A systematic and strategic system for data collection management at Statistics data collection management at Statistics Norway Norway

Seminar on New Frontiers for Statistical Data CollectionGeneva, Switzerland, 31 October-2 November 2012Bengt Oscar Lagerstrøm, [email protected],Statistics Norway

Page 2: A systematic and strategic system for data collection management at Statistics Norway

2

Outline

• Background

• What is a action plan?

• Use of action plans

• Results

• Conclusion/Further works

Page 3: A systematic and strategic system for data collection management at Statistics Norway

3

Background

• Increasing emphasis on quality management– Total quality management, continuous quality improving, six sigma,

and so forth

• At the same time, our industry have experienced– Increased non response rates, and– (much) higher survey costs

Page 4: A systematic and strategic system for data collection management at Statistics Norway

4

Respons rates, NorwayUtvikling i svarandeler i noen utvalgte personundersøkelser

0

10

20

30

40

50

60

70

80

90

100

1967 1970 1973 1976 1979 1982 1985 1988 1991 1994 1997 2000 2003 2006 2009

År

Pros

ent

AKU

Valg

LKU-t

EU-Silc

ESS

Forbruk

Tidsbruk

Page 5: A systematic and strategic system for data collection management at Statistics Norway

5

Quality in data collection

• ”Number of ton produced”– Number of interview started– Item non-response– …….

• Optimal planning– How many interviewers and when?– Reduction of non-scheduled stops– Organization of good flow

Page 6: A systematic and strategic system for data collection management at Statistics Norway

6

Lean philosophy

• Is about– structuring,– operating,– controlling,– managing, and – continuously improving (our data collection)

Page 7: A systematic and strategic system for data collection management at Statistics Norway

7

Action plans and Lean

• Purpose: Make a environment for– reducing costs– increase efficiencies, and– improve our products/statistics

• How?– Reducing “waste”– “Only one time”– Organize and standardizes

Page 8: A systematic and strategic system for data collection management at Statistics Norway

8

Why action plans?

• A tool for designing and manage data collection within an allocated resource limit

• A tool to monitor that actions leads to better quality given a set of resources

• Documentation

• But most of all, a tool for making a (different) culture around a data collection

– Improved competence through more systematically and strategic approaches

Page 9: A systematic and strategic system for data collection management at Statistics Norway

9

What is a action plan?

• A description of the planning and implementation of the set of process that make a data collection

– Quality indicators– Demming’s quality wheel (PDCA)

• A preparation for both anticipated and unexpected challenges

• A documentation– Of actions and results

• A communication strategy– Who is responsible for what and

when?

• A monitoring system– indicators

• A guidance– For next wave or a different survey

Plan

Do

Adjust

Check

Page 10: A systematic and strategic system for data collection management at Statistics Norway

10

Action plans

• Goals and key figures

• Actions– Before and during the field period– What to do, when and who?

• Indicators– response rate, – contact rate, – recruitment rate, – r-indicator etc.

Page 11: A systematic and strategic system for data collection management at Statistics Norway

11

Goal

Er målet nådd?

2. januar. Start

ja

9. januar intervju 640

nei

16. januar 920 intervju

Ja

30. januar 1140 intervju

Ja

6. februar 1200 intervju Ja, 1227

Mål 60% intervju (anslag på ca 25 avganger) , utvalg 2000, 16-79 år. Antall intervju 1200Antall minutter:  11 minutter i gjennomsnitt.   

Page 12: A systematic and strategic system for data collection management at Statistics Norway

12

Actions before the field period starts

TILTAK Hvordan? Når?Innen

Status og vurdering

Før utsending

IO-brev med noteringshefte Brev sendt ut, Marit 29.11.2011 Ok

Påminningsbrev Brev sendt ut, Marit 28.12.2011 Ok

Sende telefonbrev til de med fasttelefon Brev sendes med påminningsbrev, Marit

28.12.2011

Nummerkobling Telefinn og DM-huset, Thore Uke 48 1644 telefonnummer (82,2%)

Manuell sporing 356 lagt inn i siv + eget excel-ark for sporing, Arnhild

Uke 48. 5 IO til frafall. 351 forsøkt sporet. Fant 262 (74,6%)

Utsending IO-brev til ”returer” Oppdaterte adresser og sendt påminningsbrev, Marit

28.12.2011 OK

Oppstartmøte feltplan. Kom vi greit i gang? Bemanning CATI/lokale

Marit, Arnhild, Tor

Page 13: A systematic and strategic system for data collection management at Statistics Norway

13

Actions during the field period, part 1

Etter utsending

Start Jytte 2. januar 2000 IO lastet i SIV.1906 m/tlf (95,3%)17 nektere, 1 langvarig syk og 2 avganger før start (1%)

Sjekk av cati-management innstillinger Rune 2. januar Ok

Sjekke at riktige variabler kommer opp i ringebildet / intervjuerlønn kan føres

Arnhild/intervjuer 2. januar Ok

Oppdatering av Intervjuersiden Marit/Arnhild 2. januar ok

Brifing, Oslo + Supervisorer Marit/Arnhild 2. januar Ok

Brifing, Kvg. Marit/Tor 3. januar Ok

1. sjekkpunkt. Er vi i rute? Antall statuser, intervju, timeforbruk. Mål 640 intervju (400t)

8 intervju per økt 9. januar 317 timer, 590 intervjuer

Sjekkpunktsmøte Marit/Bengt 11. januar

CAPI IO uten nummer sendes til lokale intervjuere. Lillian

9. januar

Page 14: A systematic and strategic system for data collection management at Statistics Norway

14

Actions during the field period, part 2

2. sjekkpunkt. Er vi i rute? Antall statuser, intervju, timeforbruk, Mål 920 intervju 350/(350t cati/ 50 timer capi)

5 intervju per økt, supervisorMarit/Bengt

16. januar 907 intervjuer, 197 timer

1. runde nektere Utvalgte intervjuere (bemanningstiltak). Arnhild

23. januar Sendes ut 19 jan219 IO

SMS Alle NB! Kode 35Jytte/Arnhild

16. januar Fremskynder fra 23. januar til 16. januar-OK

CAPI - oppfølging Lillian 23. januar

3. sjekkpunkt: Er vi i rute? Antall statuser, intervju, timeforbruk. Mål om 1120 intervju (250t cati/50 capi)

3 intervju per øktSupervisorMarit/Bengt

30. januar 1130 intervju

2. runde nektere Utvalgte intervjuere (bemanningstiltak), Arnhild

6. februar

SMS Jytte/Arnhild 30. januar

Ferdig. Mål: 1300 intervju 250t/1250t) 6. februar 1227 intervju (61,9%), 948 timer

Evaluering av tiltaksplan + ny HP Marit/Bengt 20. mars

Page 15: A systematic and strategic system for data collection management at Statistics Norway

15

Indicators

Goal Last quarter Check 1 Check 2 Check 3 Result

Response rate 60 58 62

Contact rate 75 75 75

Recruitment rate 80 78 82

R-indicator 0.8 0.74 0.71

Page 16: A systematic and strategic system for data collection management at Statistics Norway

16

Results, Omnibus

• Shorter field period – from 6-7 weeks to 5

• Less interviewer hours

• Standardized procedures

• Steady state

• No jumping!

2010 2011 2012*Contact rate 79 77 74Recrutement rate 70 76 78Response rate 56 58 58

R-indicator 0.77 0.74 0.76

Page 17: A systematic and strategic system for data collection management at Statistics Norway

17

Results, LFS

• Shift in ‘mode’ between 2. and 3 quarter 2011 (from decentralized CATI to centralized CATI)

– New challenges (questionnaire, contact strategies)

• Standardized procedures

• Steady state

• No jumping!

1. quarter 2011 2. quarter 2011 3. quarter 2011 4. quarter 2011 1. quarter 2012 2. quarter 2012Contact rate 0,90 0,9 0,88 0,86 0,85 0,86Recrutement rate 0,96 0,94 0,95 0,95 0,95 0,95Response rate 0,864 0,846 0,836 0,817 0,8075 0,817

R-indicator 0,81 0,81 0,82 0,82

Page 18: A systematic and strategic system for data collection management at Statistics Norway

18

Conclusions

• Standards for planning, communication, reporting, correction, documentation and evaluation of the field period

– Especially surveys with less complexes structures, few involvements (persons), or high degree of repetitive actions

• Further work– Look for more similar iterative processes in different surveys– Work more with the teams (communication and standardization)– Not only steady state, but also continuous improvement

Higher response rate Lower bias Lower cost

Page 19: A systematic and strategic system for data collection management at Statistics Norway

19

Thank you for your attention!

Bengt Oscar Lagerstrøm, [email protected],Statistics Norway