View
50
Download
1
Category
Tags:
Preview:
DESCRIPTION
Response quality in the Structural Business Survey questionnaire. Deirdre Giesen & Joep Burger Q-Conference, June 2-5 2014, Vienna. Outline. Why this research project? Some background on SBS Previous results and current step Data used Quality indicators Results Issues for discussion. - PowerPoint PPT Presentation
Citation preview
Deirdre Giesen & Joep BurgerQ-Conference, June 2-5 2014, Vienna
Response quality in the Structural
Business Survey questionnaire
Outline
• Why this research project? • Some background on SBS • Previous results and current step• Data used• Quality indicators• Results• Issues for discussion
Purpose research project
1. Develop quality indicators for quality “raw data” of business survey.
2. Explore how this quality is related to – characteristics response process– data collection design
Possible use of such indicators
– Allow quick response during data collection (also: direct feed back to respondents during completion of electronic questionnaire).
– To guide further research and adjustments in design data collection.
Structural Business Survey
– Annual survey– 2nd largest business survey at SN (about 70000
business units in sample)– Collects detailed specification of income and costs
of businesses– Most industries and size classes included– Used for European SBS and Dutch National
accounts– About 200 versions of the questionnaire: common
core part with size-class/industry specific specifications
– Redesign introduced in 20065
Our previous research (1/2)
– General quantitative indicators developed that can be used for all types of SBS:‐ Item response to 5 core variables‐ Set of consistency rules ‐ Use of “other costs” item
– Related these to time (before / after redesign), size of business, type of industry, timeliness response, mode, response burden
6
Our previous research (2/2)
– Overall quality of indicators was rather high– Not many systematic and large differences
related to background variables studied however:‐ Smaller businesses perform worse than larger
businesses‐ Some improvement on item response and
consistency after introduction of electronic questionnaires
‐ After redesign in question block on personnel one core variable clearly higher item response, other clearly worse.
7
Current step of research
– Focus on one unique questionnaire, which allows detailed analyses of the quality of cost specifications.
– Selection: general manufacturing questionnaire for size class 5 ( 20-49 employees).
– For this questionnaire: pilot study with introduction of electronic questionnaire in 2005, introduction of total redesign in 2006.
– Data for 2003-2007 analysed, about 1400 respondents each year.
8
Example of calculation in SBS qnr
9
Quality indicators
1) Quality of calculation (6 sets defined)2) Overall item response: fraction of subitems used 3) Item response on set of 11 core variables4) Fraction of total value specified for each core variable.
10
Quality of calculations
11
Overall item response (costs)
12
Item response core variables
13
Fraction of total value specified
14
– Demonstrates methodology to quickly assess quality of this type of raw data.
– Seems useful to test if and how indicators can be used for - individual feed back to respondents.- standard monitoring of data collection.
- Illustrates that also for “hard business data” questionnaire design affects response process.
- Limited use of indicators for evaluation of design characteristics without experimental data.
15
Conclusions
Questions, ideas?
– What aspects of quality in raw data in business surveys do you monitor and/or feed back to respondents?
16
Recommended