Dillman on Survey Instrument Design

Professor Ron FrickerNaval Postgraduate School

Monterey, California

Excerpted from Mail and Internet Surveys: The Tailored Design Method, second edition, by Don A. Dillman, 2000. 1

Goals for this Lecture

• Define and consider Dillman’s principles for survey instrument design– Make surveys easy for respondents to follow and

answer• Location of instructions• Formatting, including use of white space• Navigational cues• Answer placement, etc.

Advance our understanding of what makes a good survey instrument

2

Visual Consistency and Navigational Clarity Very Important

• Dillman:– “Define a desired navigational path for reading all

information presented on each page of the questionnaire.”

– “Create visual navigational guides and use them in a consistent way to get respondents to follow the prescribed navigational path and correctly interpret the visual information.”

• Make it easy to follow in order to minimize respondent errors and drop-outs

3

For Paper-based Surveys…

• …Dillman recommends:– Print on 8½” x 14” paper and fold into 8½” x 7” booklets– Print on11” x 17” paper and fold into 8½” x 11” booklets– Print on single side of 8½” x 11” paper and staple in upper

left corner

• Last option lacks a professional look but is often cheaper to produce

• Of course, these paper-size issues don’t apply to web-based surveys– But clarity and consistency of the instrument design are just

as important– Principles that follow are just as applicable to web surveys

4

Survey Format Example

55

Survey Format Example

6

Double column format using 8½” x 11” paper

Make the Survey Easy to Follow

7

A Survey that is Hard to Follow

8

Put Instructions Where Needed(1 of 2)

9

Put Instructions Where Needed(2 of 2)

10

Good First Questions are Critical (1 of 2)

1111

Good First Questions are Critical (2 of 2)

12

Only Ask One Question at a Time (1 of 2)

13

Only Ask One Question at a Time (2 of 2)

14

For Qs with Common Responses, Use Item-in-Series Format (1 of 2)

15

Item-in-Series Format (2 of 2)

16

Minimize the Use of Matrices

17

Use Visual Elements to Improve Readability

18

Use Spacing and White Space to Appropriately Group Elements

19

Keep the Format Consistent (1 of 2)

20

Keep the Format Consistent (2 of 2)

2121

Include Instructions as Part of Question When Possible (1 of 2)

2222

Include Instructions as Part of Question When Possible (2 of 2)

23

Answer Category Placement (1 of 3)

24

Answer Category Placement (2 of 3)

25

Answer Category Placement (3 of 3)

26

Put Answers in One Column

27

Be Consistent in Scale Ordering

28

Use Visual Cues to Guide Respondents (1 of 2)

29

Use Visual Cues to Guide Respondents (2 of 2)

30

Emphasize Words Sparingly

31

Which Survey Would You Prefer to Take?

32

?

What We Have Covered

• Discussed Dillman’s principles for survey instrument design– Goal: Make surveys easy for respondents to follow

and answer• Location of instructions• Formatting, including use of white space• Navigational cues• Answer placement, etc.

Advanced our understanding of what makes a good survey instrument

33