How you can get the best out of your next survey questionnaire

QuesTReviewTM

Your Solution to Excellent Questionnaire Design

Copyright HealthSurveySolutions 2016. All rights reserved


Introduction QuesTReviewTM Whether your next survey is paper, web-based or a mobile, satisfaction or

research survey, keeping your survey respondent engaged and is providing

complete and valid information, it is essential the questionnaire has been well

designed. That’s why at HealthSurveySolutions we’ve developed our new

diagnostic tool QuestReviewTM that lets you know whether survey

questionnaire will provide credible evidence or not before going into the field,

saving you both time and money.

Grounded in recognised psychological and behavioural models of how

respondents complete survey questionnaires, QuesTReviewTM (incorporating

our proprietary QuestAnalyzerTM diagnostic test) benchmarks the

questionnaire against key parameters of good questionnaire design e.g.

wording, sensitivity, appropriate response options etc. and quantitatively

coded. We use this information to create both qualitative and quantitative

feedback for the developer to provide an optimal version of the questionnaire

prior to going into the field.


http://www.healthsurveysolutions.com/

http://www.healthsurveysolutions.com/questreview.html




How QuesTReviewTMworks

QuesTReviewTM undertakes a detailed review of the questionnaire against 20 key

parameters of good practice questionnaire design, including:

Reading ease

Clarity

Knowledge/memory demands

Response options

Each parameter comprises a number of sub-categories against which the questionnaire

is assessed and quantitatively coded. For example, the clarity parameter comprises sub-

categories including word length, ambiguity, use of technical terminology. etc.


Feedback on the quality of the questionnaire is provided with the following:

1. Traffic light rating system for each question/item by parameter combined with a detailed qualitative description of the identified

weaknesses for each question/item.

2. Average score across all parameters by individual question – providing an overview of each question/item performance and need for

revision.

3. Checklist for mobile and PC application.

4. Survey difficulty – Estimated time to complete the survey

5. Overall performance rating


Findings Summary

1 2 3

No defects in the wording and or response

options of the question/item have been

identified.

Defects in the wording and or response

options. It is strongly recommend revisions are

carried out.

Major defects in the wording and or response

options of the question/item have been

identified and must be addressed prior to

translatability assessment.

Question length Use of technical terms Reference period Complex question

Flesch Reading Score 25. Very hard

to read. Question length is 37

words long. Flesch score of 65 is

acceptable. Reduce to a maximum

of 21 words.

‘Health care provider’ is

an undefined technical

term. Provide a list of

examples, e.g. practice

nurse, dietitian etc.

The “past year” is unclear.

Define clearly: e.g. “The past

twelve months.”

The question is asking two

questions in one i.e. pain and

breathing. Split the question

into two for pain and breathing.

Quest 6. Parameter: Question/item clarity


Figure 1. Sample feedbacks: Traffic light rating system for question/item by parameter

Flesch Reading Score 25. Very hard to read. Average sentence is 37 words long. Flesch score of 65 is acceptable.

First sentence does not match the first question.

Rewrite strongly recommended.

Quest 6. Parameter: clarity of instructions Flesch reading score Conflicting/inaccurate Complicated instructions Instructions instructions


Sensitive content (General) Sensitive words (Specific) Socially acceptable

Quest 12. Parameter: Question/item sensitivity

The developer has used a unbalanced scale, i.e. that there is an odd number of points on the low, or negative, side compared to the high, or positive, side. Unbalanced scales should be used with +/- response skew. Determine if responses will be skewed.

Quest 9. Parameter: Question response options Technical terms Vague Overlapping Missing Balanced/Unbalanced scale

Quest 14. Parameter: Assumptions in question/item Inappropriate assumptions Assumes constant behaviour in wording of the question/item or experience

Inappropriate assumptions about respondent's behaviour/experience.

Question assumes constant behaviour for situations that can vary.

0 1 2 3 4 5 6

Q1.

Q4.

Q12.

Q1.

Q9.

Q5.

Q2.

Q7.

Q6.

Q11.

Q10.

Q8.

Question No

Essential to undertake major revisions in accordance with feedback

Defects in the wording and or response options. It is strongly recommend revisions are carried out. .

No revisions required

Average rating

Figure 2. Average score across all parameters by question


Figure 3. Checklist for mobile and PC application

Criteria Yes No

Designed for both mobile portrait (vertical) and landscape (horizontal) view

Motivational welcome screen……………………………………………………………………………

First question/item easy to understand…………………………………………………………….

Multiple questions per screen…………………………………………………………………………..

Use of grid format questions…………………………………………………………………………….

Use of scrolling design

Text boxes as large as possible………………………………………………………………………….


Survey difficulty – Estimated time to complete the survey


How Well Does Your Questionnaire Perform?


What else can HealthSurveySolutions do for you?


For more information on QuesTReviewTM contact us at: [email protected] Tel: 44 (0) 1295 724233

mailto:[email protected]

Marketing

How you can get the best out of your next survey questionnaire