Writing survey questions

What do you want to know?

Before you begin to compose your survey questions, you understand the purpose of your survey and you need to keep this purpose in mind every step of the way.

Ask yourself what it is you want to learn from the survey.

Think about how you will use the data that you collect.

Mentally walk through the steps you will take once the data have been collected. What do you expect to learn? How do you plan to apply what you learn? What are your goals and how will the survey help

you reach those goals?

Involve others when determining the purpose of your survey

Identify which people in the organization should have input on what should be learned from the survey.

Meet with them, either one-on-one or as a group to hear their ideas.

Involve others (cont’d)

You know what you want to learn at a conceptual level, now dig a little deeper.

Ask people what issues relate to each area of your survey.

Don't forget to ask both the people from step #1 above as well as the people who are dealing with these issues on a daily basis.

Involve others (cont’d)

Construct an outline of the issues and details that you collected in steps #1 and #2.

Step back and ask yourself whether it all fits together and whether anything might be missing.

Once you are satisfied with your outline, you are ready to begin writing your survey items

 Quantitative (Numeric) Data

Working from the outline that you created, break down each issue or detail into a single coherent, concrete thought or concept.

Look for items in your outline that might be dealing with two or more different ideas or concepts.

Split these up into separate lines or bullet points. Look for items that are vague or abstract and consider how they might be made more concrete.

Quantitative

Decide what kind of rating labels you want to work with.

Often, it does not matter so much which labels you choose.

What matters is that you use them consistently in the survey and phrase all items work with your chosen labels.

Quantitative

Keeping your rating labels in mind, turn each line or bullet point in your outline into a survey item.

Do this by re-phrasing each line in the form of a question or statement that works with the rating labels.

Quantitative

At this point, you have your initial set of survey items and chances are your survey is much too long at this point.

It is now time to start looking for ways to shorten the survey.

The balance between shortening the survey and making sure you get the data you need can be a tricky one.

Quantitative

After you have finished shortening the survey, look for ways to make each item more precise.

Chances are, many of your items are suffering in this area for any of the following reasons.

Complex wording or structure

• Compound sentences, complex vocabulary words, unusual terminology, idiomatic expressions - all these things will confuse some people and make your data less precise.

• Use simple sentences and vocabulary appropriate to your audience.

Vague or overly general questions

• Are any of your questions so broad that they will not give you specific, actionable information?

• If you ask yourself what you will learn from the responses, positive or negative, from each question, you will know whether they are too vague.

Items that could be misinterpreted

• For each survey item, ask yourself whether there are any ways in which it might be misinterpreted.

• Many words have different meanings to different people.

• References to imprecise concepts such as time and distance are subjective.

Questions with a "right" answer

• It is an easy and common mistake to write items that have a socially "correct" or desirable answer.

• Also be sure you do not lead your respondents to answer in a particular way by making them think you "want" them to provide a certain response.

• Your items need to be presented neutrally.

Double-Barreled Items

These are items that ask about more than one thing. If you see the word "and" or "or" in your item, chances are it is double-barreled.

Data from items like this are useless because you have no way of knowing what part of the question each respondent was thinking of when responding

Qualitative (Text or Open Ended; Non-Numeric) Data

Qualitative data presents an opportunity to get actionable information from your survey. The selection and composition of qualitative survey items is even more important than for quantitative items. This is because you cannot ask nearly as many items that require people to respond freely.

Qualitative

If possible, consider conducting a pilot survey or gather a focus group to identify potential areas in which qualitative data might be most beneficial.

Based on the responses you receive, you might notice some problem areas where more information from respondents would be quite useful.

Qualitative

When composing qualitative items, the same rules apply as above, but you need to also be sure the questions will provoke respondents to provide insightful information.

The worst kinds of qualitative items are those that ask a "yes/no" question or those that invite a specific (and often brief) answer.

Qualitative

The flip-side of the "yes/no" item is also a problem.

If you are too broad in your qualitative items, respondents will not provide the kind of specific information that would be most useful to you.

Qualitative

The best test to see whether a qualitative item is well written is to answer the question yourself.

If it invites you to give the kind of response that you are looking for, chances are the item will work for your survey.

Qualitative

Don't overdo it. You will get more thorough responses from a few well-written qualitative items than you will from many such items, regardless of how well-written they are.

It takes time for respondents to answer qualitative items and they will quickly grow tired of doing so and skip over such items or worse yet, not complete the survey at all.