Embed Size (px)
Citation preview
SURVEYS
Sherry Woosley & Cindy Miller
Overall
Surveys
Survey Planning
What are you
trying to accomplis
h?
Surveys should be used to:
Explore attitudes, opinions, experiences, expectations, and needs
Gather information from and about large populations
Make comparisons among subgroups of the population
Compare results from year to year Gather statistically representative data
Surveys should not be used:
For audiences that are uncomfortable with numbers/statistics
For small numbers of participants
Without a clear understanding of the issues
When investigating issues of a sensitive or intrusive nature
Advantages
Can gather information from large numbers
Results may be generalizable to a larger population
Allow for statistical analysis that examines relationships among variables
Disadvantages
Important issues can be overlooked on surveys when the questions and responses are predetermined.
The quality of survey data is strongly dependent on the survey design.
Response rates and response bias are difficult to control.
Frequently Asked Questions – Other Big Picture Issues
Surveys
FAQ – What about an on-line survey?
To decide think about: Who is in your population? What is in your survey?
Do students care about this topic? How long is the survey?
What opportunities do you have to administer the survey?
What resources do you have?
On-line surveys do NOT necessarily have lower response rates.
FAQ – How many respondents do I need?
The important issues are: What percentage of people didn’t respond? How well do your respondents match your
non-respondents? Do you have enough respondents to do the
analysis you want?
There is no magic number! In most cases, a sample size calculator
does NOT adequately answer this question.
FAQ – What types of analysis are appropriate?
1. Descriptives: Looking at how students responded
2. Differences: Comparing groups of students
3. Relationships: Looking at which variables may be related
4. Change: Looking longitudinallyThe type of analysis depends on (1) the questions being asked and (2) the statistical
skills of the researcher.
FAQ – What is included in a survey report?
Need or problem and research questions Survey characteristics Survey methods (administration, sample,
analysis) Results Conclusions Implications
Audience needs will determine the level of detail included in a report.
Questions / Items
Surveys
Good Survey Questions
Wording should be simple, clear, direct, non-ambiguous, concrete, and uniformly understood.
There is no one “correct” response scale for all survey items.
Question Types
Open ended Single response Essay/paragraph
Closed Categorical (categories) Scales (Likert) Rankings Choose multiples or choose only one
Good Surveys…
Use neutral language Generate a variety of responses Use simple sentences (avoid compound
sentences and multiple phrases) Consist of only one question (beware of
double-barreled questions)
Bad Surveys…
Ask things you already know Ask things that your respondents cannot
answer Be too intrusive or personal Include
universals (always, all, none) limiters (only, just) double negatives abbreviations unconventional phrases
Bad Examples – Can you identify what went wrong?
How many times did you call your parents last year?
How much time does your teacher spend preparing for class?
How often do you eat donuts and/or drink coffee?
How often have you been harassed on this campus?
How often have you participated in ACCESS?
How often are you merely late for a class?
How often do you miss class and feel bad about it
afterwards?
How often do you never miss a class?
Strategies to Ensure Good Questions
Have the questions reviewed by experts Have the questions reviewed by
potential respondents Have the questions reviewed by
colleagues Adopt or adapt questions that have been
used successfully on other surveys Pilot test surveys
Discussion
Your turn… What did you bring? What questions do you still
have? How can you share your
expertise with others?
Questions?
Thanks for attending today’s session. More questions? Feel free to contact us.
Sherry [email protected]
Cindy [email protected]
