6/3/2015Bill Reimer1 Sampling and Questionnaires Bill Reimer October 31, 2005 [email protected]

04/18/23 Bill Reimer 1

Sampling and Questionnaires

Bill ReimerOctober 31, 2005

[email protected]://reimer.concordia.ca/teaching

mailto:[email protected]

http://reimer.concordia.ca/teaching


Outline Sampling Questionnaire construction


Sampling Objective Get a lot of information with little

cost

PopulationSample


Sample sizes for a population of 10,000

96 150 267600

2401

166 259461

1037

4147

0500

10001500200025003000350040004500

10% 8% 6% 4% 2%

Range of acceptable error (Accuracy)

Sam

ple

Siz

e

95% Confidence99% Confidence


3 Rules of Scientific Sampling

each unit in the population has an equal chance of being chosen

we must know the chance of each member being chosen

each selection must be independent from the others


To what do you want to generalize?

People Households Trees Leaves

These are the Units of Analysis


To which units do you want to generalize?

People in Canada? Women in Québec? Citizens of Montréal? Trees in boreal forests?

This is the Population


The Population

Is the collection of units to which you want to generalize

It may be abstract It may be concrete

But it must be clear


From what will you choose your sample?

A list of units? A region or place? A drawer or computer file? A particular time?

This is your sample frame


The Sample Frame may not match the Population

Population

Sample Frame

Units in th

e SF but

not the P

Uni

ts in

the

P

but n

ot th

e SF

…but try to get them close


Sampling Requirements:

Identify the population Is the sample frame reasonable? Does it meet the 3 conditions of

scientific sampling?

We can seldom meet all 3 conditions!


Types of Samples

Probability Generalizability Efficient

Non-probability Exploratory Strategic


Simple Random Sampling

Equal chance of being chosen

We know the chance

Each selection independent

From: O’Leary, Z (2004) The Essential Guide to Doing Research, London: Sage, Ch 8.


Systematic Sampling

Determine sample rate (8/40)

Random selection of 1st case

Every nth case (n=5) Watch for

regularities in the sample frame


Random selection


Stratified Random Sampling

Separate sample frame into strata

Take random (or systematic) sample from each strata

May be proportionate or disproportionate



Cluster Sampling

Divide population into clusters

Randomly select clusters

Collect data on all cases in cluster

Watch cluster composition



Cluster Sampling - Montréal


Non-Probability Samples Accidental Purposive


Convenience Sample

Choose most convenient people

Subject to multiple biases



Volunteer Sampling

Ask for volunteers Directly Ads Selected groups

Unclear biases



Strategic Sampling

Handpick respondents for strategic purposes

Useful for marginalized or small groups

Control biases by specifying criteria



Snowball Sampling

Identify strategic respondent

Ask for referrals Continue the

process Use multiple starts

to avoid network biases



Probability Samples – Multiple Mixture of various techniques


Sample Size Depends On

Size of the population Accuracy desired Confidence desired Variation in the phenomenon

investigated


Assignment Specify a research question you would like to

answer. Describe a sampling procedure along with

the data you would collect to answer the question.

Identify the following: the population the sample unit the sample frame the type of sample


Samples Non-probability

Accidental Purposive

Probability Simple random Systematic Stratified:

• Proportionate• Disproportionate

Cluster Multiple


Constructing Questionnaires


Types of Research Interviews Questionnaires

Closed and some open-ended questions Semi-structured interviews

Open and some closed-ended questions Unstructured interviews

Open-ended questions

Research Interviews are NOT what you see on TV


Advantages and Disadvantages

Questionnaires Economic Speedy Minimal

interviewer bias Anonymity Low response rates Limited exploration Comparison easy

Semi-structured High response rates Can ask

complicated questions

Can follow-up Qs Can observe Expensive Time consuming Comparison difficult


Preparing Questions Resist the temptation to start with

specific questions Clarify the RESEARCH question Identify the types of information

required to answer the research question

Within each of the types of information brainstorm for specific questions to ask

Rearrange questions


Question Order Leave sensitive issues for later - once

rapport has been established Follow from general to specific Use tables and grids where appropriate Use cards for sensitive issues if

appropriate Prepare your probes if interview Follow a natural flow


Anticipate the Analysis

Produce dummy tables Work backwards from your analysis


Format examples Contingency question Full filter question Quasi-filter question Closed question Partially open question Open question Probes


Simple, Mutually Exclusive What is your gender? (Circle the

appropriate number) Female . . . . . . . . . . . . . . . 1 Male . . . . . . . . . . . . . . . . . 2

5


Multiple Choice, not Mutally Exclusive What modes of transportation did

you used to get to school last week? (Circle all that apply)

• Metro . . . . . . . . . . . . . . . . . 1• Bus . . . . . . . . . . . . . . . . . . . 1• Car . . . . . . . . . . . . . . . . . . . 1• Bicycle . . . . . . . . . . . . . . . . . 1• Walk . . . . . . . . . . . . . . . . . . 1• Other (please specify)

_____________________ 1

6

7

8

9

10

|__|__| 11-12


Rank-order Rank order the 3 most important aspects of

your job (Place 1 beside the most important, 2 beside the next most important, and 3 beside the next most important)

• Money . . . . . . . . . . . . . . . . . . |___|• Security . . . . . . . . . . . . . . . . .|___|• New experiences . . . . . . . . . . |___|• Power . . . . . . . . . . . . . . . . . . |___|• Excitement . . . . . . . . . . . . . . |___|• Other (please specify)

________________________ |___|

13

14

15

16

17

|__|__| 18-19


Pretest, pretest, pretest Use your friends and family Discuss

Where it was clear or not clear What they thought about when

responding to each question What they thought you were getting at

Move to colleagues Check it out on people who are similar

to those you will eventually survey

Problem Questions Jargon, slang, abbreviations

What is your Internet browser? Ambiguity, confusion, vagueness

What is your income? Emotional language and prestige bias

Should we put murderous terrorists in jail? Do you support Prime Minister Chretien’s

policy on world trade? Double-barrelled questions

Should marijuana be legalized for medical or other purposes?


Problem Questions – con’t Leading questions

Do you feel that governments should have less power to interfere in business?

Beyond respondent’s competence Does your mother support extramarital

sex? False premises

How can we halt the rising crime rate?


Problem Questions – con’t Future intentions

How will you vote in the next federal election?

Overlapping or unbalanced response categories What is your opinion regarding abortion?

• Very favourable• Favourable• Neutral• Opposed


Questionnaire Construction The amount of time spent in the

preparation of the questionnaire is directly related to the value of the results.

Surveys usually cost a lot and you can't redo them.

Know what you are wanting to discover


Research Strategies Questionnaires for generalizability Exploration: well selected, in-depth

interviews Use interviews to design survey Use comparisons


Sampling and Questionnaires

Bill ReimerOctober 31, 2005

[email protected]://reimer.concordia.ca/teaching

Documents

6/3/2015Bill Reimer1 Sampling and Questionnaires Bill Reimer October 31, 2005 [email protected]