Conducting the Survey

1Module 2, Unit 5, Session 6

SESSION OBJECTIVES

• Practice determining sample sizes for all of your target audiences

• Roughly identify where the interviews will take place

• Identify standard practices for training enumerators to implement the survey

• Minimize bias and error during survey implementation

Module 2, Unit 5, Session 6 2

By the end of this session, campaign managers should be able to:

3

Qualitative Research in 4 Steps

1. Conceptualization • Problem you want to solve and scope of research

2. Research Design• Survey Design (completed) | Survey Settings (location)• Survey Sampling | Sample Size

3. Data Collection • Conducting the survey

4. Data Analysis and Reporting• Analyzing results and measure effectiveness of survey

4

1. Conceptualizing your survey

5

Conceptualizing your survey

Conceptualizing includes defining:

• The problem you want to solve

• The scope of your research project

6

Scope:

The scope in survey research is determined by the target population, or the group of people that you want to study with the survey.

7

Considerations When Determining Scope

Who do I survey, what to consider?

• Geography (location and proximity to MPA)• Age range• Gender• Employment• Gender• Socio-economic status

2. Research Design

8

Survey Design (completed) | Survey Settings (location)Survey Sampling | Sample Size

Designing your Survey

• Setting: where do I survey?

• Survey Sampling: who/what group should I survey?

• Sample Size: how many do I survey?

9

Survey Setting: where do I survey? Four Types of Setting: (Millard Brown Market Research, Ltd.)

1. Door-to-Door: best way to guarantee random and representative sample. Restriction: more time and resource required. (consensus data)

2. Other Defined Place: Conduct surveys in multiple places. Most useful in finding larger number of respondents. (business, parks, shopping centers, street corner, markets, other public places)

3. Pre-determined Location: Conduct in very specific locations to target a defined audience and excludes all other. (arriving tourists at airport, fishermen at docks)

4. Central Location Tests: specific location to conduct survey (at the end of workshop or presentation)

10

Module 2, Unit 4, Sub-Unit 1.1 11

Survey Sampling: Who/what group should I survey in population?

12

Designing your Survey: Sampling

Module 2, Unit 4, Sub-Unit 1.1 13

Designing your Survey: Sampling

Designing for your Survey: SamplingMost important and difficult part of survey

• Impossible to interview everyone in your area (at most sites)

•You want to select a “representative” sample

•The best way is though a random sampling procedure

•No magic number to tell you how many people you need to interview

14

Designing your Survey: Randomize Sampling

15

Designing your Survey: Randomize Sampling

16

*Random Sampling: every person in your target population has an equal chance of being selected to be in the sample:

Ways to randomize your sample:(Target Population must first be defined)

1. Simple Random Sample: choosing samples from a single pool of the population (picking from a hat)

2.Stratified Sampling: You would divide this large pool of subjects into several groups (strata) and then randomly select subjects from within each group. The number of subjects selected from each group is fixed by design.

17

Population

Brgy A Purok A1

Purok A2

Brgy BPurok B1

Purok B2

Brgy C Purok C1

Purok C2

Brgy D Purok D1

Purok D2Stratified Sampling: Understanding Strata

Strata (level 1)Strata (level 2)

Selecting your Sample Size• No magic number to get “valid” or “statistical significant” result. • Three Things to Take when Choosing Sample Size:

– Population Size– Confidence Level: tells you how sure you can be you that your

sample is generally representative of the population . – Confidence Interval: reflects your probability of having a sampling

error in your survey.

• General Trends for CL, CI and Sample Size: a) The larger your sample size, the smaller your confidence interval (less likely to make sampling error). b) Confidence level: The larger your sample size, the more sure you can be that their answers truly reflect the population (higher confidence level). c) Standard CL is: 95% or 99% | Standard CI is: 2.5% or 5%

18

19

Target Population Size Confidence Interval

95% Confidence

Level99% Confidence

Level50,000 5 percentage points n = 381 n = 657

3 percentage points n = 1,045 n = 1,7832 percentage points n = 2,291 n = 3,8411 percentage point n = 8,057 n = 12,486

100,000 5 percentage points n = 383 n = 6613 percentage points n = 1,056 n = 1,8152 percentage points n = 2,345 n = 3,9941 percentage point n = 8,763 n = 14,267

250,000 5 percentage points n = 384 n = 6643 percentage points n = 1,063 n = 1,8352 percentage points n = 2,378 n = 4,0921 percentage point n = 9,249 n = 15,062

1,000,000 5 percentage points n = 384 n = 6653 percentage points n = 1,066 n = 1,8462 percentage points n = 2,395 n = 4,1431 percentage point n = 9,513 n = 16,369

EXAMPLE:

20

5

15

25

45

35

0Pre-campaign Post-campaign

K l

evel

%How to know if statistically different?

X X

How do we calculate for sample size?

http://www.surveysystem.com/sscalc.htm

21

• Municipality of Bebeh = population (N) of 2500• Composed of 4 barangays• Sample size (n) = 333

22

Stratified Sampling: Example

Barangays Population size

Percent of Target Area Population

(%)Ideal sample

size

Arman 400 16 53

Bebeh 800 32 107Cathy 1100 44 147Dagsoy 200 8 27

Total 2500 100 334

Quick exercise

• Municipality of Jongoland = population (N) of 700• Composed of 3 barangays• Sample size (n) = 248

23

Barangays Population size

Percent of Target Area Population

(%)Ideal sample size

Arman 150 21.4 (22) 54.6 (55)

Labs 380 54.2 (54) 133.9 (134)Ruby 170 24.3 (24) 59.52 (60)

Total 700 99.9 (100) 248 (actual 249)

Let’s go back to scope: Questions to consider

• Do you want equal representation between men and women?– Implication: know what time/s of day to survey

• What survey setting should I use? (door-to-door, any other defined place, etc.)– Implication: where to specifically find subgroups

of target audience (i.e: fishermen)

24

Experimental Design – using a comparison group

25Module 2, Unit 4, Session 8

Treatment/Intervention/Pride Campaign

Experimental Design

Choosing a Control Group: 1. Similar Demographic and Scope: Must be close enough

to experimental group so that culture, environment, language, and other factors are very similar.

2. Unexposed to Campaign Activity: Far away enough so that people in the control group are unlikely to be exposed to any of the campaign (intervention) activities.

*note for those who read pre-reader: comparison group = control group

26

3. Data CollectionActually conducting the survey

27

Preparing for your Survey: Enumerators• Teachers• University students• Dept. of Statistics personnel• High school students• Lead Agency personnel• NGO volunteers

• Hopefully for free: minimum to keep enumerators engaged (lunch, transportation, small celebration). Money spent on this is money not spent on campaign material and activities only (core funds).

28

Preparing your enumerators• Ideally, 5-6 enumerators• Level off

– Clarify and standardize group understanding of questions and procedure of administering survey (i.e: asking questions, randomizing, time, etc.)

• Pre-test the survey questionnaire to iron out any bugs

• Prepare schedule (date, times, jurisdictions, survey settings)

• Refer to reader for other tips29

Conducting your Survey

• Pleasant personality

• A professional manner

• A good listener

• A trusted person

• Language proficient

• Well presented

• Be mindful of time

• Taking “no” as an answer? – ideal response rate is 80%

• Refer to reader for other tips 30

The Mini-Campaign in Practice and Roadblocks

31

TIME TO BE KAPALMUKS!

BAGA’G NAWONG!

32

• In groups of 4, design one question (single answer).

• Try to randomly interview at least 2 people.

• Compare notes.

33

Training and Rules for Interviewers• Be courteous.• Maintain confidentiality of the interview at all times. • Introduce the survey by saying who is running it, its general intentions,

and how the respondent was chosen (usually at random).• Be professional, have all your materials ready and keep to the purpose of

your visit.• Be familiar with the questionnaire so if there are filters and skip patterns,

you know where they lead. • Read each question exactly the same to each respondent.• Speak slowly and clearly so you can be understood.• Don’t mark an answer until the respondent states it.• If a response is incomplete, use a neutral probe, to get the respondent to

fully answer the question.• Before leaving, make sure the questionnaire is completed.• Thank the respondent and leave.

34