Survey Training and LQAS

CARRY OUT SURVEY

Why should I do a survey & why should I use the LQAS method?

Where should I conduct my survey?

PRE-SURVEY

Who should I interview?

What questions do I ask and how do I ask them?

Uses of surveys

Random Sampling

Using LQAS sampling for surveys

Using LQAS for baseline surveys

Interview locations

Selecting households

Selecting respondents

Field practice for numbering and selecting households

Developing and reviewing the survey questionnaires

Interviewing skills

Field practice for interviewing

Planning for the data collection/survey

What do I do with the information I have collected?

Field work debriefing

Tabulating results

Analyzing results

POST-SURVEY

Standard Survey Chronology (with reference to Lot Quality Assurance Sampling--

LQAS)

A

B C

DE

Key Concepts: Program Area and Supervision Areas

Together, A, B, C, D, and E represent the Program Area

A, B, C, D, and E represent 5 Supervision Areas.

Why Survey/Why LQAS

Key Concept: Coverage

An important use of surveys is to measure coverage.

What is coverage?

COVERAGE is the percentage of people in any given area (a program area or supervision area) who

a) Know a key piece of information

b) Practice a recommended behavior or

c) Receive a particular service.

Why Survey/Why LQAS

Key Concept: Coverage

Knowing coverage enables us to...

Plan by allowing us to choose priorities.

To focus our efforts on improving those knowledge and practices that have low coverage.

Over time, repeated measures of coverage show us if our efforts are leading to improvements in coverage.

Additionally, knowing the coverage is especially poor in one or more supervision areas helps us choose priorities. We can decide to focus our efforts in those supervision areas with poor coverage.

Why Survey/Why LQAS

What Surveys Can Show You

Surveys can help you identify the level of coverage of the program area as a whole, AND if there are:

large differences in coverage regarding knowledge and practices among supervision areas

little difference in coverage regarding knowledge and practices among supervision areas

Why Survey/Why LQAS

Scenario One

Indicator: Percent of women (15-49) who know 2 or more ways to prevent HIV transmission.

Scenario Two Scenario Three

A = 30

B = 40C = 80

D = 75E = 20

A = 85

B = 80C = 90

D = 85E = 80

A = 25

B = 20C = 30

D = 25E = 20

Why Survey/Why LQAS

Uses of Surveys

Identify knowledge and practices with:

1. Large differences in coverage among supervision areas (SAs).

Identify the low-coverage SAs to be able to:•learn causes of low coverage.•focus our efforts and resources on these SAs.•improve coverage of the whole program area by improving coverage in these SAs.

Identify high-coverage SAs to be able to:•study and learn what is working well.•identify things that can be applied to other SAs.

2. Little difference in coverage among SAs.

If coverage is generally high, shift resources to improve other knowledge and practices.

If coverage is generally low:•learn causes of low coverage.•identify/study other program areas to learn what is working well.•identify things that can be applied in your own program area.

Why Survey/Why LQAS

Key Concept: Random Sampling

Sampling allows you to use the “few” to describe the “whole” (to generalize from the few to the whole)

AND

Random sampling is a critical way to improve your ability to generalize in this way (it improves “external validity”)

Why Survey/Why LQAS

Key Concept: Lot Quality Assurance Sampling

A = ?

B = 80C = ?

D = 75E = 45

Indicator: Percent of women (15-49) who know 2 or more ways to prevent HIV transmission.

A special type of random sampling that allows us to use small samples to distinguish between supervision areas in relation to coverage--to see if certain areas have much higher or lower coverage than others.

LQAS allows us to make comparisons between supervision areas AND estimate overall program coverage.

Why Survey/Why LQAS

Watch!

LQAS Sampling ResultsIndicator: Percent of women (15-49) who know at least 2 ways to

prevent HIV transmission.

Total black checkers in the bag

Total black and red checkers in the bag

= =

Verify coveragein the bag for

SA A

Total black checkers in the bag

Total black and red checkers in the bag

= = Verify coveragein the bag for

SA C

Why Survey/Why LQAS

Supervision Areas: Program Area

SA “A” SA “C”

Sample Sample Sample Sample

1 6 1 6

2 7 2 7

3 8 3 8

4 9 4 9

5 10 5 10

%

%

100

100

Why Survey/Why LQAS

“Limits” of LQAS

Let’s say we want ALL SAs to achieve the result that at least 50% of all women 15-49 know at least 2 ways to prevent the transmission of HIV.

If we take a sample of 19, what is the probability of misclassifying an SA as having achieved the target (using a decision rule of 7) or of NOT having achieved the target for various TRUE population rates?

True Population Proportion who Know 2 Ways to Prevent HIV Transmission in the SA

Probability of classifying the SA as having achieved

the target of 50%(based on n=19 with

decision rule of 7 or more who know 2 ways)

Probability of classifying the SA as not having

achieved the target of 50%(based on n=19 with

decision rule of 7 or more who know 2 ways)

15% 2% 98%

20% 7% 93%

25% 18% 82%

30% 33% 67%

35% 52% 48%

40% 69% 31%

45% 83% 17%

50% 92% 8%

55% 97% 3%

60% 99% 1%

70% 100% 0%

What a Random Sample of 19 Can Tell Us

• Good for deciding what are the higher-performing supervision areas to learn from

• Good for deciding what are the lower-performing supervision areas

• Good for differentiating knowledge/practices that have high coverage from those of low coverage

• Good for setting priorities among supervision areas with large differences in coverage

• Good for setting priorities among knowledge/practices within an SA (if one intervention area is high but another is low, we would concentrate on the low-coverage intervention)

Why Survey/Why LQAS

What a Random Sample of 19 Cannot Tell Us

• Not good for calculating exact coverage in an SA (but can be used to calculate coverage for an entire program)

• Not good for setting priorities among supervision areas that have little difference in coverage among them

Why Survey/Why LQAS

So…Why Use a Random Sample of 19?

• A sample of 19 provides an acceptable level of error for making management decisions; at least 92% of the time it correctly identifies SAs that have reached their coverage target.

• Samples larger than 19 have practically the same statistical precision as 19. They do not result in better information, and they cost more.

Why Survey/Why LQAS

One More Thing… LQAS in Baseline Surveys

Generally we think of using LQAS for ongoing monitoring to assess whether we have evidence that we are meeting pre-set targets. However, it can be used at baseline to assess whether certain SAs appear to be “lagging” behind the program area average coverage.

Supervision Areas A, B, C, D and E

Indicator: Women who know 2 or

more ways to prevent HIV transmission

Number Correc

t

Average Coverage Estimate

65.3%

Equal to or Above Average Coverage?

SA A 12 Yes

SA B 9 No

SA C 16 Decision Rule11

Yes

SA D 11 Yes

SA E 14 Yes

Five SAs & One Indicator

1. Add number correct in all SAs: 12+9+16+11+14=62

2. Add all sample sizes 19*5=95

3. Average Coverage Estimate 62/95=65.3% round up to nearest 5%=70%

4. Decision rule for sample of 19 and 70% is 11

n=Average Coverage (for baselines) or Coverage Target (for monitoring/evaluation)

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95

19 -- -- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Indicator Cor-rect out of 19

Pgm.Cove-rage Est.

Rule

Equal or

Above

Women “using” condoms w/ sex

7 45% 6 Yes

Men using condoms w/ sex

4 20% 1 Yes

Women know how HIV transmitted

4 45% 6 No

Men know how HIV transmitted

13 65% 10 Yes

Women who know where to get test

6 30% 3 Yes

One SA & Five Indicators

CARRY OUT SURVEY

Why should I do a survey & why should I use the LQAS method?

Where should I conduct my survey?

PRE-SURVEY

Who should I interview?

What questions do I ask and how do I ask them?

Uses of surveys

Random Sampling

Using LQAS sampling for surveys

Using LQAS for baseline surveys

Interview locations

Selecting households

Selecting respondents

Field practice for numbering and selecting households

Developing and reviewing the survey questionnaires

Interviewing skills

Field practice for interviewing

Planning for the data collection/survey

What do I do with the information I have collected?

Field work debriefing

Tabulating results

Analyzing results

POST-SURVEY

Standard Survey Chronology (with reference to Lot Quality Assurance Sampling--

LQAS)

Where and With Whom?

Where to Conduct the Survey and With Whom

These questions are context specific. The question “with whom” is determined by program participants whom you want to survey. And the question of “where” to find them will depend on the spread of your program and whether you have lists of participants from which you can draw a sample.

In the example we have been using, the “who” of our program is women 15-49 whom we want to help to prevent getting HIV. Still knowing the “who” is not the end of question because we still need to figure out how to take a random sample of them.

In most cases program participants are scattered in various communities or locales and thus we need a sampling method that gives everyone an equal chance of being selected.

Broadly speaking I have seen two scenarios in the field:

1. A situation in which all program participants are “registered” on a list that is updated regularly. This is common in programs like microfinance where “clients” are tracked.

2. A situation in which program participants are not registered--not known by name--and scattered around a variety of communities.

Let’s look at these two scenarios and examine how to randomly sample


A

B C

DE

Programs in Which Participants are “Listed”

Let’s go back to our 5 SAs and assume that, in this case, the 5 are “bank” branches of a microfinance institution. In each branch (each SA) lists of clients are maintained.

If we are doing an education program with members and want to learn if 50% in branch “A” know 2 ways to prevent HIV transmission we can take a sample of 19 from this branch.

To do this we start by assembling a numbered list (in a completely random order--NOT by community for example) of all client names.

Next we use some means (random number table or other) to select 19 numbers at random. The numbers must be between 1 and the highest number on the list.

We then decide when and how we will interview each client whose name corresponds with the number chosen.

Programs in which Participants are not “Listed”

(but for which we know all the communities in which they live and have a general idea of population size in each

community)

This case is common in community-based programs in which “all” members of a group (women 15-49) in a certain geographic area are intended beneficiaries

A

B C

DE

If we are doing an education program with women 15-49 in an entire program area and want to assess whether 50% of them in SA A (a geographic area) know at least two ways to prevent HIV transmission the approach is a bit more challenging and usually requires “multi-stage”, “systematic” sampling. Here are the general steps (to be repeated in each SA)

Step 1. List communities and total population.Step 2. Calculate the cumulative population.Step 3. Calculate the sampling interval.Step 4. Choose a random number.Step 5. Beginning with the random number, use the sampling interval to identify communities for the 19 sets of interviews.

This ONLY gives us the communities in which we need to sample! We need another step to identify the actual women we will interview (that is why we call it “multi-stage”)



Name of the Community

Population

Pagal 152

Santai 381

Ishri 115

Garafa 97

Nevi 253

Masrag 126

Farry 188

Jilwa 216

Guimbe 554

Kilkil 92

Total 2174

Step 1. List communities and total population.

Cumulative Population

152

533

648

745

998

1124

1312

1528

2082

2174

Step 2. Calculate

the cumulative population.

Step 3. Calculate the sampling interval=Total Cumulative Population/Sample Size (2714/19=114.42)

Step 4. Choose a random number between 1 and the interval -

Step 5. Beginning with the random number, use the sampling interval to identify communities for the 19 sets of interviews. X is the first then

X+114.42, etc.

Interview Location Number

77

191, 305, 420

534

649

763, 877, 992

1106

1221

1335, 1450

1564, 1678, 1793, 1907, 2022

2136

Supervision Area A

Interview Location


Choosing the “Final” RespondentWhen the program participants are not listed (second scenario) but

after having selected the communities.

Let’s continue with the scenario with which we have been working. We want to interview women 15-49 on communities where our HIV/AIDS programming has gone on to see if women know 2 ways to prevent the transmission of HIV.

Once we have our communities selected (previous step) we need a way to identify women once we get to the community. To do this we still have two steps:

1. Choosing a place (household) in which to start the survey

2. Choosing the respondent

Let’s look at a couple of different scenarios for choosing the household and then examine how to choose a woman. Let’s say we are interviewing in Guimbe (from the previous step). We need to interview 5 women there.


Choosing the First HouseholdThough Guimbe is a big community, let’s look at several scenarios about how to select a starting household in various size communities. You

will also use this approach to select subsequent households in settings, like Guimbe, where you are to conduct more than one

interview.(I will provide a reference about how to do this in urban settings.)IF: THEN:

A complete household list is available (tax list, census, map)

• Assign a number to each house• Choose a random number between 1 and the highest numbered

house• Start with that house

The community size is about 30 households or less (not the case for Guimbe)

• Make a household list or map with the location of each household



The community size is more than about 30 households (Guimbe would be this case)

• Subdivide the community into 2-5 sections with about the same number of households in each section.

• Select one section at random• If that subdivision has more houses than you can easily

count, subdivide it into 2-5 sections and select a section at random

• Make a household list or map with the location of each household




Choosing the “Final” RespondentSo, now you have selected the household where to start. Use this table

to decide how to proceed…

IF: THEN:The type of respondent you are looking for is at the household you selected

Interview that person if she consents (see the bottom of the table for what to do next)

The type of respondent you are looking for does NOT live in the household you selected

Go the next nearest household from the FRONT ENTRANCE to the household you are at and check for an appropriate respondent at this household. Continue this process until you find the type of respondent you are looking for.

If two households are equally near, then choose the one with the closest door or flip a coin.

The type of respondent you are looking lives in the household BUT is away/absent at the time of the interview AND is more than 30 minutes away

The type of respondent you are looking for lives at the household BUT is absent BUT is within 30 minutes of where you are

Go find the respondent with the help of someone who knows where she is. IF you cannot find the person in the next 30 minutes THEN…

Go the next nearest household from the FRONT ENTRANCE to the household you are at and check for an appropriate respondent at this household. Continue this process until you find the type of respondent you are looking for. If two households are equally near, then choose the one with the closest door or flip a coin.In Guimbe you need to conduct five interviews so once you have completed the first interview

go back to the steps for selecting the first household and begin again following the approach that is appropriate to the size of the community. You will repeat this each time you must select a new respondent.

Technology

Survey Training and LQAS