Upload
tyrone-sutton
View
218
Download
0
Embed Size (px)
Citation preview
C1, L2, S1
Design Method of Data Collection
Surveys and Polls
Experimentation Observational
Studies
C1, L2, S2
NonsamplingErrors
Sampling/Chance/Random Errors
Selection bias Interviewer effects
Non-response bias Behavioural considerations
Self selection Transfer findings
Question effects Survey-format effects
Sampling
C1, L2, S3
Sources of Nonsampling Errors
Selection bias
Population sampled is not exactly the population of interest.
e.g. KARE 11 poll, telephone interviews
population
sample
C1, L2, S4
Sources of Nonsampling Errors
Non-response bias
People who have been targeted to be surveyed do not respond.
Non-respondents tend to behave differently to respondents with respect to the question being asked.
C1, L2, S5
1936 U.S. Election
• Country struggling to recover from the Great Depression
• 9 million unemployed
• 1929-1933 real income dropped by 1/3
C1, L2, S6
1936 U.S. Election
• Candidates:
– Albert Landon (Republican)
“The spenders must go!”
– Franklin D Roosevelt (Democrat)Deficit financing - “Balance the budget of the people before balancing the budget of the Nation”
This man has never been President.
C1, L2, S7
1936 U.S. Election• Roosevelt’s percentage
–Digest prediction of the election result
–Gallup’s prediction of the Digest prediction
–Gallup’s prediction of the election result
–Actual election result
43%
44%
56%
62%
• Digest sent out 10 million questionnaires to people on club membership lists, telephone directories etc.
– received 2.4 million responses• Gallup Poll used another sample of 50,000• Gallup used a random sample of 3,000 from the Digest lists to predict Digest outcome
C1, L2, S8
Sources of Nonsampling Errors
Self-selection bias
People decide themselves whether to be surveyed or not.
Much behavioral research can only use volunteers.
C1, L2, S9
Sources of Nonsampling Errors
C1, L2, S10
Sources of Nonsampling Errors
This poll is not scientific and reflects the opinions of only those Internet users who have chosen to participate. The results cannot be assumed to represent the opinions of Internet users in general, nor the public as a whole. The QuickVote sponsor is not responsible for poll content, functionality or the opinions expressed therein.
C1, L2, S13
Sources of Nonsampling Errors
Question effects
Subtle variations in wording can have an effect on responses.
e.g. “Should euthanasia be legal?”vs. “Should voluntary euthanasia be legal?”
C1, L2, S14
18 August 1980 New York Times/CBS News Poll
“Do you think there should be an amendment to the constitution prohibiting abortions?”
Yes 29% No 62%
Later the same people were asked:
“Do you think there should be an amendment to the constitution protecting the life of the unborn child?”
Yes 50% No 39%
C1, L2, S15
Ethnicity 1986 1991 1996 Single Ethnicity
94.6 94.3 81.0
European 81.2 78.1 65.8 Maori 9.1 9.6 7.6Two Ethnicities 4.0 4.5 11.2 European & Maori 2.9 2.7 4.7Two European gps 0.0 0.6 4.5
Question Effects in the NZ Census1986: “What is your ethnic origin? (Tick the box or boxes which apply to you.)
1991: “Which ethnic group do you belong to?” (Tick the box or boxes which apply to you.)
1996: “Tick as many circles as you need to show which ethnic group(s) you belong to.”
C1, L2, S16
Sources of Nonsampling Errors
Interviewer effects
Different interviewers asking the same question can obtain different results.
e.g. the sex, race, religion of the interviewer
C1, L2, S17
Interviewer Effects in Racial Questions
In 1968, one year after a major racial disturbance in Detroit, a sample of black residents were asked:
“Do you personally feel that you trust most white people, some white people or none at all?”
• White interviewer: 35% answered “most”
• Black interviewer: 7% answered “most”
C1, L2, S18
Sources of Nonsampling Errors
Behavioral considerationsPeople tend to answer questions in a way they consider to be socially desirable.
e.g. pregnant women being asked about their drinking habits
C1, L2, S19
Behavioral Considerations in Election
• Official vote counts show that 86.5 million people voted in the 1980 U.S. presidential elections.
• A census bureau survey of 64,000 households some weeks later estimated 93.1 million people voted.
C1, L2, S20
Sources of Nonsampling Errors
Transferring findings
Taking the data from one population and transferring the results to another.
e.g. Minneapolis opinions may not be a good indication of Winonan opinions.
Minneapolis
sample
Winona
C1, L2, S21
Sources of Nonsampling Errors
Survey-format effects
e.g. question order, survey layout, interviewed by phone or in- person or mail.
C1, L2, S22
NonsamplingErrors
Sampling/Chance/Random Errors
Selection bias Interviewer effects
Non-response bias Behavioural considerations
Self selection Transfer findings
Question effects Survey-format effects
Sampling
C1, L2, S23
1999 New Zealand Newspaper Graphic
C1, L2, S24
Nonsampling Errors
What nonsampling errors could be present if the results of the survey were used as representative of the views held by New Zealanders in general?
Discuss in groups for 1 minute.
C1, L2, S25
Nonsampling Errors
• Selection bias
– only those who read this paper
– only those that can afford to pay for the call
• Question effects
– how do you answer the question??
• Self-selection bias
C1, L2, S26
SurveyErrors
Sampling/Chance/ Random Errors
NonsamplingErrors
C1, L2, S27
Sampling / Chance / Random Errors
• errors caused by the act of taking a sample
• have the potential to be bigger in smaller samples than in larger ones
• possible to determine how large they can be
• unavoidable (price of sampling)
C1, L2, S28
Nonsampling Errors
• can be much larger than sampling errors
• are always present
• can be virtually impossible to correct for after the completion of survey
• virtually impossible to determine how badly they will affect the result
• must try to minimize in design of survey (use a pilot survey etc.)
C1, L2, S29
Surveys / Polls
A pilot survey is a small survey that is carried out before the main survey and is often used to identify any problems with the survey design (such as potential sources of non-sampling errors).
C1, L2, S30
Surveys / Polls
A report on a sample survey/poll should include:– target population (population of interest)– sample selection method– the sample size and the margin of error– the date of the survey– the exact question(s) being asked