Chapter 11Chapter 11Data Collection MethodsData Collection Methods

Observational research: Observational research: Observing the behavior of Observing the behavior of

individualsindividuals Observational researchObservational research

– Watching people and systematically Watching people and systematically recording their behavior.recording their behavior.

– Can be used to collect data in Can be used to collect data in experimental and non-experimental experimental and non-experimental research (text focuses on non-research (text focuses on non-experimental).experimental).

– It may be difficult or impossible to observe It may be difficult or impossible to observe some behaviors we are interested in.some behaviors we are interested in.

Observational researchObservational research Observing behavior from the outside: Observing behavior from the outside:

Naturalistic observationNaturalistic observation– Involves making systematic observations of behavior Involves making systematic observations of behavior

in the environment where it occurs naturally.in the environment where it occurs naturally.

– Used to study behavior as it naturally occurs and in a Used to study behavior as it naturally occurs and in a way that is as unobtrusive as possible.way that is as unobtrusive as possible.

– Well suited for studying behaviors that we fear would Well suited for studying behaviors that we fear would be altered or not occur at all if the participants knew be altered or not occur at all if the participants knew they were being observed (reactivity effect). they were being observed (reactivity effect).

– High external validity. High external validity.

– Can be time-consuming and inconvenient.Can be time-consuming and inconvenient.

Observational researchObservational research Observing behavior from the inside: Observing behavior from the inside:

Participant observationParticipant observation– The researchers enter the world of the people The researchers enter the world of the people

they are interested in studying and maintain they are interested in studying and maintain field notes chronicling their observations.field notes chronicling their observations.

– Usually provides rich narratives about the Usually provides rich narratives about the experiences of the observed and the observer. experiences of the observed and the observer.

– Advantages:Advantages: Introduces us to the world of others.Introduces us to the world of others. High external validity.High external validity. The researcher is able to experience the same The researcher is able to experience the same

environmental conditions as the participants.environmental conditions as the participants. Can collect information on factors that may not be Can collect information on factors that may not be

overtly observable.overtly observable.

Observational researchObservational research Observing behavior from the inside: Observing behavior from the inside:

Participant observationParticipant observation– Subjectivity is a major advantage of participant Subjectivity is a major advantage of participant

observation and it is also a major observation and it is also a major disadvantage. Researchers must be able to disadvantage. Researchers must be able to maintain/regain their objectivity and not maintain/regain their objectivity and not misrepresent or bias their observations as a misrepresent or bias their observations as a result of their experiences. result of their experiences.

– Disadvantages:Disadvantages: Time consuming and potentially dangerous.Time consuming and potentially dangerous. May be difficult to gain entrance into certain groups.May be difficult to gain entrance into certain groups. Ethical issues (e.g.. need to maintain anonymity of Ethical issues (e.g.. need to maintain anonymity of

your informants).your informants).

Observational researchObservational research

Observing behavior in a controlled Observing behavior in a controlled setting: Laboratory observationsetting: Laboratory observation

– When we observe behavior in the When we observe behavior in the laboratory, we can create the conditions laboratory, we can create the conditions necessary to make the behavior occur. necessary to make the behavior occur.

– Trade-off between external validity and Trade-off between external validity and level of control. level of control.

– May lack external validity.May lack external validity.

Observational researchObservational research Advantages and disadvantages of Advantages and disadvantages of

observational researchobservational research– In all types of observational research it is In all types of observational research it is

important to form clear definitions of the important to form clear definitions of the behaviors you are interested in observing.behaviors you are interested in observing.

– Wherever research is conducted, the researcher Wherever research is conducted, the researcher must be accountable for any harm to the must be accountable for any harm to the participants. Steps must be taken to guard the participants. Steps must be taken to guard the rights of those involved. rights of those involved.

– Observational research is often time-consuming Observational research is often time-consuming and requires observers who are trained to and requires observers who are trained to carefully record behavioral observations. carefully record behavioral observations.

Survey research: Asking Survey research: Asking people questions about people questions about

their behaviortheir behavior Surveys Surveys

– Used to measure people’s opinions and Used to measure people’s opinions and attitudes, variables that may be difficult attitudes, variables that may be difficult or impossible to observe directly.or impossible to observe directly.

Defining your research questionDefining your research question

– From start to finish, your research From start to finish, your research question or hypothesis must guide you.question or hypothesis must guide you.

– Need to be certain about your purpose.Need to be certain about your purpose.

Survey researchSurvey research

How will you ask your questions?How will you ask your questions?– Survey – refers to the action of collecting Survey – refers to the action of collecting

information.information.

– Questionnaire – a list of questions that are asked Questionnaire – a list of questions that are asked when you are collecting information.when you are collecting information.

– Need to determine the kinds of information you Need to determine the kinds of information you need and the best method of obtaining that need and the best method of obtaining that information. Also need to consider who your information. Also need to consider who your respondents are.respondents are.

– Time, money, literacy, and respondent honesty Time, money, literacy, and respondent honesty are all factors that need to be considered when are all factors that need to be considered when making the choice. making the choice.

Survey researchSurvey research

InterviewsInterviews

– Interviewing can be very expensive and Interviewing can be very expensive and time consuming, and it requires trained time consuming, and it requires trained interviewers.interviewers.

– Allows rapport to be built, which can be Allows rapport to be built, which can be an advantage and a disadvantage. an advantage and a disadvantage.

– Includes:Includes: Face-to-face interviewsFace-to-face interviews

Telephone interviewsTelephone interviews

InterviewsInterviews Face-to-face interviewFace-to-face interview

– Generally, if the population you want to Generally, if the population you want to study is available at a particular location study is available at a particular location and your questions don't take more than and your questions don't take more than about five minutes, then the simplest about five minutes, then the simplest approach is to go to where your group is approach is to go to where your group is and interview them in person. and interview them in person.

– Do not need to have respondents Do not need to have respondents complete an informed consent form complete an informed consent form before they answer your questions, but before they answer your questions, but still need to consider the ethics of still need to consider the ethics of consent. consent.

InterviewsInterviews

Face-to-face interviewFace-to-face interview

– Advantages and disadvantages of face-Advantages and disadvantages of face-to-face interviewsto-face interviews AdvantagesAdvantages

– Gather information directly from the people you Gather information directly from the people you are interested in researching. are interested in researching.

– Can explore complex issues that do not lend Can explore complex issues that do not lend themselves to multiple-choice answersthemselves to multiple-choice answers (e.g.. (e.g.. probing questions and open-ended questions). probing questions and open-ended questions). [Note: it is very important that the probes be [Note: it is very important that the probes be neutral so as not to bias responses.] neutral so as not to bias responses.]

InterviewsInterviews Face-to-face interviewFace-to-face interview

– Advantages and disadvantages of face-to-Advantages and disadvantages of face-to-face interviewsface interviews DisadvantagesDisadvantages

– Can be very time consuming.Can be very time consuming.

– Participant discomfort in discussing personal or Participant discomfort in discussing personal or embarrassing topics.embarrassing topics.

– Social desirability - the tendency for people to respond Social desirability - the tendency for people to respond in a manner that makes them appear better than they in a manner that makes them appear better than they are. are.

– Labor intensive.Labor intensive.

– Practical limitations in terms of the number of Practical limitations in terms of the number of participants.participants.

InterviewsInterviews

Telephone interviewTelephone interview

– Best choice if your research question Best choice if your research question requires interviewing a large number of requires interviewing a large number of respondents who are spread over a respondents who are spread over a large geographical area. large geographical area.

– Generally shorter than face-to-face Generally shorter than face-to-face interviews.interviews.

– Data are often coded and entered into a Data are often coded and entered into a computer directly by the telephone computer directly by the telephone interviewer. interviewer.

InterviewsInterviews

Telephone interviewTelephone interview

– Advantages and disadvantages of telephone Advantages and disadvantages of telephone interviewsinterviews Advantage: Advantage:

– Don’tDon’t have to be in the same location as the have to be in the same location as the respondent.respondent.

Disadvantages: Disadvantages:

– Selection bias as a results of individuals screening calls.Selection bias as a results of individuals screening calls.– Time consuming.Time consuming.– Require trained interviewers.Require trained interviewers.

Survey researchSurvey research

Types of questionnaires:Types of questionnaires:

– Self-administered questionnairesSelf-administered questionnaires

– Group-administered questionnairesGroup-administered questionnaires

– Mail-out questionnairesMail-out questionnaires

– Internet questionnairesInternet questionnaires

QuestionnairesQuestionnaires

Self-administered questionnairesSelf-administered questionnaires– Survey questions that are read and Survey questions that are read and

answered by the respondent with little answered by the respondent with little or no direct contact with the researcher.or no direct contact with the researcher.

– Advantages:Advantages: CheapCheap FastFast AnonymousAnonymous

– Requires literacy and for the Requires literacy and for the questionnaire to be well written.questionnaire to be well written.

QuestionnairesQuestionnaires Group-administered questionnairesGroup-administered questionnaires

– If it is possible to gather your respondents into If it is possible to gather your respondents into groups (e.g.. students in classrooms), then group groups (e.g.. students in classrooms), then group administration is probably your best option.administration is probably your best option.

– Need someone to distribute and collect the Need someone to distribute and collect the questionnaire.questionnaire.

– AdvantagesAdvantages InexpensiveInexpensive FastFast High response ratesHigh response rates

– Someone is present while the participants Someone is present while the participants complete the questionnaire. This administrator complete the questionnaire. This administrator can give verbal instructions and answer can give verbal instructions and answer questions.questions.

QuestionnairesQuestionnaires Mail-out questionnairesMail-out questionnaires

– Quick and economical method for distributing Quick and economical method for distributing your questionnaire to a large number of people your questionnaire to a large number of people spread over a large area .spread over a large area .

– Gives respondents a feeling of confidentiality Gives respondents a feeling of confidentiality and anonymity.and anonymity.

– Main problem: poor response rates. This results Main problem: poor response rates. This results in increased costs and potentially poor validity in increased costs and potentially poor validity as the respondents may not be representative as the respondents may not be representative of the population.of the population.

– You do not control who completes the You do not control who completes the questionnaire. questionnaire.

QuestionnairesQuestionnaires Internet questionnairesInternet questionnaires

– Provides respondents with a strong feeling of Provides respondents with a strong feeling of anonymity.anonymity.

– Allows you to target special groups.Allows you to target special groups.

– Many of the same problems as other survey Many of the same problems as other survey research:research: People who don’t use the Internet will not be included in People who don’t use the Internet will not be included in

an Internet survey.an Internet survey.

Social desirability and other kinds of response bias.Social desirability and other kinds of response bias.

Poor response rate.Poor response rate.

– Internet research has special problems such as Internet research has special problems such as multiple submissions from the same respondents.multiple submissions from the same respondents.

Survey researchSurvey research

General guidelines for writing survey General guidelines for writing survey questionsquestions

– Wording of your questions is important.Wording of your questions is important.

– Ensure that your questions are asking Ensure that your questions are asking what you intend them to ask by pre-what you intend them to ask by pre-testing your items. testing your items.

– Do a pilot study to detect other Do a pilot study to detect other problems before beginning the full problems before beginning the full study.study.

Writing Survey QuestionsWriting Survey Questions

Guidelines:Guidelines:– Keep your questions short and simple.Keep your questions short and simple.

This helps ensure respondents clearly This helps ensure respondents clearly understand your questions.understand your questions.

– Avoid using ‘and’ in your questions.Avoid using ‘and’ in your questions. You should measure no more than one You should measure no more than one

dimension with each question.dimension with each question.

– Do not use biased wording in your Do not use biased wording in your questions.questions.

– Be sure you avoid using double negatives.Be sure you avoid using double negatives. Can be confusing for respondents.Can be confusing for respondents.

Type of questionsType of questions

Open-ended questionsOpen-ended questions

– Items that simply have a blank space for Items that simply have a blank space for the response. the response.

– Best choice if you are not sure how Best choice if you are not sure how people will answer a question or if you people will answer a question or if you are looking for diverse responses.are looking for diverse responses.

– Disadvantage is that they may be Disadvantage is that they may be difficult to analyze.difficult to analyze.

– Can be time consuming and costly.Can be time consuming and costly.

Type of questionsType of questions

Forced choice questionsForced choice questions

– Items that include response categories.Items that include response categories.

– Easier to analyze and easier to answer.Easier to analyze and easier to answer.

– Must be sure that the response choices Must be sure that the response choices include all the possible responses people can include all the possible responses people can make.make. You may include an “other” category – be sure to You may include an “other” category – be sure to

leave a blank for people to enter their response.leave a blank for people to enter their response.

– Need to provide clear instructions that only Need to provide clear instructions that only one response may be selected.one response may be selected.

General rules for self-General rules for self-administered questionnairesadministered questionnaires

Very important that it looks professional. Very important that it looks professional. Should be easy to read and organized in a Should be easy to read and organized in a

way that is easy to follow. way that is easy to follow. Mailed questionnaires should include a Mailed questionnaires should include a

cover letter with sufficient information for cover letter with sufficient information for participants to decide to whether or not to participants to decide to whether or not to participate (like informed consent).participate (like informed consent).

Use clear organization so that your Use clear organization so that your respondent does not get lost filling it out respondent does not get lost filling it out (e.g.. try to organize the questions by (e.g.. try to organize the questions by topic).topic).

Observing group behavior: Observing group behavior: The focus groupThe focus group

Could be described as a face-to-face interview of a Could be described as a face-to-face interview of a small group of people in which participants can small group of people in which participants can interact and generate ideas from one another.interact and generate ideas from one another.

Often used to measure attitudes, opinions, or self-Often used to measure attitudes, opinions, or self-reported behavior of select groups.reported behavior of select groups.

Advantage Advantage of revealing information that may not of revealing information that may not surface in face-to-face interviews. surface in face-to-face interviews.

Dynamic nature of focus groups can lead to Dynamic nature of focus groups can lead to problems (e.g.. a few forceful members dominate problems (e.g.. a few forceful members dominate the discussion). the discussion).

Usually recorded so the content of the discussion Usually recorded so the content of the discussion can be analyzed later using qualitative techniques.can be analyzed later using qualitative techniques.