Upload
felicia-dalton
View
223
Download
0
Tags:
Embed Size (px)
Citation preview
Conducting Experiments, Survey Construction and Data Collection
RCS 67407/18/05
Practical Considerations of Research This lecture will focus on some of the
practical aspects of conducting research studies such as: How do you select research participants What should you consider when deciding how to
manipulate an independent variable How do you measure a variable What should you consider when constructing a
survey What should you consider when collecting data How do ensure control in your study
Selecting Research Participants Define your participants
People with Aphasia Women with Breast Cancer Rats with 5 Legs
Define your population Everyone in the United States People in RCS 6740 Consumers at “The Center” (CIL)
Selecting Research Participants Cont. Define your sampling technique
Probability sampling Random sampling Stratified random sampling Systematic sampling Cluster sampling
Non-probability sampling Quota sampling
Determine your sample size Remember, larger sample sizes may result in
more significant data
Manipulating the Independent Variable Construct the Operational Definition of your variables
Exercise as defined by jogging for 20 minutes, 3 times a week, for 6 weeks
Depression as defined by the Geriatric Depression Inventory Set the Stage
Provide the participants with Informed Consent Usually a document that entails the name of the study, purpose(s)
of the study, potential benefits and risks, funding sources, contact information, and a statement that they can withdraw at any time
Explain to the participants in more detail why the study is being conducted Rarely are hypotheses given to participants
Tell them the study looks at employment when the hypothesis looks specifically at bias towards the employment of people with disabilities
Can be completely truthful and precise Deception may be used
When deception is used, it is imperative that participants are told the truth during a debriefing at the conclusion of the study
Types of Manipulations Straightforward Manipulations
Presenting written, verbal, or visual material to participants
Examples: Having participants read material on substance
abuse prevention Read a passage to a participant and ask them
questions later to test recall (memory) Have a computer program that presents different
words that someone needs to pronounce to measure the level of aphasia
Most research is done in a straightforward manner Researchers can vary the difficulty of material,
motivational levels, the way questions are asked, or the characteristics of the participants
Types of Manipulations Cont. Staged Manipulations
Also called Event Manipulations Used for 2 reasons
The researcher may need to create some sort of psychological state (anxiety)
The researcher may need to stage a manipulation to recreate a real-world scenario Having a participant do one task and then having them do more
tasks at the same time Staged manipulations usually employ a confederate
Also called an accomplice A confederate is someone who appears to be another
particpant in an experiment but is really a part of the experiment
Example: Someone who purposely insults a participant in a study in order to provoke anger or frustration
Types of Manipulations Cont. Another example of the use of confederates: Asch (1956) study on conformity
Which line is bigger? 1)--------------------- 2)----------------------------- 3)-------------------
Right before a participant had to choose which line was the longest, a confederate announced an incorrect answer
Repeatedly, Asch found that people conformed to the confederate’s incorrect response
Types of Manipulations Cont. Straightforward Manipulations Vs.
Staged Manipulations: It is easier to recreate real-world situations
using Staged manipulations Staged manipulations are more extensive
in terms of cost and set up procedures It is difficult for other researchers to
replicate staged manipulations Data and results of Straightforward
manipulations are easier to interpret due to a greater sense of control
Strength of Manipulations The strength of a manipulation can effect
how results are interpreted Example: In a study on the use of a new
medication to treat diabetes, it is important that the researcher uses a sufficient dosage of the drug in order to see an effect 5 mgs of Diabetex does not reduce blood sugar
levels 10 mgs of Diabetex reduces blood sugar levels
by 3% 20 mgs of Diabetex reduces blood sugar levels
by 10%
Strength of Manipulations Cont. A strong manipulation maximizes the
differences between groups and increases the chance that the independent variable will have a significant effect on the dependent variable
Strong manipulations are especially important in the early stages of research (Why?)
Although you want a manipulation to be as strong as possible, ethical considerations should be examined You do not want to make someone so
anxious that they decide to jump off a building
Cost of Manipulations
Most researchers are limited in terms of the funding they have for a research project
Confederates and elaborate manipulations may cost a great deal
The straightforward approach is usually more budget friendly
Measuring the Dependent Variable
Types of Measures Self-report measures Behavioral Measures Physiological Measures
Measuring the Dependent Variable Cont.
Self-report Measures Used to measure attitudes, judgments
about oneself or another, emotional states, confidence in oneself or others, and other aspects of human thought and behavior
Example: Please use the following 7 point scale to answer this question. I feel that I am the best student in this class:Strongly Disagree _ _ _ _ _ _ _ Strongly Agree
Measuring the Dependent Variable Cont.
Behavioral Measures Direct observations of behaviors such as:
How many errors someone makes on a test How many times a person can lift a 10lb object How may free throws a person makes in 30 minutes
Things to consider with behavioral measures Rate: How many times a behavior occurs Reaction Time: How quickly a response occurs after a
stimulus Duration: How long a behavior lasts
Measuring the Dependent Variable Cont.
Physiological Measures Recordings of responses of the body
Galvanic Skin Response (GSR): Measures the electrical conductance of the skin to look for arousal or anxiety
Electromyogram (EMG): Measures muscle tension to look for stress
Electroencephalogram (EEG): Measures activity of brain cells such as looking at what area of the brain is active while dreaming
Magnetic Resonance Imaging (MRI): A technique used to gather detailed images of the inner body
Functional Magnetic Resonance Imaging (fMRI): Looking at the inner body while someone performs a physical or a cognitive task
Measuring the Dependent Variable Cont.
What Type of Measure(s) to use? Basically, what type(s) of measure a researcher will use
depends on the nature of the study, cost, and possible ethical considerations
If at all possible a researcher should use a combination of all three types: To study anxiety, a researcher can administer a self-
report measure asking how nervous a person thinks they are, a behavioral measure that looks at how many times a person’s voice cracks, and a physiological measure looking at fluctuations in someone’s heart rate
When administering multiple measures, it is important to remember to account for a possible order effect Example: A self-report measure on how nervous a
person is might cause them to become more nervous thus increasing their heart rate
Measuring the Dependent Variable Cont.
Sensitivity of the Dependent Variable The dependent variable should be sensitive enough to
detect differences Do you like your boss (not sensitive) On a scale of 1 to 10, how much do you like your boss
(very sensitive) Beware of the Ceiling Effect and the Floor Effect
Ceiling Effect: Task is too easy so everyone performs well Lifting a 1lb object
Floor Effect: Task is too difficult so everyone performs poorly Lifting a 200lb object
Information on Survey Construction
DESIGNING SURVEYS A good question is one that produces
answers that are reliable and valid measures of something that we want to describe
2 types of question evaluations: Those aimed at evaluating reliability Those aimed at assessing the validity of
answers
Survey Construction Cont. 5 process standards of questions and
answers Questions need to be consistently understood. Questions need to be consistently administered
or communicated to respondents. What constitutes an adequate answer should be
consistently communicated. Unless measuring knowledge is the goal of the
question, all respondents should have access to the information needed to answer the question accurately.
Respondents must be willing to provide the answers called for in the question.
Survey Construction Cont.
Possible steps to assess the extent to which questions meet process standards Focus group discussions Intensive or cognitive interviews Field pre-tests under realistic conditions
Survey Construction Cont. Some general rules for designing good survey
instruments The strength of survey research is asking people about their
firsthand experiences: what they have done, their current situations, their feelings and perceptions.
Questions should be asked one at a time. A survey question should be worded so that all respondents
are answering the same question. If a survey is to be interviewer administered, wording of the
questions must constitute a complete and adequate script such that when the interviewer reads the question as worded, the respondent will be fairly prepared to answer the question.
All respondents should understand the kind of answer that constitutes an adequate answer to a question.
Survey instruments should be designed so that the tasks of reading questions, following instructions, and recording answers are as easy as possible for interviewers and respondents.
Survey Construction Cont.
Ways of addressing validity in surveys Deriving questions from relevant
literature Expert panel
Data Collection Tips Use more than one survey or measure Come prepared
Extra measures, pens, equipment Be prepared to answer questions about the survey
Important not to sway a response Follow up to ensure all items were filled out
(999=missing, 888=don’t know) Know your survey
How to score it, the factors, the subscales If possible, have a limited amount of researchers collect the
data Enter data as quickly as possible and use a trusting
computer program (SPSS, ECXEL) Have two people enter the data (one reads, one enters)
Control ConsiderationsParticipant Expectations
When a participant knows too much about a study, they may expect to perform in a certain manner
Demand Characteristics: any feature of an experiment that might inform participants of the true purpose of a study Once someone is aware of the hypothesis of a study,
the usually do whatever it takes to confirm the hypothesis Hypothesis: People who take a drug will lessen their
anxiety levels. Once a participant knows this, they may choose to do whatever they can do become less anxious (e.g., exercise, see a counselor, meditate).
Control Considerations Cont.Ways to Control Participant Expectations
Use deception to cover up what is really being studied Administer irrelevant measures Add filler questions to measures
Filler questions are unrelated questions on a survey that should disguise the true purpose of a study
Include a Placebo group in a study Placebo effect: When it is not know if results of a
treatment can be attributed to the treatment itself or the expectancies of the treatment
Placebos may consist of giving someone a pill that is made of sugar or injecting someone with saline solution It is important to remember ethical consideration of
using a placebo group. For example, if someone in the placebo group becomes very ill, you may want to administer treatment to them
Control Considerations Cont.Experimenter Expectations
Sometimes experimenters who know the purpose of a study may expect a participant to act or perform a certain way
Also known as expectancy bias There are 2 potential sources of experimenter bias
When a researcher treats participants in the control group differently than participants in the treatment group Spending more time with the treatment group
Researchers may record or analyze observations in a biased manner Well, even though he spoke 10 words clearly, I thought
he was close on another so I will give him 11/20
Control Considerations Cont.Ways to Control Experimenter Expectations
Researchers should be well trained and highly ethical Run all conditions simultaneously so the
experimenter’s behavior is the same for all participants
Use experimenters who are unaware of the true hypothesis of the study Double-Blind Study: Neither the participant nor the
experimenter knows whether a placebo or the actual treatment is given
Once again, it is highly unethical for experimenters to influence participants or manipulate data!!!
Additional Tips for Conducting Experiments Once a study is designed, there are certain ways to
improve it Write up a thorough Research Proposal
Includes a literature review, purpose of the study, procedures of the study, copies of measures, a budget, and the manner with which the data will be analyzed
Used often for grants, Institutional Review Boards, and Ethics Committees
Conduct a Pilot Study A trial run of a study with small sample sizes Used to “get the kinks” out Used to gather participant feedback
Additional Considerations of Experiments Cont. Manipulation Checks
A manipulation check is a way to measure if the independent variable has an effect on the dependent variable Checks if the independent variable manipulation is a
successful operationalization of the conceptual variable being studied
Provides evidence for Construct Validity Allows researchers to change something in a study before it
is too late Allows researchers to see if they are getting significant
results Example: In a study looking at whether people are nicer to
attractive people, a researcher will determine whether the person who they chose to be attractive really is being viewed as so by the participants. If they are not viewed as attractive, the researchers can hire another model.
Manipulation Checks may consist of any type of measure (i.e., self-report, behavioral, physiological)
Additional Considerations of Experiments Cont. Debriefing
Debriefing is the act of meeting with the participants after the conclusion of the study to discuss the true purpose of the study, ethical considerations of the study, and educational implications of the study
Can also be used to ensure that the participants do not share any information with those still involved in the study
Can also be beneficial as researchers may gain valuable information about how the study looks and feels from the “inside”
Analyzing Data
After a quantitative study has concluded, all data should be analyzed statistically
Communicating Research to Others Once a researcher has finished a study,
they should communicate their results and findings to others Professional Meetings and Conferences (NRA)
that present the opportunity to share results in a verbal, written, or poster format
Journal Articles also are a good way to share results Peer Review is the process of a paper being
accepted or declined entry into a journal based on reviews from peers
About 90% of papers submitted to the more prestigious journals are rejected but do not let this get you down