Upload
camron-parks
View
233
Download
1
Embed Size (px)
Citation preview
Chapter 3Experiments, Quasi-Experiments, and Field Observations
Winston Jackson and Norine Verberg
Methods: Doing Social Research, 4e
© 2007 Pearson Education Canada 3-2
The Rationale of the Experiment
John Stuart Mill — Method of Difference — the experiment is the key tool for discerning causal relations
A well-designed experiment should provide clear evidence of a cause-effect relation Indicate whether or not the treatment variable
(e.g., studying with the radio on) will bring about a change in some dependent measure (grade performance), other things being equal
© 2007 Pearson Education Canada 3-3
Rationale (continued)
Internal validity: The extent to which the researcher can demonstrate that the treatment variable is having an impact on the dependent variable, and that other sources have been controlled
External validity: The extent to which the researcher can extrapolate the study findings to other groups in general
© 2007 Pearson Education Canada 3-4
Key Elements in Experimental Designs
A. Dependent variable – the effect in a cause-effect relationship
B. Independent variable – the variable the researcher manipulates to determine whether and how it will change the dependent variable the cause in a cause-effect relationship
© 2007 Pearson Education Canada 3-5
Key Elements (cont’d)
Kinds of independent variables:i. Treatment Variables – variable studied
ii. Control Variables – major influences intentionally controlled for in the experiment
iii. Confounding Variables – variables that can unintentionally obscure or enhance results
iv. Random Variables – vary without control, but are taken into account in study design (e.g., randomization)
© 2007 Pearson Education Canada 3-6
Key Elements (cont’d)
C. Levels – often two or three levels 2 x 2: two levels of the treatment variable
and two levels in a control variable
© 2007 Pearson Education Canada 3-7
Pseudo-Experimental Designs
Have limited scientific merit Also called pre-experimental designs
Share some elements of classic experiment, however, they do not permit clear causal inferences
Two types: Same group: pretest/post-test design Exposed/comparison group design
© 2007 Pearson Education Canada 3-8
A. Same Group: Pretest/Post-Test
© 2007 Pearson Education Canada 3-9
Threats to Internal Validity
i. History – concurrent events
ii. Maturation – changes in the individual subject
iii. Testing – possible of response bias
iv. Instrument Decay – unreliable measurement
v. Statistical Regression – extreme scores
© 2007 Pearson Education Canada 3-10
B. Exposed/Comparison Group
Measures are taken at only one point in time.
Problem: groups may not have been similar initially.
The result may, or may not, be due to the treatment variable.
© 2007 Pearson Education Canada 3-11
More Threats to Internal Validity
vi. Selection – Subjects selecting themselves into the study
vii. Mortality – Subjects selecting themselves out of the study
© 2007 Pearson Education Canada 3-12
Classic Experimental Designs
Two types: Between-Subjects Design Within-Subject Design
Both types of design allow a researcher to demonstrate causal inference
© 2007 Pearson Education Canada 3-13
A. Between-Subjects Design
© 2007 Pearson Education Canada 3-14
Between-Subjects Design (cont’d)
Involves a control and an experimental group The experimental group is exposed to
treatment intervention The control group is exposed to neutral
treatment
© 2007 Pearson Education Canada 3-15
Key to Experimental Design
Construct treatment and control groups to be as similar as possible before the experiment begins. This is done by: Randomization – each subject has an equal
chance of being assigned to either group (provides control over both known [control] and unknown [random] factors)
Precision matching – matching subjects between groups
Combination of the above two methods
© 2007 Pearson Education Canada 3-16
Key to Experimental Design (cont’d)
Blocking – Group subjects according to some controlled variable before randomly assigning them to a group
Baseline stability – Taking measures of the variable prior to introducing treatment
© 2007 Pearson Education Canada 3-17
Analyzing the Data
TABLE 3.2 PERCENT WANTING TO ATTEND UNIVERSITY
BY EXPOSURE AND NON-EXPOSURE TO CD-ROM
PERCENTAGE WANTING TO ATTEND UNIVERSITY
GROUP TIME 1 TIME 2 DIFFERENCE
Treatment 57.0 73.0 73 – 57 = 16
Control 55.0 61.0 61 – 55 = 6
Estimated impact of CD-ROM: 10
© 2007 Pearson Education Canada 3-18
Demonstrating a Causal Relation
1. Changes in treatment variable occur prior to changes in the dependent variable
2. The treatment and dependent variables are associated: as the treatment variable goes up, the dependent varies systematically
3. Nothing but the treatment variable has influenced the dependent variable
© 2007 Pearson Education Canada 3-19
Ruling out Confounding Effects
Ensure that context is the same
Balance the background characteristics
Neutralize confounding (sources of spuriousness) variables
Deal with random variables
© 2007 Pearson Education Canada 3-20
B. Within-Subject Designs
In the between-subjects design, the control for known and unknown factors is achieved through randomization
In the within-subject design, the control for known and unknown factors is achieved by exposing a subject to the different treatments Since the subject is the same person,
background characteristics, attitudes, and intelligence are all perfectly controlled
Also called control by constancy
© 2007 Pearson Education Canada 3-21
Within-Subject Design (cont’d)
Subjects are exposed to the various treatments Subjects’ own scores when exposed to different
treatments are compared Importance of having a baseline measure and returning
to the original condition The within-subject ABBA design:
A – measure dependent variable under original condition B – measure dependent variable under treatment
condition B – continue treatment condition and measure dependent
variable A – measure dependent variable after returning to
original condition
© 2007 Pearson Education Canada 3-22
Hawthorne Effect
Refers to any variability in the dependent variable that is not the direct result of variations in the treatment variable
Hypothesis: worker productivity would increase as lighting intensity was increased When lighting increased, productivity increased HOWEVER, when lighting was later decreased,
productivity did not decrease. WHY? Interpretation: something other than treatment
variable influenced workers – perhaps they worked faster because they knew were being observed
© 2007 Pearson Education Canada 3-23
Quasi-Experimental Designs
Approximation of experimental design: done in situations where it is not possible to: use random assignment control the nature or timing of the treatment
Example: Henry & Ginzberg: Racial Discrimination in
Employment (See Box 3.4, text pp. 75-77.)
© 2007 Pearson Education Canada 3-24
Racial Discrimination in Employment
Two job applicants matched with respect to age, sex, education, physical appearance (dress), and personality were sent to apply for the same advertised job. Only difference: one was White, one was Black
Results Both offered job 5.0% White offered job 13.4% Black offered job 4.5% Neither offered job77.1%
© 2007 Pearson Education Canada 3-25
Field Experiments
Researcher intervenes in a natural settings Direct observations, “real” behaviour
Researcher intervention Greeting stranger
Proxemics: norms surrounding personal space and the conditions under which such space will or will not be violated
Examples: cutting-through behaviour, greeting behaviour, helping behaviour
© 2007 Pearson Education Canada 3-26
Naturalistic Observational Studies
Observe and record behaviour that occurs in a natural setting with those being observed unaware that they are being studied Do not attempt to alter social environment No intervention, simply record behaviour
Tally sheets are designed, then used to record the behaviour Andrew Harrell’s Grocery Cart Safety study
© 2007 Pearson Education Canada 3-27
Samples of Student Research Projects
Dressing for winter Parking violations Gender and smoking Professor/student
participation: gender Seat belt compliance Speeding Antigonish Buying healthy food
ABM behaviour Termination of
conversations Drinking patterns Smoking behaviour in
teens Stop sign Tipping
© 2007 Pearson Education Canada 3-28
Steps in Doing Study
1. Restrict observations
2. Review of literature
3. Develop hypotheses
4. Define terms
5. Develop a tally sheet (See Figure 3.5, p. 90)
6. Transfer data to master table (see Figure 3.6, p. 90)
7. Creating tables (Tables 3.4, 3.5, 3.6, p. 91)
8. Writing the report
© 2007 Pearson Education Canada 3-29
Field and Observational Studies: An Assessment
Weak on generalizations Strong on validity (real behaviour) Making causal inferences a challenge Multivariate a problem Probing strong with participant observation,
in-depth interviews, and focus groups Probing weak with naturalistic observational
© 2007 Pearson Education Canada 3-30
Advantages and Disadvantages of Experimental Designs Advantages:
Ease of making clear causal inferences
Disadvantages: Low external validity: poor on generalization to
a larger population Concerns about the artificiality of lab Poor on probing, poor on multivariate Experiments cannot study all topics