Lab 8Sampling and Design

Agenda• Sampling• Research Designs

– True Experiment– Quasi Experiment– Non-Experimental / Observational

• How to find and evaluate a design

Sampling

Sampling• When you want to test a research idea, you

need to decide on a population to which the idea applies. (Children? Adults with depression? Prisoners?)

Define your populationPopulation = characteristics of all possible cases from which you will draw a sample

Population and Sample• You cannot study everyone in your

population.You need to find a reasonable sample

that reflects your population. Representative? Randomly selected? Biased? Reflective of population?

Generalizability

Sampling Plan• Ensures accuracy and confidence in our

sample based on an expected margin of error.

• Margin of error: How “off” is our sample from the population?

• Probability level: How confident are we?

Representative Sampling Plan• Ensures that the selected sample

adequately represents the population• Achieved by probability sampling

• Why do researchers use non-probability sampling so often?

Probability Samples• Simple random sample

Each person has same chance of being selected into sample

• Stratified random sampleInclude same proportion of people in your sample as in the population(90% US American students,10% international students)

Non-Probability Samples• Convenience samples

People are selected because they are available.

• Extreme groupsPeople are selected because they are on extreme ends of the continuum of the construct of interest (e.g., self-esteem)

Representativeness• Are random samples always going to be

representative of the population?

• What happens to statistical accuracy as sample size increases?

http://www.stat.sc.edu/~west/applets/binomialdemo2.html

Sampling Distribution• Our sample may show sampling error, i.e.,

have a value or outcome that deviates from what we expect.

• These deviations are to be expected and are known as sampling error variance.

• Minimize sampling error: Large, random, representative sample Multiple samples, replications Make individuals in sample as similar as possible

Research Design

Types of Design: Experimental • Experimental- systematic manipulation

of independent variable in order to observe the effect on the dependent variable – True Experiment

• Random assignment to groups– Quasi Experiment

• Non-random Assignment

Types of Design: Experimental cont’d• Random Assignment- grouping where any given

participant has an equal chance of being assigned to the experimental group as to the control group

– Comparable groups formed

– ONLY difference between participants in the groups is the independent variable condition

• Example: if we use random assignment, we expect that we should have about the same number of males and females, about the same level of cognitive ability across groups, etc… in each of our experimental conditions.

Types of Design: Experimental cont’d• Non-random group assignment

– Necessary-• when you have too few subjects for random

assignment to work effectively• When for practical / ethical purposes you can’t

randomly assign– Threat to your ability to make a causal

inference

Types of Design: Non-Experimental / Observational• No manipulation or random assignment. Independent

variable conditions are naturally existing.

• Archival- using data that has already been collected by another person

• Survey-collecting data by administering questionnaires in order to collect data. Note: while you do not use assignment, you can/will use random or non-random sampling!

• Correlational- using correlation to test the strength of the relationship between two variables.

How do you get from your research question to a research design?• Step 1: Look at your research question

and identify if you need an experimental design or not.

• Step 2: If experimental, decide if you need a true experiment or quasi-experimental design

YES?– For True Experimental Design

• Repeated Measures Design- experimental design where you compare pre-test scores with post-test scores and infer that differences are due to your manipulation of the IV.

– For Quasi-Experimental• Pretest-Posttest Nonequivalent Control Group Design-

same as above only non-random group assignment (quasi-experimental design)

Step 3: Decide if you need a pretest

Step 3 continued• No pretest? You have a between groups design,

however it is NOT repeated measures. – For True Experimental

• If you do not have a pretest, you have something called a randomized two-group design. While you have randomly assigned people to your conditions, all you’re looking at are the differences between groups on your post-test, because you have no pretest measure to compare people on.

– For Quasi-Experimental• static-group comparison design- differences in DV are compared

across groups without a pre-test measure in either group. Remember, one group will still receive a treatment while the other serves as the control.

Step 4: Do you have more than 2 groups?• YES?

• Pretest-Posttest Two Group Design- Repeated measures design where there are 2 groups

• Solomon Four-Group Design- 2 groups take pre-test and post-test AND 2 groups only take the post-test

Step 4 cont’d• Only One Group?

– Quasi-experimental • Interrupted-Time Series Design- This type of design

uses multiple pretests prior to your treatment and multiple posttests following the treatment in order to see if the treatment produces changes over and above secular trends

• One Group Pre-Test Post-Test Design- is also called simple panel research – this is a within groups design, since it looks at changes in your outcome within just one group, using the same participants prior to the treatment as after the treatment

• Step 5: If you have multiple pretests (and multiple posttests)…

• Interrupted Time Series Design- This type of design uses multiple pre-tests prior to your treatment and multiple post-tests following the treatment.

Type of Design Statistical Test Non-experimental/Observational Correlation

Regression Analysis of Covariance ANOVA Idependent Measures T-Test Repeated Measures T-Test

Randomized Two Group Design Independent Measures T-Test Pretest-Posttest Two Group Design

Repeated Measures T-Test

Solomon Four Group Design ANOVA Pretest-Posttest Nonequivalent Control Group Design

Repeated Measures T-Test

Static Group Comparison Independent Measures T-Test Interrupted Time Series Design Repeated Measures ANOVA One Group Pretest-Posttest Design

Repeated Measures T-Test (within group)

HomeworkSlide 1 of 4

Read the following three scenarios, each taken from published articles.

1. The purpose of the present study was to assess empirically the potential impact of the popular media on people’s attitudes toward U.S. corporations and their leadership. The sample consisted of 542 university students in Tokyo, Japan. The average age of the sample was 20.8 years, and 87.6% of the sample was male. Twelve sections of a management course required of all commerce students were randomly assigned to four experimental conditions. Two of the groups completed a questionnaire in class before Roger & Me was released in Japan. Approximately half of the students who completed the survey, and half who did not, then viewed Roger & Me in a local theater. Subsequently, all students completed the same questionnaire.

HomeworkSlide 2 of 4

2. The purpose of the current research is to determine if pregnant women ages 35 years and older have greater conflict in adapting to pregnancy than do younger pregnant women. Pregnant women who were older and more mature (ages 35 years and older) were compared with pregnant women who were younger (ages 32 years and younger). Women ages 33 and 34 years were excluded from the sample so that the two groups would be distinct and have differences in their mean ages for statistical purposes. A sample of women who were in their 3rd trimester of pregnancy (27-42 weeks gestation) was recruited from prenatal care providers and childbirth education classes. In the 3rd trimester, most women are completing the tasks that lead to their self-identification as a mother. Nurses who worked in prenatal care offices and prenatal education classes were told of the study, and their cooperation was invited. Women who met the study criteria were approached by nurses in their prenatal health care facility or prenatal education class. Women who expressed interest in the research program received a letter explaining the study. If they agreed to participate, they completed questionnaires attached to the introductory letter. Consent to participate was suggested by completion and return of the questionnaires. Women who chose not to participate are unknown, so no information is available about them. Using the Lederman Prenatal Self-Evaluation Questionnaire II, which contains seven scales measuring conflict in psychosocial tasks during pregnancy, older pregnant women had significantly less fear of helplessness and loss of control in labor than did younger pregnant women.

3. We examined the effectiveness of a cognitive intervention to help adolescents cope with stress and other forms of negative emotional arousal. Through this intervention procedure, youths learned how to identify and monitor stress-promoting cognitions, restructure these cognitions into more adaptive thoughts, use self-instructions to control stress-engendering self-statements, and practice

and apply these acquired skills. A total of 21 adolescent males, 16–17 years old, participated in the study. The project was conducted at a college preparatory, parochial high school for boys in a midwestern city. Admission to this school is based on competitive examinations taken by students from over 100 elementary schools in the surrounding area. In recent years, 98% of the graduates have enrolled in college. The nature and purpose of the project was described to juniors and seniors by the head guidance counselor, and any students who were interested in learning more about the stress-management project were asked to attend an orientation meeting. Originally, 30 youths attended this orientation meeting, during which the research was described in full by the researchers and parental and youth consent forms were distributed. Twenty-four youths returned signed consent forms and volunteered for the project. These youths were randomly assigned to either the experimental group that received training or a waiting list control group. Before the training was completed, three youths in the experimental group dropped out of the study because of schedule conflicts. As a result, the final sample was composed of 9 experimental youths and 12 control youths. All youths were White with the exception of one Black youth in the experimental group. Both groups were given pre- and post-assessments in 1-hr group sessions on the outcomes of interest.

Homework Slide 3 of 4

HomeworkSlide 4 of 4

1. For each scenario, identify the population, the sample, and the sampling method. Was the sampling adequate? (3 points)

2. For each scenario, identify the research design and explain why it was chosen. (3 points)

3. For your project, describe and justify a) the population, the sample, and the sampling method and b) the research design. (4 points)

APA Citations• Whenever you write about material

from one of your articles in your paper, you need to cite (give credit to the original source) it in the text. This way you avoid plagiarism!

Reference Section• References

– All articles cited in your paper should be listed in your references:

Dinkus, B., Petes, G. O., & Kuhn, D. (2000). Memoryin toddlers: A review. Journal of Consulting and Clinical Psychology, 112, 56-73

Dunn, D. F. (2002). Brain changes after statistical training. Neuropsychology, 17, 456-475.

Oswald, T. F. & Dinkus, B. (1999). Statistics and brain size. Psychological Bulletin, 42, 34-43.

APA in-text citations• Example:

Besides such “wired-in” differences in sensory and perceptual capacities, natural selection has favored many open-ended learning and memory biases fitted to species' ecological tasks. For example, rats, who have poor vision and rely on taste and smell to find food at night, easily condition aversions to novel tastes but not to visual stimuli (Garcia & Koelling, 1966). Quail, on the other hand, who have excellent vision and rely on visual cues in food choice, show the opposite learning bias (Wilcoxon, Dragoin, & Kral, 1971).

• Discussing Previous Research (use a citation)