Types of research studies

1

CASE STUDIES

EXPERIMENTS

DESIGN STUDIES

SURVEYS

MITE 6025 Session 2 (van Aalst)9/26/2013

What is an experiment?

A study that tests a hypothesis about a causal effect using at least two independent samples

The experimental sample receives the intervention of interest

The control sample does not receive that intervention but is equivalent in all other respects

2

2. E

xpe

rim

ents

MITE 6025 Session 2 (van Aalst)9/26/2013

Hypotheses

A hypothesis is an educated guess

about what is going to happen during the

experiment

It usually is based on an implication

(prediction) of a theory

MITE 6025 Session 2 (van Aalst) 39/26/2013

2. E

xpe

rim

ents

Pre- and post-test

Having experimental and control samples not enough

Also need to test both samples before (pre-test) and after the treatment (post-test)

4 MITE 6025 Session 2 (van Aalst)9/26/2013

2. E

xpe

rim

ents

Simplest experimental design

Experimental Control

• Pretest on

dependent

variable

• Pretest on

other relevant

variables

• Pretest on

dependent

variable

• Pretest on

other relevant

variables

• Treatment • No treatment

• Posttest on

dependent

variable

• Posttest on

dependent

variable

Generalization to

population

Random selection

from population

Random

assignment to

conditions

MITE 6025 Session 2 (van Aalst) 59/26/2013

2. E

xpe

rim

ents

Sample results for dependent

variable

Experimental Control

Pretest 50.1 (10.2) 48.4 (10.7)

Posttest 72.0 (8.5) 60.9 (10.9)

The samples are similar at the pretest

The experimental sample outperforms the control sample on post-test

Need to calculate the probability of getting this effect

MITE 6025 Session 2 (van Aalst) 69/26/2013

2. E

xpe

rim

ents

Preparation (pretests)

English proficiency test

Chinese proficiency test

General science achievement test

(Chinese)

Specific science test (same language as

task)

MITE 6025 Session 2 (van Aalst) 79/26/2013

2. E

xpe

rim

ents

Task design

A reading task in English and Chinese

Exactly the same content in both versions

The same reading level in each language

Not too long

MITE 6025 Session 2 (van Aalst) 89/26/2013

2. E

xpe

rim

ents

Finishing up (post-tests)

Specific science post-test for both

samples

Time for completion for both samples

MITE 6025 Session 2 (van Aalst) 99/26/2013

2. E

xpe

rim

ents

SummaryStage Experimental sample Control sample

Preparation • English proficiency

test

• Chinese proficiency

test

• General science test

(Chinese)

• Specific science test

(English)

• English proficiency

test

• Chinese proficiency

test

• General science test

(Chinese)

• Specific science test

(Chinese)

intervention • Reading task

(English)

• Task completion

time

• Reading task

(Chinese)

• Task completion

time

Finishing up • Specific science test

(English)

• Specific science test

(Chinese)

MITE 6025 Session 2 (van Aalst) 109/26/2013

2. E

xpe

rim

ents

Sample results

Experimental (n = 30) Control (n = 30)

English test 672 (130) 520 (160)

Chinese test 620 (150) 590 (140)

General science test 50.1 (10.0) 42.8 (10.3)

Specific science pre-test 40.0 (9.5) 35.8 (10.2)

Specific science

posttest

60.1 (9.9) 57.9 (13.2)

Completion time 17.5 (3.5) min 25.7 (3.9) min

MITE 6025 Session 2 (van Aalst) 119/26/2013

2. E

xpe

rim

ents

Extensions

More than one treatment

More dependent variables

More control variables

MITE 6025 Session 2 (van Aalst) 129/26/2013

2. E

xpe

rim

ents

Quasi-experiments

When used

random selection and/or random assignment are impossible

Sample equivalence is difficult to achieve

E.g. the use of in-tact classes

Assignment to classes not random

One class does the treatment, the other class does not

Classes may be taught by different teachers

MITE 6025 Session 2 (van Aalst) 139/26/2013

2. E

xpe

rim

ents