Current Methodological Issues in Single Case Research

David Rindskopf, City University of New York

Rob Horner, University of Oregon

Purposes Inclusion of Single-case Designs in education

research literature. Interpretation of Single-case Research

Visual Analysis Statistical Analysis

Impact of Single-case Research Identification of “Evidence-based Practices” Inclusion of Single-case Research in Meta-

analyses

Toward a Comprehensive Analysis of Single-case Designs

Rob Horner, George Sugai,

Hariharon Swaminathan, Tom Kratochwill, and Joel Levin

Focusing the Discussion We are concerned here only with single-case experimental

designs* ABAB Reversal/withdrawal Designs Multiple Baseline Designs Alternating Treatment Designs• Note: these designs are sometimes conceptualized as within,

combined, and between-series, respectively.

Excluded are AB designs, ABA designs, Interrupted Time Series Case Studies.

The three types of designs can be extended into randomized experiments (Kratochwill & Levin, 2007).

Single Case Research Logic Individual is unit of analysis Dependent Variable

Operationally defined Measured directly Repeated measurement

Independent Variable Measured for implementation fidelity

Replication of effect within an experimental design To control for threats to internal validity

Analysis Needs

Analysis

Questions

Group Designs

Single-case Designs

Experimental Control

(Statistical Significance)

Design

Analysis

RCT, RD

Statistical

MBL, ABAB, etc

Visual Analysis

Effect Size Cohen’s d, r, etc

PND, ?

Social Validity

X

X

X

Visual Analysis Documenting Experimental Control

Three demonstrations of an “effect” at three different points in time. An “effect” is a predicted change in the dependent

variable when the independent variable is actively manipulated.

Visual Analysis includes simultaneous assessment of: Level, Trend, Variability, Immediacy of Effect,

Overlap across Adjacent Phases, Similarity of Data Pattern in Similar Phases.

(Parsonson & Baer, 1978; Kratochwill & Levin, 1992)

Visual Analysis Interpreting experimental control always

involves assessment of data from the whole study (all phases), not just assessment of two adjacent phases. Assessment of an “effect” is done with 2 adjacent

phases. Assessment of experimental control, however,

requires evaluation of all data in all phases.

First Demonstration of Effect

Second Demonstration of Effect

Third Demonstration of Effect

Visual Analysis:

1. Change in Level

2. Change in Trend

3. Change in Variability

4. Immediacy of Effect

5. Overlap

6. Similarity of Data Patterns and Phases

Parsonson & Baer, 1978;

Kratochwill & Levin, 1992

First Demonstration of Effect

Second Demonstration of Effect

Third Demonstration of Effect

Comparison of actual against projected data

(Analysis of Transition States versus Analysis of Steady States)

First Demonstration of Effect

Second Demonstration of Effect

Third Demonstration of Effect

First Demonstration of Effect

Second Demonstration of Effect

Third Demonstration of Effect

Statistical Analyses of Single-case Research Documenting Experimental Control/

Statistical Significance: No statistical model currently replicates the

conceptual logic used in visual analysis.

Effect Size Not a variable traditionally addressed in Single-

case Analyses. It needs to be.

Documenting “Evidence-based Practices” via Single-case Research Define “practice” Build professional consensus

Defining a “Practice” A “practice” is a procedure, or set of procedures,

designed for use in a specific context, by individuals with certain skills/features, to produce specific outcomes for specific individuals.

Operationally defined procedures Target population/ Context Implementer Characteristics Defined outcomes

Large practices…small practices

Toward a Professional Consensus Five studies documenting experimental

control Conducted in at least three different locations

by at least three different researchers At least 20 different participants Documenting an effect size of ??

Example of using Single-Case Research to document Evidence-based Practice Functional Communication Training to reduce

problem behavior in school, home and community At least Five Studies

Bird, Dores, Moniz & Robinson (1989) Brown et al., (2000) Carr & Durand (1985) Durand & Carr (1987) Hagopian, Fisher, Sullivan, Acquisto & LeBlanc (1998) Mildon, Moore, & Dixon (2004) Wacker et al., (1990)

At least three settings /scholars Six locations, Research Groups

At least 20 participants 42 different participants document effect

Implications for Single-Case Research Measures

Define assumptions about trajectory Baseline

Document predictable patterns More data points if trend More data points if higher variability

Data points per phase At least 5 points per phase More data when trend and variability is high

Combination of visual and statistical analysis Need for effect size measures in single-case designs

For individual studies For meta-analyses

Summary The inclusion of single-case research findings in a

larger body of education research will require attention to added strategies for interpretation, and added strategies for application

Interpretation Better definition of visual analysis procedures Agreement on effect size index

Application Definition of “evidence-based practices” Protocol for inclusion of single-case research in meta-

analyses.