Education 795 Class Notes

Quasi-Experimental DesignPath Analysis

Note set 9

Today’s Agenda

Announcements (ours and yours

Q/A

Quasi-experimental design

Path analysis as an approach to variance partitioning and causal modeling

Quasi-Experimental Design

Historical HappeningsStatistical analyses evolved to meet needs of experimental designsQuasi-experimental designs evolved in the social sciencesResearchers continue to used experimental analyses for quasi and non experimental designsStatistical analyses emerged to meet the challenges of quasi and non experimental designsResearchers adopt new and improved techniquesNew statistical analyses continue to emerge to meet the many challenges that quasi and non experimental designs face…

non-random assignmentno manipulation of treatment

Nomenclature

Quasi-experimental designs refer to studies where no random assignment is in place

We cannot separate the irrelevant causal forces hidden within the ceteris paribus of random assignment (Cook & Campbell, 1979).

Refer to quasi-experimental when there is a treatment in place but no random assignment and we are interested in ‘causal effects’.Refer to non-experimental when we want to explain differences among groups

The Term “Treatment”

Treatments can be: interventions

direct manipulation of a variable

naturally occurringabrupt and precisely dated

training programs

exposure to a condition

The Omitted Variables Conundrum

Y

X

Y e

X

Y e

AA B

When the error is correlated with the treatment (X) we cannot separate out the “treatment” effect from spurious effects

How Do We Deal With This Problem to Get at Causation?

Return to a regression-based approach, and introduce a special kind of regression called Path Analysis

Introduce Structural Equation Modeling

Review Regression Formula

Raw score depiction:

where each b:is the unique and independent contribution of that predictor to the modelfor quantitative IVs, the expected direction and amount of change in the DV for each unit change in the IV, holding all other IVs constantFor dichotomous IVs, the direction and amount of group mean difference on DV, holding all other IVs constant

Review Venn Diagram

CorrelationRegression Coefficients

Venn Diagrams

CorrelationPartial

Correlation

Relationships among 3 variables

B

S

A

B

S

A

B

S

A

B

S

A

A.

B.

C.

D.

Path Analysis

A structured approach to regression analysis allowing intervening variables

NomenclaturePath AnalysisLISREL ModelsCausal ModelsANCOVA ModelsLatent Variable ModelsStructural Equation Models

Causation

Most controversial topic in philosophy of science and has been characterized as ‘a notorious philosophical tar pit’ (Davis, 1985, p. 8)

The history of the topic extends over centuries

Random Selection of Quotes

‘Cause is the most valuable concept in the methodology of the applied sciences’ (Scriven, 1968, p. 79)

‘Let’s drop the word cause and bring educational research out of the middle ages’ (Travers, 1981, p. 32)

‘It would be very healthy if more researchers abandon thinking of and using terms such as cause and effect’ (Muthien, 1987, p. 180)

No causation without manipulation (Holland & Rubin, 1986)

Return to the Role of Theory

Causal analyses are based on theory.The temptation to apply sophisticated state-of-the-art methodologies seem irresistable

It is important to recognize when a given methodology is inapplicable

‘In sum, the formulation of a causal model is an arduous and long process entailing a great deal of critical thinking, creativity, insight and erudition’ (Pedhazur & Pedhzur, 1991, p. 699)

Definitions

Exogenous Variable– variable with arrows ONLY going out of it in the model (Strictly predictor)

Endogenous Variable—variable with arrows going IN—it can also have arrows going out (Outcome and possibly a mediated predictor)

Definitions

Direct Effect—the effect of a variable that has a direct path to the outcome.

Indirect Effect—the effect of a variable on an outcome that travels through (is mediated by) other variables in the model

Total Effect—Sum of the direct and indirect effects for one variable on the outcome

Definitions

X, Z predictors, Y, outcome

SpuriousnessThe relationship between X and Y is said to be spurious if Z causes X and Y

Unexplained covariationBoth X and Y are exogenous and so variation between them is not explained by the model

Example: Understanding the Effects of Frog Ponds

Theoretical discussion started by odd findings related to student achievement

Reference-group theory: Environmental press or relative deprivation?

Is it better to be a small frog in a big pond, or a big frog in a small pond?

Path Analysis Example

Bassis model

With respect to academic self-evaluation, is it better to be a small frog in a big pond, or a big frog in a small pond?

Bassis Model

Descriptive statistics

Bassis Results

Decomposition of r

Correlation between two endogenous variables:

r = Direct Effect + Indirect Effects + Spuriousness

Correlation between an endogenous variable and an exogenous variable:

r = Direct Effect + Indirect Effects + Unspecified Covariance

Calculating Path Coefficients

Compute the appropriate regression analyses, and organize the resulting coefficientsFirst, calculate the indirect components by multiplying the involved coefficientsSecond, sum the indirect components and add to the direct coefficient (if any) to calculate the total effect

Computing Exercise

Calculate the direct and indirect effects of:

a) Selectivity on 4-year academic self-ratingb) HS grades on college grades

Comparing Models

One interesting use of path analysis is to directly compare models

Higher education as a contextualized experience:

Are frog pond effects similar in majority / specialized institutions for students served by such specialized institutions?

African American Students inPWIs and HBCUs

PWIs

HBCUs

Women Students in Coed and Single-sex Institutions

Coed

Womens’colleges

For Next Week

Read Pedhazur Ch 7 p 152-157

Read Aldrich & Nelson, ALL