Non-Experimental Design Where are the beakers??. What kind of research is considered the “gold standard” by the Institute of Education Sciences? A.Descriptive

Non-Experimental DesignNon-Experimental Design

Where are the beakers??Where are the beakers??

What kind of research is considered What kind of research is considered the “gold standard” by the Institute of the “gold standard” by the Institute of Education Sciences?Education Sciences?

A.A. DescriptiveDescriptive

B.B. Causal-ComparativeCausal-Comparative

C.C. CorrelationalCorrelational

D.D. ExperimentalExperimental

Why?Why?

Why does most educational Why does most educational research use non-experimental research use non-experimental

designs?designs?

There are ethical and logistical There are ethical and logistical considerations that often impede the use considerations that often impede the use of experimental studies.of experimental studies.

What is the purpose of What is the purpose of non-experimental designs?non-experimental designs?

Describe current existing characteristics Describe current existing characteristics such as achievement, attitudes, such as achievement, attitudes, relationships, etc.relationships, etc.

There is no manipulation of an There is no manipulation of an independent variableindependent variable

First…First…Some Thought QuestionsSome Thought Questions

Causal-Comparative DesignCausal-Comparative Design

A study in which the researcher attempts to A study in which the researcher attempts to determine the cause, or reason, for pre-determine the cause, or reason, for pre-existing differences in groups of individualsexisting differences in groups of individuals

At least two different groups are compared on At least two different groups are compared on a a dependent variabledependent variable or measure of or measure of performance (called the “effect”) because the performance (called the “effect”) because the independent variableindependent variable (called the “cause”) has (called the “cause”) has already occurred or cannot be manipulated already occurred or cannot be manipulated


A “kissing cousin” to A “kissing cousin” to correlationalcorrelational research design.research design.

Ex-post factoEx-post facto– Causes studied after they have exerted Causes studied after they have exerted

their effect on another variable.their effect on another variable.


DrawbacksDrawbacks– Difficult to establish causality based on Difficult to establish causality based on

collected data.collected data.– Unmeasured variables (Unmeasured variables (confoundingconfounding

variables) are always a source of potential variables) are always a source of potential alternative causal explanations.alternative causal explanations.

Causal-Comparative ExampleCausal-Comparative Example

Green & Jaquess (1987) Green & Jaquess (1987) – Interested in the effect of high school Interested in the effect of high school

students’ part-time employment on their students’ part-time employment on their academic achievement.academic achievement.

– Sample: 477 high school juniors who were Sample: 477 high school juniors who were unemployed or employed > 10 hours/wk.unemployed or employed > 10 hours/wk.

Correlational DesignCorrelational Design

Determines whether and to what degree Determines whether and to what degree a relationship exists between two or a relationship exists between two or more quantifiable variables.more quantifiable variables.

Example of CorrelationExample of Correlation

Correlational DesignCorrelational Design

The degree of the relationship is The degree of the relationship is expressed as a coefficient of correlationexpressed as a coefficient of correlation

ExamplesExamples– Relationship between math achievement Relationship between math achievement

and math attitudeand math attitude– Relationship between degree of a school’s Relationship between degree of a school’s

racial diversity and student use of racial diversity and student use of stereotypical languagestereotypical language

– Your topics?Your topics?

Correlation coefficient…Correlation coefficient…

-1.00 +1.00

strong negative strong positive

0.00

no relationship

Advantages of Correlational DesignAdvantages of Correlational Design

Analysis of relationships among a large Analysis of relationships among a large number of variables in a single studynumber of variables in a single study

Information about the Information about the degreedegree of the of the relationship between the variables being relationship between the variables being studiedstudied

CautionsCautions

A relationship between two variables A relationship between two variables does not mean one causes the other does not mean one causes the other (Think about the reading achievement (Think about the reading achievement and body weight correlations on p. 189)and body weight correlations on p. 189)

Possibility of low reliability of the Possibility of low reliability of the instruments makes it difficult to identify instruments makes it difficult to identify relationshipsrelationships

CautionsCautions

Lack of variability in scores (e.g. Lack of variability in scores (e.g. everyone scoring very, very low; everyone scoring very, very low; everyone scoring very, very high; etc.) everyone scoring very, very high; etc.) makes it difficult to identify relationshipsmakes it difficult to identify relationshipsLarge sample sizes and/or using many Large sample sizes and/or using many variables can identify significant variables can identify significant relationships for statistical reasons and relationships for statistical reasons and not because the relationships really exist not because the relationships really exist (Avoid (Avoid shotgunshotgun approach) approach)

CautionsCautions

Need to identify your sample to know Need to identify your sample to know what is actually being compared.what is actually being compared.

If using predictor variables, time interval If using predictor variables, time interval between collecting the predictor and between collecting the predictor and criterion variable data is important.criterion variable data is important.

Correlational DesignsCorrelational Designs

Guidelines for interpreting the size of Guidelines for interpreting the size of correlation coefficientscorrelation coefficients– Much larger correlations are needed for Much larger correlations are needed for

predictions with individuals than with groupspredictions with individuals than with groups

Crude group predictions can be made with Crude group predictions can be made with correlations as low as .40 to .60correlations as low as .40 to .60

Predictions for individuals require Predictions for individuals require correlations above .75correlations above .75


Guidelines for interpreting the size of Guidelines for interpreting the size of correlation coefficientscorrelation coefficients– Exploratory studiesExploratory studies

Correlations of .25 to .40 indicate the need Correlations of .25 to .40 indicate the need for further researchfor further research

Much higher correlations are needed to Much higher correlations are needed to confirm or test hypotheses confirm or test hypotheses


Criteria for evaluating correlational studiesCriteria for evaluating correlational studies– Causation should not be inferred from Causation should not be inferred from

correlational studiescorrelational studies– Practical significance should not be confused Practical significance should not be confused

with statistical significancewith statistical significance


Criteria for evaluating correlational studiesCriteria for evaluating correlational studies– The size of the correlation should be The size of the correlation should be

sufficient for the use of the results sufficient for the use of the results (individuals vs groups)(individuals vs groups)

– Prediction studies should report the accuracy Prediction studies should report the accuracy of predictions for new subjectsof predictions for new subjects

– Procedures for collecting data should be Procedures for collecting data should be clearly indicatedclearly indicated

Think…Think…

If you were going to take your action If you were going to take your action research topic, and create a causal-research topic, and create a causal-comparative study, what would it look comparative study, what would it look like?like?

--OR----OR--If you were going to take your action If you were going to take your action research project, and create a research project, and create a correlational study, what would it look correlational study, what would it look like?like?

Documents

Non-Experimental Design Where are the beakers??. What kind of research is considered the “gold standard” by the Institute of Education Sciences? A.Descriptive