Correlation & Causal Comparative Research

Class 6

This Week’s Schedule

• Today – Review and continue w/ statistical analysis• Tuesday– 3 individual meetings 9-10am– Full class (Stats & Method) 10-11am– 3 individual meeting 11am-12noon

• Wednesday– 9am-10am – Music ed history (Skype w/ Eastman Class)

Show and tell!!– 10:00am-11:00am – Qualitative Research-Full class– 11:00am-12noon-3 individual meetings

This Week’s Schedule

• Thursday– 4 Project Presentations (20 minutes)– 2 qualitative/historical 5ish minute presentations

(in pairs & a trio)– Disseminating research

• Friday– 5 Project Presentations– 2 qualitative/historical 5ish minute presentations

(in pairs & a trio)

Assignments• Tuesday – Work on presentations, projects, etc.• Wednesday

– Read Queen Bees and Wanna Bees chapt. 1 OR 6– Read one historical article from the Journal of Historical

Research in Music Education. Be prepared to write or discuss– Chapter 3 – Method

• Thursday & Friday– Project Presentations (20 minute w/ 5 minutes for questions &

discussion)– Informal presentation in pairs of a qualitative or an historical

article• Monday, July 22 by 5pm – Final Project Proposal

Who & When

• Tuesday Meetings– 9-10 (3)– 11-noon (3)

• Weds Meetings– 11-noon (3)

• Thursday– Project Presentations

(4)– Qual./Hist.

presentations (2 pairs)

• Friday– 5 presentations– Qual./Hist. trio

presentation

APA Format

• Headers– Chapter title = Level 1– Others = Level 2 (Flush left)

• Remember title page, page #s• Running Head – < 50 characters total. Goes in the

header flush left• Research Question after purpose statement & before

need for study (Header?)• Commas = …apples, oranges, and grapes.

Types of Data – Revisedsimple to complex; lowest to highest

• Nominal/Categorical = numbers as labels– Male/female (1 or 2) – Sop/Alto/tenor/bass (1, 2, 3, 4)

• Ordinal = ranks– Contest ratings

• Interval = Scale (equal distance b/w each number)– Contest scores (1-100)– Lack of meaningful zero (0 on test = no knowledge?, 0 temperature =

arbitrary) or meaningful ratios (2x as smart?)• Ratio =

– Equal interval data– True zero possible (0 decibels, 0 money)– Ratios can be calculated in a meaningful way [2x as loud, ½ money,

height, weight, depth (a lake can dry up) (?), etc.]

Terms

• Inferential statistics• Parametric vs. non-parametric• Assumptions/Parameters– Variances?– Randomization

• Mean vs. variance• Used to compare 2 groups and no more?• Independent vs. dependent (paired or correlated)• One tail vs. Two tail tests?

Terms

• What is I have more than 2 groups? I need a…?

• If there is a significant difference in the test above, then what do I need to do?

• Why do we test the significance of the difference in variances?

• What if the variances are sig. different?

Statistical Significance

• Probability that result happened by chance and not due to treatment– Expressed as p– p < .1 = less than 10% (1/10) probability…– p < .05 = less than 5% (1/20) probability…– p < .01 – less than 1% (1/100) probability…– p < .001 – less than .1% (1/1000) probability…

• Computer software reports actual p• alpha level = probability level to be accepted as

significant set b/f study begins• Statistical significance does not equal practical

significance

Statistical Power

• Likelihood that a particular test of statistical significance will lead to the rejection of null hypothesis– Parametric tests more powerful than nonparametric. (Par.

more likely to discover differences b/w groups. Choice depend on type of data)

• The larger the sample size, the more likely you will be to find statistically significant effects.

• The less stringent your criteria (e.g., .05 vs. 01 vs. 001), the easier it is to find statistical significance

Statistical Tests

http://pspp.awardspace.com/ (Windows)http://bmi.cchmc.org/resources/software/pspp (Mac)

http://vassarstats.net/

See Handout from Friday

• Awareness of non-parametric tests• 3 groups, ordinal data?• 2 groups, interval data?• 2 groups, nominal/categorical data?• Relationship b/w two groups, ordinal data?

Independent Samples t-test• Used to determine whether differences between two

independent group means are statistically significant• n = < 30 for each group. Though many researchers

have used the t test with larger groups.• Groups do not have to be even. Only concerned with

overall group differences w/o considering pairs– [A robust statistical technique is one that performs well even if its

assumptions are somewhat violated by the true model from which the data were generated. Unequal variances = alternative t test or better Mann-Whitney U]

• Application: Explore Data– Compare science tests of inst & non-inst. students

Correlated (paired, dependent) Samples t-test

• Used to determine differences between two means taken from the same group, or from two groups with matched pairs are statistically significant– e.g., pre-test achievement scores for the whole song group

vs. post-test achievement scores for the whole song group• Group size must be even (paired)• N = < 30 for each group

• Application: Compare Reading & Math test scores of Instrumental Students

Compare 2 means

• Need sample of at least 10• Work like Independent and dependent t tests• Independent– Mann Whitney U

• Application: Data set #3. Is there a sig. diff. b/w Final ratings at Site 1 vs. site 2?

• Pairs or dependent samples– Wilcoxon signed ranks

• Application: Data set #2. Is there a sig. difference b/w rating of judges 1 & 2?

ANOVA

• Analyze means of 2+ groups• Homogeneity of variance• Independent or correlated (paired) groups• More rigorous than t-test (b/w group & w/i group

variance). Often used today instead of T test.• F statistic• One-Way = 1 independent variable• Two-Way/Three-Way = 2-3 independent variables

(one active & one or two an attribute)

One-Way ANOVA

• Calculate a One-Way ANOVA for data-set 1 – All non-instrumental tests

• Post Hoc tests– Used to find differences b/w groups using one test. You

could compare all pairs w/ individual t tests or ANOVA, but leads to problems w/ multiple comparisons on same data

– Tukey – Equal Sample Sizes (though can be used for unequal sample sizes as well)

– Sheffe – Unequal Sample Sizes (though can be used for equal sample sizes as well)

ANCOVA – Analysis of Covariance

• Statistical control for unequal groups• Adjusts posttest means based on pretest

means.• [example]

http://faculty.vassar.edu/lowry/VassarStats.html

• [The homogeneity of regression assumption is met if within each of the groups there is an linear correlation between the dependent variable and the covariate and the correlations are similar b/w groups]

Effect Size (Cohen’s d) http://www.uccs.edu/~faculty/lbecker/es.htm

http://www.uccs.edu/~lbecker/

• [Mean of Experimental group – Mean of Control group/average SD]• The average percentile standing of the average treated (or experimental)

participant relative to the average untreated (or control) participant.• Use table to find where someone ranked in the 50th percentile in the

experimental group would be in the control group• Good for showing practical significance

– When test in non-significant– When both groups got significantly better (really effective vs. really

really effective!

• Calculate effect size:– Treatment group: M=24.6; SD=10.7– Control Group: M=10.8; SD=7.77

Cohen's Standard Effect Size Percentile Standing Percent of Nonoverlap

  2.0 97.7 81.1%

  1.9 97.1 79.4%

  1.8 96.4 77.4%

  1.7 95.5 75.4%

  1.6 94.5 73.1%

  1.5 93.3 70.7%

  1.4 91.9 68.1%

  1.3 90 65.3%

  1.2 88 62.2%

  1.1 86 58.9%

  1.0 84 55.4%

  0.9 82 51.6%

LARGE 0.8 79 47.4%

  0.7 76 43.0%

  0.6 73 38.2%

MEDIUM 0.5 69 33.0%

  0.4 66 27.4%

  0.3 62 21.3%

SMALL 0.2 58 14.7%

  0.1 54 7.7%

  0.0 50 0%

Chi-Squared

• Measure statistical significance b/w frequency counts (nominal/categorical data)

• http://www.quantpsy.org/chisq/chisq.htm • Test for independence: Compare 2 or more proportions• Goodness of Fit: compare w/ you have with what is

expected– Proportions of contest ratings (I, II, III or I & non Is)– Agree vs. Disagree

• Weak statistical test

Correlation

PearsonSpearman

Cronbach’s alpha (α)

Correlational Research Basics

• Relationships among two or more variables are investigated

• The researcher does not manipulate the variables

• Direction (positive [+] or negative [-]) and degree (how strong) in which two or more variables are related

Uses of Correlational Research

• Clarifying and understanding important phenomena (relationship b/w variables—e.g., height and voice range in MS boys)

• Explaining human behaviors (class periods per weeks correlated to practice time)

• Predicting likely outcomes (one test predicts another)

Uses of Correlation Research

• Particularly beneficial when experimental studies are difficult or impossible to design

• Allows for examinations of relationships among variables measured in different units (decibels, pitch; retention numbers and test scores, etc.)

• DOES NOT indicate causation– Reciprocal effect (a change in weight may affect body image, but body

image does not cause a change in weight)– Third (other) variable actually responsible for difference (Tendency of

smart kids to persist in music is cause of higher SATs among HS music students rather than music study itself)

Interpreting Correlations– r

• Correlation coefficient (Pearson, Spearman)• Can range from -1.00 to +1.00

– Direction• Positive

– As X increases, so does Y and vice versa• Negative

– As X decreases, Y increases and vice versa– Degree or Strength (rough indicators)

• < + .30; small• < + .65; moderate• > + .65; strong• > + .85; very strong

– r2 (% of shared variance)• % of overlap b/w two variables• percent of the variation in one variable that is related to the variation

in the other.• Example: Correlation b/w musical achievement and minutes of

instruction is r = .86. What is the % of shared variance (r2)?– Easy to obtain significant results w/ correlation. Strength is most

important

Application

• Rate your principal & school quality on a scale of 1-7 • Principal: (1=highly ineffective; 2=ineffective;

3=somewhat ineffective; 4=neither effective nor ineffective; 5=somewhat effective; 6=effective; 7=highly effective

• School cleanliness: (1=very dirty; 2=dirty; 3=somewhat dirty; 4=neither dirty or clean; 5=somewhat clean; 6=clean; 7=very clean)

• Type of data? Calculation (Pearson or Spearman?)• Reliability (Cronbach’s alpha)

www.gifted.uconn.edu/siegle/research/.../reliabilitycalculator2.xls

Interpreting Correlations (cont.)• Words typically used to describe correlations

– Direct (Large values w/ large values or small values w/ small values. Moving parallel. 0 to +1

– Indirect or inverse (Large values w/small values. Moving in opposite directions. 0 to -1

– Perfect (exactly 1 or -1)– Strong, weak– High, moderate, low– Positive, Negative

• Correlations vs. Mean Differences– Groups of scores that are correlated will not necessarily have

similar means (e.g., pretest/posttest). Correlation also works w/ different units of measurement.

50 75 9 40 62 1435 53 2024 35 4515 21 58

Statistical Assumptions• The mathematical equations used to determine various correlation

coefficients carry with them certain assumptions about the nature of the data used…– Level of data (types of correlation for different levels)– Normal curve (Pearson, if not-Spearman)– Linearity (relationships move parallel or inverse)

• Non linear relationship of # of performances & anxiety scores = Young students initially have a low level of performance anxiety, but it rises with each performance as they realize the pressure and potential rewards that come with performance. However, once they have several performances under their belts, the anxiety subsides. (

– Presence of outliers (all)– Ho/mo/sce/da/sci/ty – relationship consistent throughout

• Performance anxiety levels off after several performances and remains static (relationship lacks Homoscedascity)

– Subjects have only one score for each variable

Correlational Approaches for Assessing Measurement Reliability

• Consistency over time– test-retest (Pearson, Spearman)

• Consistency within the measure– internal consistency (split-half, KR-20,

Cronbach’s alpha)– Spearman Brown Prophecy formula

• 2r/(1 + r)

• Among judges– Interjudge (Cronbach’s Alpha)

• Consistency b/w one measure and another– (Pearson, Spearman)

Reliability of Survey

• What broad single dimension is being studied?– e.g. = attitudes towards elementary music– Preference for Western art music– “People who answered a on #3 answered c on #5”

• Use Cronbach’s alpha– Measure of internal consistency– Extent to which responses on individual items

correspond to each other

Spearman Brown Prophesy Formula

• Reliability = ___n x r___ 1+(n-1)r

• n=number of times we multiply items to get new test length (10 item to 20 item – n=2)

• For a test of 10 items w/ reliability (α) of .60– (15 items) 1.5 x .60/1+(1.5 - 1).60 = reliability for test 1.5x size– (20 items) 2 x .60/1+(2-1).60 = reliability for a test 2x size– (25 items) 2.5 x .60/1+(2.5 – 1).60 = reliability for test 2.5x size– (5 items) .5 x .60/1+(.5 – 1).60 = reliability for test .5 size