Assignment 7 slides

PSYCHOLOGY 3800 LAB 002

•  two-way ANOVA feedback link

•  chi-square (!2) test: overview and applications

•  example analysis in SPSS- omnibus test

•  post hoc options in a chi-square analysis

•  this week’s assignment

Feedback on the assignment can be found on the lab blog in the form of a list of commonly made errors:

http://uwo3800g.tumblr.com/post/79396735608/assignment-5-commonly-made-errors

•  in this unit, we are no longer obtaining scores from participants, calculating means based on the scores, and comparing those means using statistical tests

" instead, we are working with frequency data and nominal variables

frequency data - the number of times that some event occurs - the number of objects or individuals who fit a give criterion or category - represented as a count or tally

nominal variables - classify data into categories - values in each category represent tallies rather than scores

(one variable; not carried out in your assignment)

Application #1: Goodness of Fit

•  testing whether observed frequencies are the same as theoretical (expected) frequencies for a single nominal variable

Example: A fair coin should come up heads half the time during a coin toss

" toss a coin repeatedly and record the number of heads " compare the observed number of heads to the expected number of heads to determine whether these values differ significantly " for this test, we want a small chi-square value (and a non-significant result), telling us that our observed values are not differing significantly from what is expected theoretically

…and don’t forget: correlation (or relation) ! causation

(two variables)

Application #2: Test of Association

•  testing whether two nominal variables are related

•  this test is similar to a correlation, only we cannot determine the degree of the relation (only whether or not there is some association)

(two variables)


Example: Is pet ownership related to an individual’s experience with depression?

Procedure: •  obtain sample of individuals

•  ask them to indicate their pet status: o  currently own a pet o  previously owned a pet o  never owned a pet

•  ask them to indicate their experience with depression o  have episodes of major depression o  have episodes of mild depression o  have no experience with depression

•  record number of individuals who fit in each category

Hypotheses:

HO: there is no significant relation between pet ownership and experience with depression (variables are independent)

HA: there is a significant relation between pet ownership and experience with depression (variables are dependent)

(two variables)


Design:

3 (depression) x 3 (pet ownership) chi-square test for association

Depression Pet ownership Current Previous Never

Severe 20 5 16 Mild 11 14 34 None 4 5 15

Example Data: Observed Frequency of Pet Ownership and Experience with Depression

35 24 65

41

59

24

Total participants: 124 " 35 + 24 + 65 = 124 or " 41 + 59 + 24 = 124

(two variables)


Example Data: Calculating Expected Frequency of Pet Ownership and Experience with Depression

(two variables)


Depression Pet ownership Current Previous Never

Severe

Mild

None !

(35)(41)124

=11.57

!

(35)(59)124

=16.65

!

(35)(24)124

= 6.77!

(24)(41)124

= 7.94

!

(24)(59)124

=11.42

!

(24)(24)124

= 4.64!

(65)(41)124

= 21.49

!

(65)(59)124

= 30.93

!

(65)(24)124

=12.58

35 24 65

41

59

24

!

E =column sum " row sum

total

(two variables)


Depression Pet ownership

Current Previous Never

Severe 20 5 16

Mild 11 14 34

None 4 5 15

Depression Pet ownership

Current Previous Never

Severe 11.57 7.94 21.49

Mild 16.65 11.42 30.93

None 6.77 4.64 12.58

The chi-square test compares the observed values to the expected values:

observed

expected

The chi-square test compares the observed values to the expected values:

•  the manner in which expected values are derived suggests consistent effect across all cells (think main effect… no relation between variables can be concluded)

•  if test is significant: some cells’ observed values differs from expected values, no longer suggesting a consistent effect (think interaction)

•  significant effect points to significant relation between variables

•  omnibus test cannot give us more details than this… must do post hoc procedures

(two variables)


•  independent replication o  each individual contributes to count in one cell only o  allows us (and SPSS) to derive an accurate total value to calculate the expected frequencies

•  normal sampling distribution of (E-O) o  distribution not tested directly o  assumed to hold true if many low expected values are not calculated for the data o  can assess low expected count via Cochran’s rule

•  cannot conduct chi-square test reliably if: •  expected value < 1 for 1 or more cells

•  expected < 5 for 20% or more cells

•  check footnote of “Chi-Square Tests” table in output

•  if either of the above options is true: a) collapse cells (in collapsing, will have to adjust p-value)

or b) get more subjects (or, for your assignment: increase the counts

within your cells)

pet ownership

1 = current 2 = previous 3 = never

depression status

1 = severe 2 = mild 3 = none

there are 20 individuals who currently own a pet (Pet: 1) and

have severe depression (Depression: 1)

there are 34 individuals who never owned a pet (Pet: 3) and have mild

depression (Depression: 2)

To make sure that SPSS reads the structure of your data correctly, do the following:

Data ! Weight Cases…

Analyze " Descriptive Statistics " Crosstabs

specify that you would like Chi-square output

Statistics Menu

Request: -observed counts -expected counts -unstandardized residuals

Cells Menu

Click “OK” to run analysis (output will open up in separate output window)

Observed Values

•  these are the values you typed into SPSS (or I did, in this case)

Expected Values

•  these are the values that SPSS calculates based on the row and column totals, and the total sample size

Residual Values

•  these values represent the difference between the observed and expected values in each cell

Example: 20 – 11.6 = 8.4

"2(4) = 13.062, p < .05

•  no issues with expected count identified (need at least 20% of cells to have expected frequency less than 5 to violate Cochran’s rule)

"  a significant relation exists between pet ownership and depression "  post hoc analyses needed to dissect effect

Significance of Relation Between Variables

Option #1: Extract Cells •  extract cells from full data table that have large residuals •  large residuals suggest some interaction •  choosing to ignore levels that do not suggest an association

Overall goal: create a 2 x 2 table that is more easily interpretable (i.e., compare two levels of one variable to 2 levels of the other)

Option #2: Collapse Cells •  collapse cells that exhibit similar types of effects •  will allow for a clearer, more magnified understanding of effects

!  re-run chi-square test on reduced data

•  the highlighted 4 cells have the largest residuals •  can extract these 4 cells to create a 2 x 2 table

Data " Select Cases…

select “If condition is satisfied” and click on “If…”

in this window, we identify which cells we wish to extract for additional analysis

((Depression=1) & (Pet=1)) | ((Depression=1) & (Pet=3)) | ((Depression=2) & (Pet=1)) | ((Depression=2) & (Pet=3))

Note: clicking “OK” will bring up your output window,

but this is just to show you the syntax that you ran… there are no new results to

examine just yet.

Inspect your data. Make sure that only the cells you want active are marked with a “1” under the “filter_$” column (and are not crossed out)

We have selected: current pet owners (1) with severe depression (1) current pet owners (1) with mild depression (2) non-pet owners (3) with severe depression (1) non-pet owners (3) with mild depression (2)


Re-run the chi-square test " SPSS will have remembered your previous selections so simply click “OK” in the menu that appears

this is your extracted data (it will help you with

your conclusion)

this is your "2 test (it will help you to

determine the significance

of your effect)

•  when you start collapsing/extracting cells, you are (in essence) making it easier for yourself to obtain significant results

•  need to adjust our p-values to be more conservative (more strict) to ensure that our results are sound

•  to ensure accurate results: use the Bonferroni correction to adjust p-values when data is extracted, and a conservative p-value when the data is collapsed or collapsed/extracted (combination of both)

Using a Bonferroni Correction: Steps

(1) calculate the number of possible 2 x 2 comparisons (k) that could be made using the following formula:

(2) divide .05 (the largest acceptable p-value) by k

(3) compare the significance value of your "2 test (performed on your new table) to your Bonferroni adjusted p-value

" if significance value in table is smaller than the adjusted p-value, conclude that test was significant at p < .05

!

k =r!

2!(r " 2)!#

c!2!(c " 2)!

r = number of rows in original table c = number of columns in original table

used when analyzing purely extracted cells

Using a Bonferroni Correction: Example

(1) number of possible 2 x 2 comparisons (k):

(2) divide .05 by k:

(3) D.R. reject HO (and conclude significant effect) if p < .006

!

k =3!

2!(3 " 2)!#

3!2!(3 " 2)!

!

=(3)(2)(1)(2)(1)(1!)

"(3)(2)(1)(2)(1)(1!)

!

=6

(2)(1)"

6(2)(1)

!

k =62"62

!

= 3 " 3

!

= 9

!

.059

= .006

p-value is indeed < .006

!2(1) = 8.194, p < .05

•  no issues with expected count identified (Cochran’s rule not violated)

"  conclusion

•  for individuals who currently own pets: greater frequency of severe depression reported than mild depression

•  effect is reversed for people who have never owned pets: greater frequency of mild depression rather than severe depression

•  cause/effect cannot be concluded so consider possible explanations

•  the highlighted cells for mild depression and no depression show pattern in same direction (increasing frequency across pet ownership categories) •  can be collapse and compared against severe depression

•  the highlighted cells for previous pet owners and non-pet owners show pattern in same direction •  can be collapse and compared against current pet owners

•  create two new variables representing the collapsed categories: " dep_severe_other " pet_current_other

•  specify 0 decimal places for both

•  specify both as a nominal measure

•  coding those with severe depression as 1 •  coding those with mild or no depression as 2

•  coding current pet owners as 1 •  coding previous and non-pet owners as 2

Data " Select Cases…

select “All cases” to turn off previous filters


Re-run the chi-square test with the new re-coded variables

this is your collapsed data (it will help you with your

conclusion)

this is your "2 test (it will help you to

determine the significance

of your effect)

(1) compare the significance value of your "2 test (performed on your new table) to a conservative p-value

" ideal choice: p < .001

Pick a Conservative p-Value: Steps

used when analyzing purely collapsed cells or a combination of collapsed and

extracted cells

(2) if significance value in table is smaller than the adjusted p-value, conclude that test was significant at conservative value

p-value is indeed < .001

!2(1) = 12.774, p < .001

•  no issues with expected count identified (Cochran’s rule not violated)

"  conclusion

•  for individuals with severe depression: relatively similar frequencies in terms of current ownership vs. no current ownership

•  for individuals with limited-to-no experience with depression: substantially fewer current pet owners versus no current pet ownership

•  cause/effect cannot be concluded so consider possible explanations

•  APA style results section

•  2 pages maximum (not including additional pages for tables)

•  no data file available " you will be making up your own data o  significant obtained chi-square (interaction) o  Cochran’s rule is not violated o  data makes sense (theoretically, in real world) o  total of at least 600 participants (can be more)

•  please include all output and hand calculations

Introduction •  description of study •  description of variables •  design used in analysis •  hypotheses (Ho, HA)

Overall Chi-Square Test •  assessment of assumptions •  obtained statistic and significance •  conclusion

Post Hoc Test •  description of approach and rationale •  assessment of assumptions •  obtained statistic and significance (with adjusted p-values as needed) •  conclusion

Tables •  two tables: original data (main analysis), post hoc data •  APA formatting for both tables

Final conclusion •  discussion of all findings in applied, non-statistical terms

Education

Assignment 7 slides