Small N - Large N: Some Small N - Large N: Some AlternativesAlternatives

Ray Kent

University of Stirling

Research Methods Festival, Oxford, July 2006

Limitations of mainstream Limitations of mainstream quantitative methodsquantitative methods

• The focus is on the variableThe focus is on the variable• The thinking in linearThe thinking in linear• The main pattern sought is The main pattern sought is

covariationcovariation

Count

49 1 50

1 49 50

50 50 100

Set Member

Non-member

VariableY

Total

Set Member Non-member

Variable X

Total

Cramer’s V =0.96

Traditional analysis expects to see this:

Or this:

Hours viewing per week per household

160140120100806040200

Sp

en

d p

er

we

ek

on

co

nve

nie

nce

fo

od

(£

)50

40

30

20

10

0

r = 0.86

(Var X)

(Var Y)

Heavy television viewing is a sufficient, but not necessary condition for large expenditure on convenience food

Phi (Cramer’s V) = 0.37 Lambda = 0.0

Count

53 24 77

43 0 43

96 24 120

Large

Not large

Spend on conveniencefood

Total

Not heavy Heavy

Television viewing

Total

But we often get this:

Or this:

r = 0.3

Hours viewing per week per household

160140120100806040200

Sp

en

d p

er

we

ek

on

co

nve

nie

nce

fo

od

(£

)50

40

30

20

10

0

Further limitationsFurther limitations

• Not good at handling causal or Not good at handling causal or logical relationshipslogical relationships

• Poor at handling complexityPoor at handling complexity

Some common misusesSome common misuses

• The use (even reliance) on statistical The use (even reliance) on statistical inference on non-random samples or inference on non-random samples or total populationstotal populations

• Causal inferences based on Causal inferences based on establishing covariationestablishing covariation

• Poor, vague wording of hypothesesPoor, vague wording of hypotheses

Some alternatives to mainstream Some alternatives to mainstream statisticsstatistics

• Combinatorial logicCombinatorial logic

• Fuzzy-set analysisFuzzy-set analysis

• Neural network analysisNeural network analysis

• Data miningData mining

• Bayesian methodsBayesian methods

• Chaos/tipping point theoryChaos/tipping point theory

Combinatorial logicCombinatorial logic

Instead of comparing variable Instead of comparing variable distributions, we see cases as distributions, we see cases as combinations of characteristicscombinations of characteristics

A data matrix on SPSS

Configuration Cause X1 Cause X2 Cause X3 Frequency Outcome Y 1 1 1 1 30 1 2 1 1 0 15 1 3 1 0 0 5 1 4 1 0 1 12 0 5 0 1 1 2 0 6 0 1 0 5 0 7 0 0 1 0 NA 8 0 0 0 14 0

1=cause/ outcome present 0=cause/ outcome absent NA= not applicable

X1 is a necessary, but not sufficient, cause of Y

The frequency of 2k combinations of 3 binary causal variables plus binary outcome

1.0 0.50.75 0.25 0.0

Fully in

Mostly in

Neither in nor out

Mostly out

Fully out

A fuzzy set

0.25 0.5 0.75 1.00.0

0.25

0.5

0.75

1.0

Membership in set X1

Mem

bers

hip

in s

et Y

X1 is a necessary, but not sufficient, condition for Y to occur

The degree of membership of X1 sets a ceiling on the degree of membership of Y

0.25 0.5 0.75 1.00.0

0.25

0.5

0.75

1.0

Membership in set X1

Mem

bers

hip

in s

et Y

Figure 4 A Fuzzy-set Sufficient but not Necessary Condition

X1 is a sufficient, but not necessary, condition for Y to occur

High membership of X1 acts as a floor for high membership of Y

Some other alternatives

• Neural network analysisNeural network analysis

• Data miningData mining

• Bayesian methodsBayesian methods

• Chaos/tipping point theoryChaos/tipping point theory