Using quantitative methods as exploratory techniques in qualitative

research projects.

Richard Bell

University of Melbourne

Disclaimer

• There is nothing new in all this

• analyses carried out with standard statistical software (here, SPSS)

The Scope of this Presentation

• Some preliminary remarks about qualitative & quantitative data analysis

• A few examples

• Some discussion about how to use quantitative tools in qualitative contexts

Some preliminary remarks about qualitative & quantitative data analysis

• The common view of the qualitative/quantitative divide

• Some myths about quantitative data analysis• The nature of data• The purpose of data analysis• A very brief history of quantitative methods

for qualitative data

Quantitative Qualitative Objective Subjective

Tests theory Develops theory focus is concise and narrow focus is complex and broad Reduction, control, precision Discovery, description, understanding, shared

Measurable Interpretive Report statistical analysis. Report rich narrative, individual interpretation.

Basic element of analysis is numbers Basic element of analysis is words/ideas. Researcher is separate Researcher is part of process

Context free Context dependent Hypotheses Research questions

Reasoning is logistic & deductive Reasoning is dialectic & inductive Establishes relationships, causation Describes meaning, discovery

Uses instruments Uses communication and observation Designs: descriptive, correlational, quasi-

experimental, experimental Designs: phenomenological, grounded theory, ethnographic, historical, philosophical, case

study. Sample size: determined by statistical power required

Sample size is not a concern; seeks "information rich" sample

The Qualitative / Quantitative divide

Some myths about Quantitative data analysis

• It is all about NHST (Null Hypothesis Significance Testing)

• It is all about inferential statistics

• It is only a confirmatory procedure

• There is one way (the right way) to do things

• It measures things

The nature of data

• All data can be either quantitative or qualitative

• Saying ‘this piece of data can be assigned to the same class as another piece of data’ allows it to be treated quantitatively

• Numbers can always be treated as qualitative data

The Purpose of Data Analysis

• “Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise”

The Purpose of Data Analysis

• “Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise”

• John Tukey (1962) Annals of Statistics– ‘Data analysis must progress by approximate

answers, at best, since its knowledge of what the problem really is will at best be approximate’

History of quantitative methods for qualitative data

• Louis Guttman (1941) The quantification of a class of attributes

• Cyril Burt (1950) The factorial analysis of qualitative data

• James Lingoes (1968) The multivariate analysis of qualitative data

• Forrest Young (1981) Quantitative analysis of qualitative data

‘Traditional’ Quantitative Methods for Qualitative Data

• Miles & Huberman (1994)– hierarchical cluster analysis

• Giegler & Klein (1994)– correspondence analysis

• Bazely (2002)– cluster analysis– correspondence analysis

Some examples of Quantitative methods for Qualitative data

• Giegler & Klein analysis of personal advertisements

• Demographics from a market research survey

• Miles & Huberman school innovation table

• A current study of social withdrawal in early psychosis

Giegler & Klein

• Examined personal advertisements in a number of German magazines

eg

Young man, 35 y, 176cm, slim with car, good income, looks for a lovely high-bosomed and well-developed partner for a common future.

Dimension 1

2.52.01.51.0.50.0-.5-1.0

Dimension 2

2.0

1.5

1.0

.5

0.0

-.5

-1.0

-1.5

-2.0

MARGIN

Column

Row

Separate

Single

Social

Wowser

HedonistOld

60yo

45yo

30yo

Nationality

TravelFamily

Friendly

Erotic

Values

Image

Figure

Sex

Compassion

Fitness

High SES

H&WMR

H&WMP

H&WMS

H&WFR

H&WFPH&WFS

WNMR

WNMP

WNMS WNFR

WNFP

WNFSEXPMR

EXPMP

EXPMS

EXPFR

EXPFP

EXPFS

WAZMR

WAZMPWAZMS

WAZFR

WAZFP

WAZFS

TIPMR

TIPMPTIPMS

TIPFRTIPFPTIPFS ZMR

ZMP

ZMS

ZFR

ZFPZFS

Correspondence Analysis Representation

Correspondence Analysis

• Finds set(s) of weights for row categories and set(s) of weights for column categories so that the correlation between the sums of the weights is maximized

• Can produce separate maps of relationships between categories of rows or columns

• Can produce a joint map of categories of rows or columns

Generalization

• Two aspects of Correspondence Analysis– weights– correlation

• Generalizes to more complex data structures– weights– correlation models

• multiple regression

• principal components

The model

• Called ‘Alternating Least Squares’

• Procedures devised in the 1970’s – Forrest Young– Yoshio Takane– Jan De Leeuw (‘Albert Gifi’)

• Generally known now as ‘optimal scaling’

Optimal Scaling

• ‘a data analytic technique which assigns numerical values to observation categories in a way which maximizes the relation between the observations and the data analysis model while respecting the measurement character of the data’ (Young, 1981, p.358)

Alternating least squares

Find Optimal Scaling of Categories

Find Relational Coefficients

Back to Geigler & Klein data

• 36 rows of matrix composite rows, eg

• row ZFS– Z indicates magazine (6)– F indicates sex of writer (2)– S indicates ‘image’ (3)

• self

• desired partner

• relationship

Categorization

Magazine Sex Concept Fitness Compassion Figure Values Erotic

Z F Self 44 99 50 11 101

Z F Seeking 41 12 9 11 85

Z F Relationship 6 0 12 5 3

Z M Self 67 97 67 18 207

Z M Seeking 80 9 11 9 37

Z M Relationship 1 0 3 4 1

WN F Self 8 14 17 18 107

WN F Seeking 19 1 4 38 59

WN F Relationship 20 0 0 3 0

WN M Self 9 7 4 3 42

WN M Seeking 11 2 6 3 19

WN M Relationship 1 0 1 0 0

Giegler & Klein data as a four-way table

Multiple Correspondence Analysis

[HOMALS]

Dimension 1

2.01.51.0.50.0-.5-1.0-1.5

Dim

en

sio

n 2

.8

.6

.4

.2

0.0

-.2

-.4

-.6-.8

Category

Concept

Sex

Magazine

Erotic

Values

Figure

Compassion

Fitness.

Relationship

Seeking

Self

M

F WN

Z

Not just tables of frequencies

• Rows are units of interest (documents, cases etc)

• Suppose columns are different variables and contain coding within variables

• eg

Demographic variables in a market research survey

Age

Group

Position in Household

Education Level

Work

Status

Marital Status

Country of

Birth 18-19yrs Male head Primary yr6 Full-time 1st marriage Australia 20-24yrs Female

head Secondary yr8

Part-time 2nd marriage New Zealand

25-29yrs Other female

Secondary yr9

Don't work

living together UK

30-34yrs Other male Secondary yr10

Unemployed

Divorced-Separated

Italy-Malta

35-39yrs

Secondary yr11

Other Widowed Greece-Cyprus

40-44yrs

Secondary yr12

Single Other

Europe 45-49yrs Trade Qual Middle East 50-54yrs Tech-CAE Asia 55-59yrs Uni-I Other 60-64yrs Uni-II Bach deg PG deg

Suppose we wished to form a composite

• Find weights for categories of variables

• to maximize correlations among them

• & find principal component to maximize variance of weighted sum

Transformation: Marital Status

Optimal Scaling Level: Nominal.

Variable Principal Normalization.

Categories

Single

Widowed

Divorced-Separated

living together

2nd marriage

1st marriage

Qu

an

tific

atio

ns

2

1

0

-1

-2

-3

Transformation: Work Status

Optimal Scaling Level: Nominal.

Variable Principal Normalization.

Categories

OtherUnemployedDon't workPart-timeFull-time

Qu

an

tific

atio

ns

2.0

1.5

1.0

.5

0.0

-.5

-1.0

-1.5

-2.0

Transformation: Age Group

Optimal Scaling Level: Ordinal.

Variable Principal Normalization.

Categories

60-64yrs

55-59yrs

50-54yrs

45-49yrs

40-44yrs

35-39yrs

30-34yrs

25-29yrs

20-24yrs

18-19yrs

.

Qu

an

tific

atio

ns

2

1

0

-1

-2

-3

-4

-5

Age Group

Position in House

Educon Level

Work Status

Marital Status

Position in Household -.481

Education Level -.167 .178

Work Status -.004 .163 .234

Marital Status -.515 .469 .297 .158

Country of Birth .119 -.029 -.232 -.086 -.158

Correlations among transformed variables

Dimension 1

Age Group -.736

Position in Household .731

Education Level .529

Work Status .329

Marital Status .807

Country of Birth -.318

Cronbach's Alpha .659

Component Loadingsof Transformed Variables

Object scores dimension 1

3.002.50

2.001.50

1.00.50

0.00-.50

-1.00-1.50

-2.00

Distribution of

Demographic Aggregate120

100

80

60

40

20

0

Std. Dev = 1.00

Mean = -.00

N = 536.00

202330N =

House owned or rented

Rent houseOwn house

Ob

ject

sco

res

dim

en

sio

n 1

4

3

2

1

0

-1

-2

-3

Component Loadings

Variable Principal Normalization.

Dimension 1

1.0.50.0-.5-1.0

Dim

en

sio

n 2

.8

.6

.4

.2

0.0

-.2

-.4

-.6

-.8

Country of Birth

Marital Status

Work Status

Education Level

Position in Househol

Age Group

Summary Tables

• An example from Miles & Huberman

• 12 school sites evaluated on various criteria

• Results summarized in a table

N/AN/AX-OProville

(X)X-OX-ODunHollow

XXXBurton

(X)(X)XXAstoria

(X)(X)XXLido

(X)(X)(X)(X)XXClaston

(X)(X)X(X)XPerryParkdale

N/AXXX(X)XCarson

N/AXXXXTindale

(X)XXXXXBanestown

XN/A(X)XXXXPlummet

XXXXXXXMasepa

Basic constructs, attitudes

Transfer

Self-efficacy

UnderstandingsRelationshipsRepertoireDaily Routines

SITE

No changeNot

ApplicableNot

ApplicableNo changeNo changeNo changechange-

revertProville

Change-oneNo changeNo changeNo changeNo changeChange-revert

change-revert

DunHollow

No changeNo changeNo changeNo changeNo changeChange-several

no change

Burton

No changeNo changeNo changeChange-severalChange-oneChange-several

change-several

Astoria

Change-oneNo changeNo changeChange-oneNo changeChange-several

change-several

Lido

Change-oneNo changeChange-one

Change-oneChange-oneChange-several

change-several

Claston

Change-oneNot Applicable

Change-one

Change-oneChange-several

Change-one

change-several

PerryParkdale

No changeNot Applicable

Change-several

Change-severalChange-several

Change-one

change-several

Carson

No changeChange-several

Change-several

Change-severalNo changeChange-several

change-several

Tindale

No changeChange-one

Change-several

Change-severalChange-several

Change-several

change-several

Banestown

Change-several

Not Applicable

Change-one

Change-severalChange-several

Change-several

change-several

Plummet

Change-several

Change-several

Change-several

Change-severalChange-several

Change-several

change-several

Masepa

Basic constructs, attitudes

TransferSelf-efficacy

UnderstandingsRelationshipsRepertoireDaily Routines

SITE

Possible Research Questions

• Which variables predict the degree of change?

• Find weights for categories that maximize correlations

• find multiple regression coefficients

Source of Innovation

Daily Routines

Repert-oire

Relation-ships

Under-standings

Self-efficacy Transfer

Basic constructs attitudes

Source of Innovation

1.000 .219 .560 .072 .066 .245 -.212 .078

Daily Routines .219 1.000 .606 .529 .875 -.097 .374 .278 Repertoire .560 .606 1.000 .261 .652 .255 .155 .134 Relationships .072 .529 .261 1.000 .597 -.264 .524 .377 Understandings .066 .875 .652 .597 1.000 .062 .551 .139 Self-efficacy .245 -.097 .255 -.264 .062 1.000 -.250 -.204 Transfer -.212 .374 .155 .524 .551 -.250 1.000 .091 Basic constructs, attitudes .078 .278 .134 .377 .139 -.204 .091 1.000

Correlations Transformed Variables

Standardized Coefficients Correlations

Beta Std. Error Zero-Order Part Importance Source of Innovation .367 . .079 .218 .029 Daily Routines -.337 . -.656 -.116 .221 Repertoire -.478 . -.439 -.234 .210 Relationships -.497 . -.822 -.304 .409 Understandings .440 . -.727 .109 -.320 Self-efficacy -.066 . .255 -.049 -.017 Transfer -.452 . -.754 -.313 .340 Basic constructs, attitudes

-.242 . -.525 -.201 .127

A current data set

• PhD project by Simone Pica

• People with psychosis featuring social withdrawal– 19 young people suffering from psychosis with

symptoms of social withdrawal– Unstructured interviews– Standard psychiatric measures also completed

Aim:

• Linking categories evident in interviews (qualitative data) to standard quantitative measures

Raw materialUm, when I got home I thought it was probably a good thing I didn’t

go because um, it sort of relates to motivation as well, I wasn’t really that motivated to go out and deal with people and stuff. If more of my friends were there, I’d probably would have gone, if it was a party and all my friends were there I would have thought cool you know, I’d have to go even if I only had a few dollars, that’s cool, I can go without drinks, cigarettes, I’d just want to be there you know but probably because there would have been only a couple of people I would have known there and the rest of them I wouldn’t have known. I sort of thought no, I wouldn’t have a good time because if I wanted to meet people, I like meeting people, but when I meet people I always have to talk about my psychosis, and whenever I have to talk about my psychosis, its like everyone is listening you know, and they all just stop what they are doing and they listen, “psychosis, what is that?” and then I have to explain everything about it and they are all listening type of thing, honing in type of thing.

Classified material• 3. EXPERIENCED DIFFICULTY COMMUNICATING• He couldn’t talk because he became jumbled, he couldn’t focus

on one thing he kept thinking about whether his ex-friend was going to mention the letter to other people there

• He stayed in small groups of people throughout the evening in order to avoid saying something inappropriate that would draw attention to him

• When he felt comfortable he found it easier to talk• He found that the comfortable feeling didn’t last, it wore off when

the ‘wall’ came and he found it difficult to think of things to talk about

• When he was with the group of people he didn’t know what to talk to people about so he remained silent

• He didn’t know what to talk about because he couldn’t think of anything intelligent to say

• When he was with people and he didn’t know what to talk about his mind was blank, he didn’t think anything

felt differentstressed

uncomfortabledifficulty

communicatingconcern about others

views of them

1 Absent Present Absent Absent

2 Absent Present Present Present

3 Absent Absent Absent Present

4 Present Absent Absent Present

5 Present Present Present Present

6 Present Absent Present Present

Qualitative Data: eg Presence of categories in interview transcripts

DSM-IIIR diagnosis Frequency Percent Cumulative Percent

Schizophrenic 11 55.0 57.9

Schizophreniform 3 15.0 73.7

Schizoaffective 2 10.0 84.2

Delusional 2 10.0 94.7

Bipolar 1 5.0 100.0

Qualitative measures: eg DSM diagnosis

PAS Child PAS Adolesc PAS Adult

14 6 9

21 6 8

35 8 11

46 5 4

54 5 5

64 6 7

74 4 5

Quantitative Measures: eg Premorbid Adjustment Scales

OVERALS

• A tool for relating sets of variables

• Variant that is a common statistical model is canonical variate analysis (producing a canonical correlation between two sets of variables

• OVERALS – Allows for more than two sets– Allows variables to be categorical or ordinal

DIM1

1.0.8.6.4.20.0-.2-.4

DIM

21.0

.8

.6

.4

.2

0.0

-.2

-.4

-.6

-.8

SET

Diagnosis

Interview

SANS

PAS

DSM-IIIR Dimension 2

DSM-IIIR Dimension 1

self boring

want to be alone

shy/inferior

stigma judged reject

.concern others views

.stressed incommunica

Attention

Anhedonia

Avolition

Alogia

Affect Adult

Adolesc

Child

Dimension 1 Transformation Plot for DSM-IIIR

DSM-IIIR

BipolarDelusionalSzoaffectiveSzphreniformSzphrenic

Ca

teg

ory

Qu

an

tific

atio

ns

for

DS

M-I

IIR

1

0

-1

-2

-3

Dimension 2 Transformation Plot for DSM-IIIR

DSM-IIIR

BipolarDelusionalSzoaffectiveSzphreniformSzphrenic

Ca

teg

ory

Qu

an

tific

atio

ns

for

DS

M-I

IIR

2

1

0

-1

-2

-3

Interpreting Results from Quantitative Analyses

• Even hypothesis testing is qualitative (accept/reject)

• Evaluation of model fit (variance accounted for) always subjective

• Most commonly the interpretation of – Factors or components, – discriminant functions, – and canonical variates

• always subjective & qualitative

Multiple Correspondence Analysis

[HOMALS]

Dimension 1

2.01.51.0.50.0-.5-1.0-1.5

Dim

en

sio

n 2

.8

.6

.4

.2

0.0

-.2

-.4

-.6-.8

Category

Concept

Sex

Magazine

Erotic

Values

Figure

Compassion

Fitness.

Relationship

Seeking

Self

M

F WN

Z

Note: Males are seeking erotic good figure in Z

Note: Females focus on Values & Fitness with respect to Self in WN

Relationship & Compassion far

apart

Return to qualitative data

• Examine subsets defined by groupings of variables (eg ads from males seeking relationships emphasizing Figure) for other possible connections

• Examine outliers (those with both compassion and relationship aspects of ads)

Transformation: Age Group

Optimal Scaling Level: Ordinal.

Variable Principal Normalization.

Categories

60-64yrs

55-59yrs

50-54yrs

45-49yrs

40-44yrs

35-39yrs

30-34yrs

25-29yrs

20-24yrs

18-19yrs

.

Qu

an

tific

atio

ns

2

1

0

-1

-2

-3

-4

-5

Changes by age

No changes by age

Return to coded data

• Recode age variable into two groups

• Examine other codings

Conclusions

• Linking qualitative and quantitative analyses is both– simpler– and more flexible

• than most researchers think

Conclusions

• Qualitative researchers should use quantitative tools more

• Quantitative researchers should use qualitative data more