Why do we need mixed-methods? Should we

differentiate integrationversus mixed-methods?

Alan Bryman,

Management Centre,

University of Leicester, UK.

The current situation

• Exciting times

• ‘Paradigm wars’

• Epistemological arguments

• Pragmatism

• Washing machines and questionnaires!

• Not addressing epistemological issues today

My Research on Multi-Strategy Research

• Funded by Economic & Social Research Council – Research Methods Programme

• 2 main strands today:1. Traditional content analysis of journal articles

using mixed-methods research

2. Interviews with social researchers who’ve used it

Research methods

• Wanted to map general characteristics of mixed-methods research – content analysis

• Wanted perspectives of mixed-methods researchers + contingencies involved – semi-structured interviews

Content analysis

• Searched Social Sciences Citation Index for: quantitative and qualitative; triangulation; multi(-)method; mixed method in titles and abstracts of English language articles

• 5 disciplines: sociology; social psychology; organizational behaviour; human and cultural geography; media and cultural studies

• 1994-2003 232 articles

Content analysis

• Articles foregrounded mixed-methods

• Emphasis on mixed-methods in terms of data collection and analysis

• Major focus on rationales for mixed-methods research

Classifying Forms of Mixed-Methods Research

• Distinction between rationale and practice

• Rationale = stated purpose(s) of integrating quantitative and qualitative research

• Practice = actual use(s) made of integrating quantitative and qualitative research

• Used both Greene et al. scheme and my own grounded scheme

Classifying Forms of Mixed-Methods Research

• Greene et al. (triangulation; complementarity; development; initiation; expansion)

• Parsimonious but only 2 rationales coded (primary & secondary)

• Developed alternative scheme

Alternative Scheme

• Triangulation

• Offset

• Completeness

• Process

• Different research questions

• Explanation

• Unexpected results

• Instrument development

• Sampling

• Credibility

• Context

• Illustration

• Utility

• Confirm & discover

• Diversity of views

• Enhancement

• Other/unclear/not stated

Highlights of Findings: Rationale

• No rationale in 27% of all articles

• Main categories in terms of rationale: Enhancement 32% Completeness 13% Sampling 13% Triangulation 13%

Highlights of Findings: Practice

• Main categories in terms of practice: Enhancement 52% (rationale 32%) Triangulation 35% (rationale 13%) Completeness 29% (rationale 13%) Illustration 23% (rationale 2%) Sampling 19% (rationale 13%)

Highlights of Findings: Rationale and Practice

• Rationale and practice not always in lineRationale often not reflected in how multi-

strategy research actually usedPractice often doesn’t chime with rationales

given

• Examples from contingency table analysis

Highlights of Findings: Rationale and Practice –

The Case of Triangulation• Of the 29 articles citing triangulation as a

rationale, 19 used it that way, i.e. one-third of articles citing triangulation as rationale didn’t use multi-strategy research that way or didn’t report doing so. Other prominent uses of articles citing triangulation were: enhance (13); completeness (10); and illustration (8).

Highlights of Findings: Rationale and Practice –

The Case of Triangulation• Other way around

• 80 articles used a triangulation approach but only 19 of them gave it as a rationale, i.e. three-quarters of articles using triangulation didn’t cite it as a rationale

• Suggests triangulation hard to resist when opportunity arises

Highlights of Findings: Rationale and Practice –

The Case of Completeness

• Completeness was a rationale for 31 articles and 84% of them used it that way

• But when practice is examined, 61% of all articles using a completeness approach didn’t specify it as a rationale

Highlights of Findings: Rationale and Practice –

The Case of Enhancement

• 73 articles specified enhance as rationale, a quarter of them didn’t use multi-strategy research this way

• 121 articles used multi-strategy research this way, but over half of them hadn’t specified it as a rationale

• Several other examples of mismatches

Highlights of Findings: Rationale and Practice

• Often mismatch between rationale and practice

• Mixed-methods research a moveable feast

Minority Strategy:The Gatling Gun Approach

Gatling Gun Strategy

4 or more rationales:

• 6 articles in terms of rationale

• 33 articles in terms of practice

Themes from Semi-Structured Interviews

• Similar to content analysis• Mixed-methods research increasingly expected

Concern for many Research questions important Particularistic versus universalistic discourses Not due to confusion – lack of guidelines about mixed-

methods issues; textbook account too simple; ambivalence about role of research questions; lack of prescription

Uncertainty

What do we mean by mixing, etc.?

• Mixing vs. integration

• Use of verbs

• What does bringing together of quantitative and qualitative research entail?

• Mixed-methods research or multi-methodology/multi-method research?

Is Integration Occurring?Content analysis findings

• Content analysis findings

• Genuine integration – 18% of articles

• Parallel presentation – 47% of articles

• Looked for evidence of findings being brought together comprehensive picture interweaving both

Is Integration Occurring?Semi-structured interviews

Most expressed concern. Main themes:• Different audiences• Greater faith in one; also familiarity• Design issues• Time-lines differ• Skill specialisms• One more striking or interesting• Objectivist vs. constructionist accounts• Journal publication issues

Bryman Goes Reflexive

• Mine was a mixed-methods project

• Justified using both content analysis and qualitative interviewing quite well

• Outcomes consistent with rationales

• Mixed-methods research linked to my research questions

• Integration of data not adequately achieved

Back to the Title

Why do we need mixed-methods?

• We often don’t need it

• But difficult to decide when we do

• Good deal of uncertainty about when to use a mixed-methods approach

Should we differentiate integration vs. mixed-methods?

• Don’t get preoccupied with the verbs

• Interviewees saw problem

• Mixing findings like a conversation

• Forging an overall account

• Lack of templates

Should we differentiate integration vs. mixed-methods?

• Tended to emphasize doing mixed-methods research

• Need more attention to representation of mixed-methods findings in publications

• Quality criteria