by John W. Creswell, Ph.D. Department of Educational Psychology,

University of Nebraska-Lincoln, and Founding Co-Editor, Journal of Mixed Methods Research

1© Please do not duplicate or use these slides without the express permission of the author.

University of Pretoria, October 21, 2008University of Pretoria, October 21, 2008

Topics

IntroductionsPositioning myselfA definition of mixed methods researchSteps in the process of designing a mixed

methods study

2

Positioning myself

View research as set of interactive components; not always linear

Focus on rigorous data collection and analysisWork as an applied research methodogistTrained in quantitative, self-trained in

qualitative, lst generation mixed methods writerServe as a consultant on mixed methods on

projectsWork on projects in an order not in proposal

format; then I reassemble into proper format

A few more thoughts about myself…

Published by Sage Publications, Pearson Education (Merrill Educ.)

A Definition of Mixed Methods Research

How would you combine two types of data?

Qualitative Text Data

This is a sample of a text file of words that might be collected on interview transcripts, observation fieldnotes, or optically-scanned documents.

Quantitative Numeric Data

2 3 4 2 5 2 3 1 1 2 3 4 2 3 3 2 1 1 1 5 3 4 1 2 3 1 4 4 5 5 4 1 2 1 4 3 3 5 1 4 2 3 1 5 5 2 2 1 5 3 5 1 3 1 5 3 2 2 5 1 3 2 4 4 3 1 2 4 2 2 4 1 5 5 4 2 1 5

7

Quantitative Data Qualitative Data

MethodologyMethod

ParadigmPerspective Use of mixed methods

in other designs

Mixed Methods

A mixed methods researcher…

Collects both quantitative and qualitative data“Mixes” them“Mixes” them at the same time (concurrently) or one

after the other (sequentially)Emphasizes both equally or unequally

9

A definition

• Mixed methods research is both a method and methodology for conducting research that involves collecting, analyzing, and integrating quantitative and qualitative research in a single study or a longitudinal program of inquiry. • The purpose of this form of research is that both qualitative and quantitative research, in combination, provide a better understanding of a research problem or issue than either research approach alone.

10

Collecting both quantitative and qualitative data Quantitative data

InstrumentsChecklistsRecords

Qualitative dataInterviewsObservationsDocumentsAudio-visual materials

11

Quantitative and qualitative data analysis

Qualitative analysisUse text and

images,For codingFor theme

developmentFor relating themes

Quantitative analysisUse statistical

analysis,For descriptionFor comparing

groupsFor relating

variables

12

Mixing or linking the data

Results

Connect data:

Results

Converge data:

Embed the data:

Quan data

Qual data

Qual Quan

Qual Quan

Typical situations in which mixed methods is used…

To compare results from quantitative and qualitative research

To use qualitative research to help explain quantitative findings

To explore using qualitative research and then to generalize findings to a large population using quantitative research

To develop an instrument because none are available or useful

To augment an experiment with qualitative data

What is the reason for using mixed methods?

The insufficient argument – either quantitative or qualitative may be insufficient by itself

Multiple angles argument – quantitative and qualitative approaches provide different “pictures”

The more-evidence-the-better argument – combined quantitative and qualitative provides more evidence

Community of practice argument – mixed methods may be the preferred approach within a scholarly community

Eager-to-learn argument – it is the latest methodology“Its intuitive” argument – it mirrors “real life”

Designing a Mixed Methods Study

Preliminary considerationsCreating a titlePosing a general questionListing the types of data collection and analysisMaking explicit your worldviewIdentifying your research designDrawing a figure of your designWriting a purpose statementWriting research questionsCompleting a research plan

16

• Research problem• Content – any topics• Fit the problem to mixed methods (arguments)

•Access to both qualitative and quantitative data

•Background and resources

•Receptive audience

Create a working title

Writing the titleShortTopicParticipantsInclude the words “Mixed methods”Neutral –neither quan or qual

17

Pose the general question to be answered

Write it as a questionLook to see how it is phrasedMake sure that it is specific enough and

focused (an answerable question)Ask yourself, “when I end the study, what

question would like to have answered?”

18

List your types of data collection (a review)

Quantitative data (closed-ended)InstrumentsBehavioral

checklistsRecords

Qualitative data (open-ended)InterviewsObservationsDocumentsAudio-visual materials

19

Activity – List your sources of data

Quantitative Sources of Data

Qualitative Sources of Data

20

List your approach to data analysis (a review)

Qualitative analysisUse text and

images,For codingFor theme

developmentFor relating themesDesign-type

Quantitative analysisUse statistical

analysis,For descriptionFor comparing

groupsFor relating

variablesDesign-type

21

22

PostpositivismDeterminationReductionismEmpirical observation and measurementTheory verification

ConstructivismUnderstandingMultiple participant meaningsSocial and historical constructionTheory generation

Advocacy/ParticipatoryPoliticalEmpowerment issue-orientedCollaborativeChange-oriented

PragmatismConsequences of actionsProblem-centeredPluralisticReal-world practice oriented

Determine your worldview…

Several stances on philosophy in mixed methods…

One paradigm (pragmatism, transformative) (Tashakkori & Teddlie, 2003; Mertens, 2003)

Multiple paradigms (dialectic perspective) (Greene, 2007)

Linking paradigms to design features) (Creswell & Plano Clark, 2007)

Epistemological stance (ontology, epistemology, axiology, methodology) (Guba & Lincoln, 2005)

Shared beliefs in a research field (Morgan, 2007)

What it isHow it informs your study

23

Make explicit your interpretive lens (theory)

Advocacy lens (feminist, racial, ethnic, disability, sexual orientation) (Mertens, 2003)

Social science lens (social science theory)

Components:What is itWho has used it in your fieldHow it will shape your study (rephrase your

guiding research question, if a lens applies)

24

State your procedures (methods)

Procedures for handling your qualitative and quantitative data

Sequence – concurrent or sequential or bothEmphasis – emphasis on qualitative or

quantitativeSometimes both concurrent and sequential

phases are usedDesigns may include more than two phasesThink about using a simple, elegant design

25

26

QUANData & Results

QUANData & Results InterpretationInterpretation

QUALData & Results

QUALData & Results

QUANPre-test Data & Results

QUANPre-test Data & Results

QUANPost-test

Data & Results

QUANPost-test

Data & Results

Intervention

qualProcess

qualProcess

InterpretationInterpretation

Triangulation Design

Embedded Design

Concurrent Mixed Methods Designs

Parsimonious designs (Creswell & Plano Clark, 2007)

27

Sequential designs

QUANData & Results

QUANData & Results InterpretationInterpretation

qualData & Results

qualData & Results

Following up

QUALData & Results

QUALData & Results

quanData & Results

quanData & Results InterpretationInterpretation

Building to

Before-intervention

qual

Before-intervention

qual

QUANIntervention

Trial

QUANIntervention

Trial

After-

intervention qual

After-

intervention qual InterpretationInterpretation

Exploratory Design

Explanatory Design

Sequential Embedded Design

28

Design Name Equal priority QUAN emphasis QUAL emphasis

Concurrent, triangulation QUAL+QUAN QUAN+qual QUAL+quan

Concurrent, embedded n/a QUAN(qual) QUAL(quan)

Explanatory, sequential, quan first QUANQUAL QUANqual quanQUAL

Exploratory sequential, qual first QUALQUAN qualQUAN QUALquan

Sequential, embedded n/a (qual) QUAN

QUAN (qual)

(quan) QUAL

QUAL (quan)

Design options

Explanatory sequential design

Quantitative Data*

Number of cigarettes CES-D6

Qualitative Data*

Semi-structured interviews, audio recorded and transcribed

* Data collected 10 times over the course of a

calendar year for 40 participants

Quantitative Analysis

Graphic plot of CES-D6 scores over time for each participant Graphic plot of cigarettes/day values over time for each participant

Case Selection

Qualitative Analysis

Description of each case

Identification of life events occurring during critical months where

smoking increased or decreased

Thematic analysis of life events for each

case Cross-case thematic

analysis

Interpretation

Why did changes in smoking occur?

Quant

itative Data Collection

(quan)

Qualitative Data Collection

Interpretation –based on quanad QUAL results

Qualitative Data Analysis(QUAL)

Case Selection Quantitative

Data Analysis

Selected 5 casesmaximally varyingIdentified criticalmonths in which smoking varied

Source: Creswell, Plano Clark, Shope, McVea. (in progress)

+

Phase I Qualitative Research - Year 1

Qualitative Data Collection Unstructured Interviews -50 participants 8 observations at the site16 documents

Qualitative Data Analysis Text Analysis: Using QSR N6

Qualitative FindingsDevelopment of codes and themesfor each site

Phase II Quantitative Research - Year 2

Quantitative Instrument Development

Quantitative Test of the Instrument

Create approximately a 80-iteminstrument plus demographics

Administer survey to 500 individuals

Determine factor structure of items and conduct reliability analysis for scales

Quantitative ResultsDetermine how groups differusing ANOVA test

Exploratory sequential design

32

Process – collectionand analysis of qualitative

data

(before, during, after trial)

Embedded research design

Experiment

InterventionQUANData collectionPre-test

QUANData collectionPost-test

Mixing the quan and qual data

Type of Mixing

Type of Design

Why Mixing Occurs

Where Mixing Occurs in Research Process

Connecting Sequential One phase builds on the other

Between data analysis (Phase 1) and data collection (Phase 2)

Merging Concurrent Bring results together

After analysis of both quan and qual – typically in discussion

Embedding Sequential or Concurrent

Either building or bringing results together

Either between phases or in discussion after analysis

33

Methodological issues

Concurrent DesignsUse strategies to

explore contradictory findings

Use parallel questionsSelect sub-sample of

quantitative for qualitative

Be sensitive to bias from one data collection to the other

Sequential designsIn Explanatory Design,

select qual sub-sample from quan sample

In Explanatory Design, consider alternatives for followup qual sampling

In Exploratory Design, samples can differ

In Exploratory Instrument Design, consider qual data analysis approaches for developing instrument

34

35

This mixed methods study will address _________________ (overall content-aim of the study). A triangulation mixed methods design will be used, and it is a type of design in which different but complementary data will be collected on the same topic. In this study, _________________ (quantitative instruments) will be used to test the theory of _____________ (the theory) that predicts that __________________ (independent variables) will influence ________________ (positively, negatively) the __________ (dependent variables or outcomes) for ____________ (participants) at __________ (the research site). Concurrent with this data collection, qualitative _______________ (type of qualitative data, such as interviews) will explore ____________ (the central phenomenon) for _________________ (participants) at _____________(site). The reason for collecting both quantitative and qualitative data are to bring together the strengths of both forms of research to ___________________ (e.g., compare results, validate results, corroborate results).

This mixed methods study will address _________________ (overall content-aim of the study). A triangulation mixed methods design will be used, and it is a type of design in which different but complementary data will be collected on the same topic. In this study, _________________ (quantitative instruments) will be used to test the theory of _____________ (the theory) that predicts that __________________ (independent variables) will influence ________________ (positively, negatively) the __________ (dependent variables or outcomes) for ____________ (participants) at __________ (the research site). Concurrent with this data collection, qualitative _______________ (type of qualitative data, such as interviews) will explore ____________ (the central phenomenon) for _________________ (participants) at _____________(site). The reason for collecting both quantitative and qualitative data are to bring together the strengths of both forms of research to ___________________ (e.g., compare results, validate results, corroborate results).

Write a purpose statement for a triangulation design

Write a purpose statement fora embedded design

This mixed methods study will address _________________ (overall content aim of the study). An embedded mixed method design will be used, and it is a design in which one data set provides a supportive, secondary role in a study based primarily on the other data set. The primary purpose of this study will use ________________ (quantitative instruments) to test the theory of _____________ (the theory) that predicts that __________________ (independent variables) will influence ________________ (positively, negatively) the __________ (dependent variables or outcomes) for ____________ (participants) at __________ (the research site). A secondary purpose will be to gather qualitative data _______________ (type of qualitative data, such as interviews) that will explore ____________ (the central phenomenon) for _________________ (participants) at _____________(site). The reason for collecting the secondary database is ________________ (e.g., to address different question, to provide support for the primary purpose).

36

37

Write a purpose statement for an explanatory design

This study will address _______________ (content-aim of the study). An explanatory mixed methods design will be used, and it will involve collecting qualitative data after a quantitative phase in order to explain or follow up on the quantitative data in more depth. In the first quantitative phase of the study, ______________ instrument data to be collected from _______________ (participants) at ___________ (research site) to test _______ (the theory) that explains why ______________ independent variables) relate to the ____________ (dependent variables). The second qualitative phase will be conducted because ______________ (intent of the qualitative phase). In this exploratory follow-up, the ______________ (central phenomenon) will be tentatively explored with ___________ (participants) at _____________ (the research site). The reason for the exploratory follow-up is to _______________ (e.g., to help explain or build upon initial quantitative results).

38

This study addresses _________________ (content-area of the study). The purpose of this exploratory sequential design will be to __________________ (e.g., develop an test an instrument, generate a taxonomy). The first phase of the study will be a qualitative exploration of ________________ (the central phenomenon) by collecting ___________________ (types of data) from _________________ (participants) at _____________ (the research site). The second quantitative phase will follow up on the qualitative phase for the purpose of __________(intent of this followup). In the quantitative phase, ___________ (instrument data) will be collected from ____________ (participants) at ______________ (research site). Quantitative research questions/hypotheses will be formulated after the completion of the initial qualitative phase. The reason for collecting qualitative data initially is that _____________________ (e.g., instruments are not available, variables are not known, there is little guiding theory).

Write a purpose statement for an exploratory design

Qualitative research questions

Qualitative central questionBegin with “what” or “how”Focus on single phenomenonUse exploratory verbs (discover, understand,

explore)Non-directional languageA general question (allowing participants’

perspectives to emerge)

39

Quantitative research questions

Can be hypotheses or questionsState variables – independent, dependent,

mediating, covariatesDevelop from theory Use distinct measures for independent and

dependent variables Order variables from independent to

dependent

40

Writing research questions/hypotheses in mixed methods research

Write qualitative research questions and write quantitative research questions/hypotheses

Also write a mixed methods research question

Write these questions separately

41

A new type of research question: A mixed methods question

Three ways to write this question:

Methodologically-focused:

To what extent do the qualitative results confirm the quantitative results?

Content-focused: How do the interviews with adolescent boys support the quantitative

results that their self-esteem changes during the middle school years?

Hybrid of quantitative and qualitative elements:

What results emerge from comparing the exploratory qualitative data about boy’s self-esteem with outcome quantitative instrument data measured on a self-esteem instrument?

Order the topics for your plan

Title Introduction

(Problem) Worldview/theoretical lens Audience Purpose Statement Research Questions

(Literature Review) Methods

Type of Mixed Methods Design (also add definition of mixed methods) Types of Data Collection Types of Data Analysis Sequence/Emphasis/ Mixing Procedures Figure of procedures Anticipated methodological issues

Ethical issues anticipated Validity issues Researcher resources and skills References, Appendices

43

Additional resources Books:

Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.

Creswell, J. W. (2008). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage Publications.

Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco: Jossey-Bass.

Mertens, D. M. (2005). Research methods in education and psychology: Integrating diversity with quantitative and qualitative approaches (2nd ed.). Thousand Oaks, CA: Sage Publications.

Plano Clark, V. L., & Creswell, J. W. (2008). The mixed methods reader. Thousand Oaks, CA: Sage.

Tashakkori, A. & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage Publications.

Tashakkori, A. & Teddlie, C. (Eds.) (2003). Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage Publications.

44

Additional resourcesArticles and Chapters:

Caracelli, V. J., & Greene, J. C. (1993). Data analysis strategies for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 15 (2), 195-207.

Creswell, J. W., Plano Clark, V. L., Gutmann, M., & Hanson, W. (2003). Advanced mixed methods research designs. In: A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 209-240). Thousand Oaks, CA: Sage.

Creswell, J. W., Plano Clark, V. L., & Garrett, A. L. (2008). Methodological issues in conducting mixed methods research. In M.M. Bergman (Ed.), Advances in mixed methods research. London: Sage.

Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11 (3), 255-274.

Ivankova, N. V., Creswell, J. W., & Stick, S. (2006). Using mixed methods sequential explanatory design: From theory to practice. Field Methods, 18(1), 3-20.

Morgan, D. L. (2007). Paradigms lost and pragmatism regained. Journal of Mixed Methods Research, 1, 48-76.

Morse, J. M. (1991). Approaches to qualitative-quantitative methodological triangulation. Nursing Research, 40, 120-123.

45

by John W. Creswell, Ph.D. Department of Educational Psychology,

University of Nebraska-Lincoln, and Founding Co-Editor, Journal of Mixed Methods Research

46© Please do not duplicate or use these slides without the express permission of the author.

University of Pretoria, October 21, 2008University of Pretoria, October 21, 2008