Analyzing & evaluating Analyzing & evaluating qualitative dataqualitative data

Kim McDonoughNorthern Arizona University

Epistemology in qualitative Epistemology in qualitative researchresearchTheory of knowledge

◦Constructionist—Meaning/knowledge vary across people and over time

◦Holistic—complex topic is viewed from multiple perspectives

◦Subjectivist–Knowledge/meaning is personal

◦Context-based—Knowledge is negotiated socially and historically

Search for knowledge◦Truth is relative & context specific◦Goal is interpret how people

construct meaning◦Start with experiences and build

theory inductively ◦Data provides information that must

be interpreted◦Interpretation is shaped by the

researcher’s own experiences

““Good” qualitative Good” qualitative researchresearchMaintains a single focus/idea/problemUses rigorous data collection

◦Sufficient time in the field◦Many sources from multiple perspectives◦Detailed summary of each source

Employs rigorous data analysis◦Multiple levels of abstraction◦Verification of the accuracy of the findings

Engaging report◦Clear, detailed writing◦Accurately reflexes complexity of the context

Appropriate topics for Appropriate topics for qualitative researchqualitative researchThe research question asks how or

what◦Focus on understanding

The topic needs to be explored◦Theory doesn’t exist yet, is

insufficient, or hasn’t been tested in a particular context

◦The small pieces that make up the big picture haven’t been identified yet

◦Time & resources are available ◦A receptive audience exists

The natural setting is the focus of inquiry◦Obtain information from participants

in their natural environment◦Reveal contextualized

patterns/truthsThe researcher’s interests

◦Take a personal role in the project◦Serve as a voice for the participants

Basic tenets of qualitative Basic tenets of qualitative analysisanalysisThe search for patterns in data

and ideas that help explain the existence of those patterns

The goal◦Reduce huge amounts of text to

manageable units for further analysis◦Interpret the contribution of those

manageable units to existing knowledge or practice

Steps in data analysisSteps in data analysisPreliminary steps: Organize the data

◦Organize the data◦Label/identify source of the data◦Convert to appropriate text units if

necessary◦Enter any numeric information into

spreadsheet◦Determine which data sources are

available for each participant◦Make decisions about inclusion or

exclusion criteria based on completeness of data

Form general impressions◦Read all the data multiple times

Do not analyze each data source separately

Do not prioritize one source of data over another

◦Get a sense of the whole before trying to break it into parts

◦Make notes—short phrases, ideas, or key concepts

Analysis: Form initial categories◦Search for categories, themes, or

dimensions◦Identify & name the major themes◦Describe each of the themes◦Check descriptions to refine

overlapping categories

Classify data segments by themes◦Read the entire data set again◦Identify segments in all sources that

belong to each theme◦Keep track of segments that don’t fit

with a theme

Evaluation: Assess themes & segment assignment◦Evaluate whether the themes are

appropriate in light of all the segments

◦Decide if all the segments fit with an existing theme

◦Rename/combine/separate themes if necessary

◦Create new themes if necessary

Interpretation◦Make sense of the patterns in the data◦Step back from the summarizing the

data and find links to larger meaning Based on insights/intuition/hunches Based on an existing construct, idea,

theory, practice Based on similarity/divergence from

previous research findings

Verifying accuracy in Verifying accuracy in qualitative data analysisqualitative data analysisProlonged engagement/persistent

observation/time on site◦Building trust with participants, learning

the culture, checking misinformation and distortions, time on site

Triangulation/Diversity of method◦Use multiple sources, methods,

investigators◦Elicit multiple perspectives◦Identify corroborating evidence from

multiple sources

Clarifying researcher bias◦State past experiences, biases,

prejudices & orientations that may have shaped the inquiry & interpretation

Peer review/debriefing◦An external check of the

research process◦“The devil’s advocate”

Member checks◦Solicit the informants’ views about the

credibility of the findingsExternal audits

◦Someone with no connection to the study

◦Allow an external auditor to examine the process and product of analysis

◦Determines whether the findings, interpretations, conclusion are supported by the data

Rich, thick description◦Describe in detail and participants

and the setting◦Allows readers to make decisions

about transferabilityNegative case analysis

◦Refines working hypotheses as data analysis/interpretation unfolds

◦Revise hypotheses until all cases fit◦Account for outliers and exceptions

Deciding which techniques to use◦Use at least two (Creswell)◦Easiest, cost-efficient, most popular

procedures Triangulation Rich, thick description Member check

DiscussionDiscussionProcedures you have used for

verifying evaluation◦Successful ones?◦Challenges or concerns?

Current qualitative projects?Suggestions for data sources &

perspectives?