Tim Winchell

Analytical Techniques for Public Service

The Evergreen State College

Winter 2011

Qualitative Data Analysis

“It wasn’t curiosity that killed the cat.

It was trying to make sense of all the data curiosity generated.”

-Halcolm

Qualitative Data

Written text

Conversation, interview, or consultative transcriptions

Focus group transcriptions

Field notes

Diaries

Legal transcripts

Newspaper clippings

Journal or magazine articles

Photographs

Maps

Illustrations

Paintings

Musical scores

Tape recordings

Films (McNabb, p. 368)

“Qualitative Data… have been gathered during the conduct of

interpretive or postpositivist research studies. They exist most often

as some sort of narrative.” Examples include:

Advantages

Grounded in a specific context/situation

Real life events/ settings; lived experience

Deep layers of meaning; rich description filled with differing perspectives, symbolism, metaphor, and meaning.

“Descriptions form the bedrock of all qualitative reporting.” (Patton, p. 438)

“The devil is in the details.”

Difficulties

Labor intensive

Requires creativity

Conceptual sensitivity

Non- formulaic (Polit & Beck, p. 570)

Research Bias

Cost of processing/coding data

Small sample size, many variables

Very limited generalizability

Credibility

One Synopsis of the Challenges

“The challenge of qualitative analyses lies in making sense of massive

amounts of data. This involves reducing the volume of raw information,

sifting trivia from significance, identifying significant patterns, and

constructing a framework for communicating the essence of what the data

reveal…

There are no formulas for determining significance. No ways exist of perfectly

replicating the researcher’s analytical thought processes. No straightforward tests

can be applied for reliability and validity.

In short, no absolute rules exist except perhaps this: Do your very best with your

full intellect to fairly represent the data and communicate what the data reveal

given the purpose of the study.” (Patton, p. 432-433)

Data Management

The complexity of the project drives the level of organization needed

Format field notes consistently

Index notes, so you can find documents easily

Make sure you can read them!

Have a sensible system for cross referencing your notes

Please remember to: Maintain data confidentiality as much as possible

Secure your data when not in use

Maintain participant confidentiality

And… Another Data Back Up Reminder!

“Thomas Carlyle lent the only copy of his handwritten

manuscript on the history of the French Revolution, his master

work, to philosopher J. S. Mill, who lent it to Mrs. Taylor. Mrs.

Taylor’s illiterate housekeeper thought it was waste paper and

burned it. Carlyle behaved with nobility and stoicism, and

immediately set about rewriting the book. It was published in

1837 to critical acclaim and consolidated Carlyle’s reputation as

one of the foremost men of letters of his day. We’ll never know

how the acclaimed version compared with the original or what

else Carlyle might have written in the year lost after the

fireplace calamity.” (Patton, p. 441- emphasis added)

Your General Approach?

Grounded

Start data collection with few preconceived notions about what’s

going on….no pre-formed coding scheme)

….or Framed?

Specific events, behaviors you intend to look for, with coding scheme

already partially developed. Oftentimes use diagrams to explain ideas.

All analyses benefit from diagramming and concept mapping, as Babbie discusses (p. 405).

Qualitative Analysis:

The General Process

Data Reduction

Coding

Data Display

Conclusion Drawing

These are not linear, but concurrent processes

The less “framed” and more grounded the process, the more they are concurrent: constant comparison

Data Reduction

First, we transform data from field notes or transcriptions

Write up and/or transcribe field notes and print.

Which of the data are most useful?

“Developing some manageable classification system or coding scheme is the first step of analysis.

Without classification there is chaos and confusion.

Content analysis, then, involves identifying, coding, categorizing, classifying, and labeling the primary patterns in the data.” (Patton, p. 463)

Consider….

For extensive research projects, summarize interviews with a

brief cover sheet

Who, what, where, when, importance, summary of key contacts

Coding schemes…must match the complexity of the project

Use similar semantics

Identifying concepts, patterns, memos

What is Coding?

In short, codes are shorthand descriptors of:

Setting and context

Subjects’ perspectives, which could include their thinking about people and objects

Processes, activities, and/or strategies

Relationships and social structures

Any preassigned coding schemes (Bogdan & Biklen, 1992, p. 166-172, as quoted in Creswell, p. 193)

Creswell recommends analyzing data using codes readers would expect to learn more about, find surprising, and address larger theoretical issues in the literature. (p. 193)

Variations….

Start categorizing early… Or …..

Dive deeper into the data and avoid making judgments too early… make tentative observations about what might be happening….

To further analyze what is happening:

Write memos to yourself

Use “concept mapping” (Babbie, p. 405)

Build preliminary typologies

Try to use outcome/ process matrices (Patton, p. 468-477)

Open Coding….One Approach

Start with a sample of the data

Read responses carefully…

Keep research questions in mind

Make rough categories of these descriptors that seem to

belong together and code them with a key word.

Utilize constant comparison- similarities and differences.

Work to saturation.

Farm to School Example

Why do local farmers participate in the local farm to school program?

Resp.1: It makes the most business sense to me….

Possible code: ‘business sense,’ busin.

Resp. 2: “It gives me great pride to think of my organic produce being consumed

locally by my family members, friends, and church members and their children.

Possible code: “service,” serve

Farm to School Example

Business Sense (Busin.) Service (Serve)

1. Most business sense

3. Reduces transport costs

3. Ability to hire more

4. Reduces environmental impact- transport

6. Stability of local school district market

1. Belief in organic produce

being consumed locally

1. Organic production for

nuclear family, friends, &

church members & their

children

2. Service to local community

5. Some contribution to local

school district (lower prices

received)

Write ongoing memos and abstracts Write ongoing memos and abstracts

Comprehending:

The Basic Goal of this Stage

Identify important phenomena

Identify broad themes

Document codes that emerge

Begin to speculate about what might be happening

Write ongoing memos and abstracts

Axial Coding

Explore the relationships between and among codes

Look for:

“Contexts

Causal Conditions

Phenomenon central ideas

Strategies for addressing the phenomenon

Intervening conditions

Action/ interactions

Consequences” (Gibbs video)

Develop subcategories, linked by a “paradigm.”

Paradigm includes conditions, actions/ interactions, and consequences (Polit & Beck, p. 584)

Employee Self Care Example How could agencies promote employee self care in their organizations?

Organizational Changes

(OrgCh.)

Employee Changes

(EmpCh.)

Policies

Management Training

Supervisory Best

Practices

Employee Awareness

Health Education Initiatives

Medical Coverage Incentives

Individual Health Surveys/ Contracts/ Teaming

Employee Best Practices

Selective Coding

Identify core phenomenon

Develop story line around the core concept(s)

Compare and contrast the core concept(s) to other selective

coding categories (Gibbs video)

Findings are integrated and refined

Include diagrams (Polit & Beck, p. 584)

Data Display

Playing with typologies and displays is a part of the analysis

process

See Miles and Huberman, Qualitative Data Analysis

Make sense of the data by playing with visual means of

representing the patterns that are emerging from the analysis

Process and outcome flow charts/ matrices

“Interpretation, by definition

involves going beyond the descriptive data. Interpretation means attaching

significance to what was found,

making sense of findings,

offering explanations,

drawing conclusions,

extrapolating lessons,

making inferences,

considering meanings,

and otherwise imposing order on an unruly but surely

patterned world.” (Patton, p. 480)

Theorize: Cause and Effect?

Classic Conditions for Establishing Cause and Effect

Variables Covary

Covariance is not spurious

Logical time order

A lucid explanation is available

Or …clusters of phenomena, identify things that tend often to show up together, even if the causal connection is not clear

Qualitative Analysis- Visually

Analysis of Medical Errors

“Figure 1 classifies the stage in the diagnostic testing process and the transition points within and between stages at which errors can occur, and presents representative occurrences that fall into each of them.” (Harris, et al.)

Early Introduction of Soft Foods by

Young Mothers

Verification

Triangulate from multiple sources or methods

Use several researchers as a reliability check.

Use rich, thick description in order to provide for the shared experience

Clarify research bias up front

Look for disconfirming evidence

Spend prolonged time in the field to develop an in-depth understanding

Use peer debriefing

Use an external auditor to review findings (Creswell, p. 196)

Complete several case studies. (Yin, 2003)

Review finding with participants.

If it’s just you, double or triple check your data and conclusions

Standards

Be true to the data

Don’t get too carried away by particularly eloquent,

memorable, or “simple” respondents—this creates a cognitive

bias

Always check and recheck both the data and conclusions you

draw from it

Qualitative Validity

Traditional Criteria for Judging

Quantitative Research

Alternative Criteria for Judging

Qualitative Research

Internal validity

External validity

Reliability

Objectivity

Credibility

Transferability

Dependability

Confirmability (Trochim, 2006)

Drawing Conclusions

Summary of data and results of coding analysis

Patterns and themes

Clusters of similar findings?

Case comparisons

Powerful metaphors

Any data for which your theory can’t provide a reasonable explanation?

Final Thoughts

Data Management and Analysis work hand in hand

Coding is technical work, which is improved upon with advanced practice, study, and interpretation

Remember to consult additional resource materials

(Some are listed at the end of the PowerPoint)

Utilize the Internet judiciously

Qualitative data software resources are reviewed in many publications and on-line

Workshop Case:

TESC Alumni Relations

Research Interest

Why do colleges and universities have alumni programs?

Research questions

What are TESC graduates’ perceptions of TESC’s alumni programs?

What kind of alumni program do they want?

How do they recall their experience as TESC students?

What connects them to the College?

What nourishes that connection?

What can AR do to improve those connections?

Workshop Methods/ Results

Overview

Draft questions; approval from Alumni Relations

Zoomerang online survey

1647 responses

One researcher

Pluses: clear conclusions, grounded in data

Minus: not validated by second researcher

Workshop Exercise

Code 2 or 3 pages of the data from the responses to the Alumni survey question.

“What was the best part of your experience at Evergreen?”

Code individual responses

What are the most common codes?

What do these data tell you/us about these alumni ? About Evergreen?

Resources

YouTube Search “qualitative research coding”

Graham R. Gibbs Qualitative Research Coding Series

Open Coding:

http://www.youtube.com/watch?v=gn7Pr8M_Gu8

http://www.youtube.com/watch?v=vi5B7Zo0_OE&fe

ature=related

http://www.youtube.com/watch?v=n-

EomYWkxcA&feature=related

http://www.youtube.com/watch?v=AwmDRh5l7ZE&

feature=related

Resources YouTube Search “qualitative research coding”

Graham R. Gibbs Qualitative Research Coding Series

Axial Coding:

http://www.youtube.com/watch?v=s65aH6So_zY&feature=related

Selective Coding:

http://www.youtube.com/watch?v=w9BMjO7WzmM&feature=related

Grounded Theory:

http://www.youtube.com/watch?v=4SZDTp3_New&feature=related

http://www.youtube.com/watch?v=dbntk_xeLHA&feature=related

Morgan, D. L. (1997). Focus Groups as Qualitative Research (2nd Ed.). Sage Publications: Thousand Oaks, CA.

Software Resources

Computer Programs:

See Babbie, p. 406-416

Data analysis strategies for qualitative research- Research

Corner

http://findarticles.com/p/articles/mi_m0FSL/is_6_74/ai

_81218986/?tag=content;col1

Software for qualitative research

http://homepages.vub.ac.be/~ncarpent/soft/soft_softsites

.html

Software for qualitative research

http://www.audiencedialogue.net/soft-qual.html

References

Babbie, E. (2010). The Practice of Social Research (12th Ed.). Wadsworth Publishing: Belmont, CA.

Creswell, J. W. (2003). Research Design: Qualitative, Quantitative, and Mixed Methods Approached (2nd Ed.). Sage Publications: Thousand Oaks, CA.

Harris, et al. Mixed Methods Analysis of Medical Error Event Reports: A Report from the ASIPS Collaborative

http://www.ncbi.nlm.nih.gov/bookshelf/br.fcgi?book=aps2&part=A2024

McNabb, D. E. (2002) Research Methods in Public Administration and Nonprofit Management: Quantitative and Qualitative Approaches. M.E. Sharpe: Armonk, NY.

References II

Miles, M. B., & A.M. Huberman. (1994). Qualitative Data Analysis. (2nd Ed.). Sage Publications: Thousand Oaks, CA.

Patton, M. Q. (2002). Qualitative Research & Evaluation Methods (3rd Ed.). Sage Publications: Thousand Oaks, CA.

Polit, D. F., & Beck, C. T. (2004). Nursing Research: Principles and Methods (7th Ed.). Lippincott Williams & Wilkins: New York, NY.

Trochim, William M. K. (2006). Research Methods Knowledge Base. http://www.socialresearchmethods.net/kb/qualapp.php

Yin, R. K. (2003) Case Study Research (3rd Ed.). Sage Publications: Thousand Oaks, CA.

Acknowledgements

Making Sense of Qualitative Data

TESC MPA Program ATPS Winter 2010 Geri/Gould/McBride