Upload
james-mullooly
View
6.511
Download
2
Embed Size (px)
DESCRIPTION
Using Qualitative Data Analysis Software Michelle C. Bligh, Ph.D. Claremont Graduate University March 18, 2005
Citation preview
Using Qualitative Data Analysis Software
Michelle C. Bligh, Ph.D.
Claremont Graduate University
March 18, 2005
Why Qualitative Assessment?
“Study the box.”
What is Qualitative Research?
"Qualitative inquiry is an umbrella term for various philosophical orientations to interpretive research.” - Glesne and Peshkin (1992)
"Qualitative research is a loosely defined category of research designs or models, all of which elicit verbal, visual, tactile, olfactory, and gustatory data in the form of descriptive narratives like field notes, recordings, or other transcriptions from audio- and videotapes and other written records and pictures or films.” -Preissle
Advantages of Qualitative Research
Greater depth and detailRichness and holismFlexibility/lack of constraintsFocus on naturally occurring, ordinary events in
their natural settingsData are collected in close proximity to the situation Influences of context are not stripped awayAllow emphasis on processes, of how and why
rather than just what
Advantages of Qualitative Research (continued)
UndeniabilityLead to new integrations/interpretationsCan avoid pre-judgments/halo effectsConsistencySupplement, validate, explain, illuminate,
or reinterpret quantitative data
Disadvantages of Qualitative Research
Extremely time-consuming/labor intensiveData overloadSubjectivity/researcher biasReactivityDependent on researcher’s attributes/skillsPsychologically draining
Sources of Data
Open-ended questionsLogs, journals, or diariesObservationsStoriesCase studies Individual ‘interviews’/Oral examsDiscussion groups/Focus groupsEtc.
Your Approach Depends On…
1. The focus of your study and the themes you want to address
2. The needs of those who will use the information
3. Your resources (time, energy, money, software available)
Qualitative Analysis (Miles & Huberman)
Data reduction– Selecting, focusing, simplifying
Data display– Creating organized, compressed
representations of informationConclusion Drawing and Verification
– Deciding what things mean and testing them for plausibility/validity
Coding
Coding is analysis Codes are tags or labels for assigning units of
meaning to the descriptive or inferential information compiled
It is the meaning that matters Codes are used to retrieve and organize the
chunks of information, so you can quickly find, pull out, and cluster the segments relating to a particular topic
Types of Codes
Descriptive: attributing a class of phenomena to a segment of text (e.g., spelling)
Interpretive: include a more complex, underlying meaning (e.g., unsupported argument)
Pattern: inferential and explanatory; group codes into a smaller number of themes or constructs; analogous to cluster and factor analysis in statistics (e.g., thoroughness)
The process of coding
Create a provisional “start list”– Usually anywhere from 12 – 60 – Get them on a single page for reference– Make sure they are organized/structured
Create code definitions Revise coding scheme
– Filling in: adding, reconstructing preexisting codes– Extension: recoding with a new theme or insight– Bridging: seeing new relationships– Surfacing: identifying new categories
The process of coding (cont.)
Structure is key: codes should relate to one another, they should be part of a governing structure
Structure includes larger, more conceptually inclusive codes, and smaller, more differentiated codes
Pattern codes should represent a web of meaning that is grounded in the data
Uses of Qualitative Software
Data reduction– Retrieving text that has pre-determined
significance
Text exploration– Helping researcher recognize underlying
themes of the text
Advantages of CAQDAS
Makes the sheer volume of data more manageable
Helps to selectively retrieve information– Can summarize results in structured lists and tables
Helps to evaluate the weight of supporting vs. non-supporting data– Can report results in comparative ways
Helps to provide linkages to other types of data and perspectives– Can integrate qualitative and quantitative data
Types of CAQDAS
Text retrieval– Examples: the General Inquirer, CATA,
TEXTPACK, WordStat, Diction, ZyINDEX, The Text Collector
Text analysis– Examples:
• Atlas/TI,
• ETHNOGRAPH,
• NUDIST
How to Choose
What kind of computer user am I?Am I choosing for one project or for many?What kind of projects and databases will I be
working on?What kinds of analyses am I planning to do?How important is it to maintain close
proximity to the data?What are your financial constraints/access to
programs?
Text Retrieval Programs
Designed to search for, retrieve, and/or count words and phrases
Search programs– Used in preliminary data analysis to determine
whether and where pre-specified words and phrases appear and in what context
Content Analysis programs– Take inventories (make frequency distributions) of all,
or pre-specified, words contained in text
Text Retrieval: Primary Questions
What words are addressed in a text?Where are particular words used in a text?How do documents differ in terms of
vocabulary usage?What concepts are addressed in a text?To what extent are concepts of interest
addressed in a text?
Typical Features of Text Retrieval Programs
Generate text frequency distributionsGenerate vocabulary comparisons among different
textsWork with key-word listsGenerate key-word in context lists (KWIC)Search for root words (innovat*)Generate words category counts and statisticsConduct proximity searches (w/i 5 words)Conduct Boolean operator searches (innovation if
creativity not w/i 5 words)
Text Analysis Programs
Developed explicitly for the purposes of description, interpretation, and theory building
Facilitate identifying and coding elements of theoretical interest, establishing relationships and building connections
A.k.a. Code-and-Retrieve Programs (HyperQual2, Kwalitan, the Data Collector)/Code-Based Theory Builders (ATLAS/ti/NUDIST, Code-a-Text)
Primary Questions
How often do specific codes occur?How often do specific code sequences
occur?Are code sequences indicative of themes?Are code linkages indicative of conceptual
relationships?
Primary Functions of Text Analysis Programs
Attaching codes to segments of textSearching for and assembling coded
segments of textSearching for code sequences (look for
closely related or overlapping codes to identify patterns and relationships)
Counting frequencies of codes, code sequences, or counter-evidence
Practical Issues
Different types of programs can be used in concert or sequentially
Text must be computer readable: transcription, scanning, or importing
Special attention must be paid to formatting issues
All CQDA programs still require interpretation on the part of the researcher
Practical Issues (continued)
Reliability problems usually due to the ambiguity of word meanings, category definitions, or coding rules
Construct validity: constructs should be correlated with other measures of the same construct
Hypothesis validity: constructs should relate in theoretical ways to other constructs
Face validity: constructs should appear to measure what they do
Semantic validity: persons familiar with the language of the texts should agree that the list of words in a category have similar meanings
Advantages of CAQDAS
Stability of the coding scheme leads to increased consistency
Explicit coding rules yielding comparable results across multiple graders and over time
Saves time, freeing instructor to focus on interpretation and explanation
Easy manipulation of text to create different types of output and emphases
Ability to process large amounts of data in less time and saves paper
Limitations of Text Retrieval Programs
Lack of natural language processing capabilities (ambiguous concepts, broader context is lost)
Insensitivity to negation, irony, tone Inability of researcher to provide a completely
exhaustive listing of key words Inability of software to resolve references back
and forth to words elsewhere in the textCan result in “word crunching”: transforming
rich meanings into meaningless numbers
Limitations of Text Analysis Programs
Initial time investment Initial monetary investmentOutput can be tricky for studentsCan lead to a tendency to focus on details
rather than the big pictureThey don’t do the analysis for you!