AZAIR Mixed Methods 07

8/13/2019 AZAIR Mixed Methods 07

1/22

Using Mixed Method Approaches

in Institutional Research

Rick Kroc

University of Arizona

Richard Howard

University of Minnesota

Arizona Association for Institutional Research

Annual Meeting

Tucson, Arizona March 2007


2/22

Purpose of Decision Support

is

to reduce the risk to the decision

maker


3/22

Structure and Subjectivity


4/22

Research Process


5/22

Ways of Knowing

Two Classic Paradigms

Positivist Constructivist

there are multiple realities

and that truth is ever

changing, dependent oncontext and the individual

(subject and researcher)

there is a single reality or truth

across time and contexts, and

that this truth can be understood

through the objective study ofindependent variables


6/22

Conceptual Model of the

Constructivist Research ProcessAfter a literature review and pilot study,

define the Study:ProblemPurposeQuestion

Data Collection &

Management

Data Restructuring/Coding &

Analyzing/Read-Theme

Data Verification withRespondents:

FACTUAL ERRORAnalysis Verification with

Reviewers:

Interpretation Error

COMMUNICATE

New Theory/Model

TEST

Theory/Model


7/22

Conceptual Model of the Positivist

Research Process

After a literature review and pilot study

define the project

ProblemPurpose

QuestionOperationalize theory or model constructs

Data Collection and

Management

Data restructuring and

analysis

(Interpretation of Results)

Communication

of new

knowledge

For internal decision support-

test meaning of the findings

For external contexts-test/replicate forgeneralization


8/22

RESEARCH PARADIGMS

APPROACHES

Constructivist Positivist

QuantitativeQualitative

Consumer Generalizes Researcher Generalizes


9/22


10/22

Use of Mixed Method Approaches

in Institutional Research

Provides answers to two questions:

What- What happened and what was theoutcome? Answers tend to be

arrived at using quantitative methods

Why- Why did people behave the way theydid? Answers tend to be arrived at using

qualitative methods


11/22

What - goals are - how measuredSupports Accountability (External)

Whygoals were met or notpeople

issuesOften the focus ofAssessment/Evaluation (Internal)

So What?


12/22

Ways of Knowing

A Third Paradigm

Positivist Constructivist

there are multiple realities

and that truth is ever

changing, dependent on

context and the individual

(subject and researcher)

there is a single reality or truth

across time and contexts, and

that this truth can be understood

through the objective study of

independent variables

PragmatismUse what ever approach

and methodology that will

best answer the question.


13/22

Charge from President Likins

As Chair of the NCAA Division I Presidential Task Force

on the Future of Intercollegiate Athletics, I have both the

opportunity and the responsibility to examine the

culture of higher education athletics programs in

America. My responsibilities begin at home. My task

force experience as well as events at colleges anduniversities around the country led me to conclude that

it is appropriate to examine the environment defined by

the culture of high-level athletic competition here at The

University of Arizona. At the national level a seriousmovement to strengthen the academic experience of

student-athletes is well underway. We must examine the

presence of any cultural influences here that have the

potential of distorting the shared values of academics

and athletics in higher education.


14/22

Focus of the Study

The study was divided into two phases. Phase I,mostly quantitativein nature, was designed tolook somewhat broadly at the two fundamentalstudy areas: Student athletes academic experience; and

Student athletes culture and environment.

Using qualitativemethods, Phase IIwas designedto dig more deeply into the critical issuesidentified from Phase I. Detailed, targetedrecommendations would be developed from thesetwo phases.


15/22

Domain of the Study

Athletes academic

experience

Academic preparation

Progress to degree Student engagement

Academic support

Post-eligibility academic

experience Other academic issues

Athletes culture and

environment

Alcohol and drugs

Dishonesty and cheating Gambling

Racism

Sexism

Sexual behavior Violence

Weapons


16/22

Phase 1Quantitative

Board of Regents and NCAA reports on athletes gradesand graduation rates (ten year history).

Institutional student records data.

Health and Wellness Survey, administered annually toassess student alcohol, drug, and sexual behaviors (sevenyear history).

Dean of Students Code of Conduct and Code of AcademicIntegrity violations (five year history).

Coalition on Intercollegiate Athletics (COIA) report.

Annual University Survey of Graduating Seniors, assessing

student engagement, satisfaction, and other academicissues (five year history).

NCAA GOALS survey (student-athletes and student non-athletes).

Additional UA-specific survey questions appended toGOALS survey (student-athletes and student non-athletes).


17/22

Phase IIQualitative

Focus groups

Interviews

Follow-up surveys

Populations:

Students (athletes and non-athletes)

Coaches

Faculty Staff

Advisors


18/22

The Value of a Mixed Methods

Approach

Athletic culture spans a spectrum of

areas, some of which (alcohol use, for

example) have become fairly wellunderstood over the years from

quantitative methods, others of which

(physical violence, weapons use) needthe deeper exploration and understanding

provided by qualititative methods.


19/22


Approach

The formative and summative aspects of

this study, which is more like program

evaluation in many ways than like atraditional research study, lend

themselves to a mixed methods

approach.


20/22


Approach

Audiences and stakeholders consist of

some people who respond best to

systematic, inferentially rigorous,quantitative data, but also others who

want to hear the complex, richly detailed

stories derived from the individuals whoare immersed in the culture. Different

audiences resonate to different

approaches.


21/22


Approach

Mixed methods also means that multiplemethods have been employed, which isessential to using triangulation as a

means to establish greater validity of thefindings. Any single approach used in themessy world of social science researchwill be flawed. If a wide array of methodsleads to consistent conclusions, thenanalysts and stakeholders feel moreconfident about conclusions and

recommendations.


22/22

Questions??

Documents

AZAIR Mixed Methods 07