Transforming IR Leadership and Creating Data-Informed Decision Cultures

Presentation found at https://irds.iupui.edu/reports-presentations/conference-presentations/index.html

IUPUI

Michele Hansen, Ph.D., Assistant Vice Chancellor

Institutional Research and Decision Support (IRDS)

Caleb J. Keith, Ph.D., Director of Institutional Effectiveness

and Survey Research, IRDS

Steven S. Graunke, Ed.D., Director of Institutional Research

and Assessment, IRDS

Our Vision/Mission

We provide timely, accurate, and actionable information to

support decision making at IUPUI. Decision makers have the

information necessary to achieve maximum effectiveness in regard

to IUPUI’s Strategic vision, mission, values, and goals related to the

success and learning of our students as well as overall institutional

effectiveness. Our vision is to be a nationally recognized leader in

institutional research and decision support in higher education.

• IRDS new office in August

2015.

• Reorganized all data-

generating, decision support

offices under one structure to

leverage expertise and

integrate data across units.

• Had opportunity to organize

new office guided by

Statement of Aspirational

Practice for Institutional

Research.

3

Institutional Research and Decision Support

Context

IUPUI

Statement of Aspirational Practice For

Institutional Research – Association of

Institutional Research (AIR)

Statement of

Aspirational Practice

Student Focus

Expanded Definition of

Decision Makers

Structures and

Leadership for IR

Leadership for IR

Function

IUPUI Selected as 1 of

10 Founding

Institutions

IUPUI

IRDS Activities and Reports to Support

Decision Making

Institutional Research

(Retention and Graduation

Rates)

Strategic Enrollment

Management

Assessment of Student Learning

and Development

Survey Research Methods

Course Evaluations

Institutional Effectiveness and Strategic

Planning Decision Support

Program Evaluation and

Program Review

(assessing what works and what

needs improvement)

IUPUI

Executive Vice Chancellor

Chief Academic Officer

(Kathy E. Johnson)

Director

Institutional Research and Assessment

(Steve S. Graunke)

PAE 5AD

Associate Director

(Teresa Troke)

IVY Tech/IUPUI Management Analyst

(Vacant)

Assistant Director Enrollment Management

Analyst

(Norma Fewell)

Assistant Director

(Wendy Lin)

Director

Institutional Effectiveness and Survey Research

(Caleb Keith)

Assistant Director

(Robert Janik)

Course Evaluation & Data Processing Coordinator

(Kate Forrest)

Data Specialist

(Rick L. Morgan) Management Analyst

(Clif Marsiglio)

Director

Evaluation and Psychometric Services

(Howard Mzumara)

Assistant Vice Chancellor Institutional Research and Decision

Support

(Michele J. Hansen)

Institutional Research Coordinator

(Jessicah Rauch)

Web Developer/Analyst

(Alex Kitcoff)

Senior Administrative Secretary

(Janet Epps)

Two Graduate Student Research Assistants for IRDS

(reports to Robbie and Jessicah/Steve)

Institutional Research and Decision Support (IRDS) Organizational Chart

IUPUI

IRDS Key Practices and Strategies to

Expand Notion of Decision Makers

Consider the information needs of faculty, staff, administrators, and students.

Developed a comprehensive integrated data infrastructure (student, faculty, staff, facilities, financial, research, community impact, and strategic planning indicators) to provide unprecedented access to data for all decision makers across IUPUI, especially those on front line with students and students themselves.

Optimized the use of enabled technology such as Tableau to put data in the hands of decision makers. The technology-enabled data reports allow users to explore, interact with, and analyze information that is actionable and tailored to specific decisions.

IUPUI

Organization Around Decision Maker• Multiple decision makers across a campus – and IRDS is organized so those

decision makers have access and we serve as facilitators

• Chancellor and Chancellor’s Cabinet

• Schools

• Academic Programs

• Division of Undergraduate Education

• Enrollment Management

• Division of Diversity, Equity, and Inclusion

• Academic and Faculty Affairs

• Staff Affairs (Human Resources and the Office of Equal Opportunity)

• Etc.

• Understand anatomy of decision

• Data infrastructure

• Relationship building

• Access to decision makers

• Building capacity for others (outside of IR) to collect data appropriately

• Technology that allows for decision makers to have access to data

• Ongoing training, consultation, and support

Building a Capacity for Decision Making

INDIANA UNIVERSITY–PURDUE UNIVERSITY INDIANAPOLIS

Practices and Strategies to Enhance IR

Leadership Function

Lead by supporting and coordinating all institutional research, institutional effectiveness, mandatory reporting, business analytics, and other data-focused decision-support activities.

There is a significant focus on building relationships with individuals throughout the institution.

Have spent the last year engaged in understanding data and information structures and capacities.

Critical aspect of reorganization was to connect disparate pieces of information (student, faculty, staff, budget, human resources, campus climate for diversity).

Developed a Student Data Policy Advisory Group, Survey Research Policy, Faculty and Staff Individual Data Advisory Councils

10

INDIANA UNIVERSITY–PURDUE UNIVERSITY INDIANAPOLIS

Data Governance and Policies

• Student Data Policy Advisory Group

• Staff and Faculty Data Policy Advisory Group

• Survey Research Policy

INDIANA UNIVERSITY–PURDUE UNIVERSITY INDIANAPOLIS

IR Leadership and Creating a Data-Informed

Decision Cultures

1. Matrix Network Model of

Institutional Research

– We have small teams within IRDS who

work collaboratively with decision

makers in schools and in units to

ensure that that we are providing them

with quality information and data tools

so that they are empowered and

effective decision makers. Process is

integrative, nimble, and data is

actionable and aligned with strategic

planning decisions.

IUPUI

A Strategic Information Council launched at IUPUI in February, 2017 and

will meet biweekly. The major purpose of this Council will be to regularly and

strategically mine our data repositories, including advising tools and

institutional analytics developed by [vendors] or available through Canvas, in

order to identify soluble barriers to student success and to address them

systematically. The council will be tasked with nurturing a deliberate cycle of

piloting responses to identified barriers, evaluating their efficacy, and rapidly

scaling them up or implementing policy to support undergraduate student

retention and success in coordination with appropriate councils, faculty

governance groups, and the Center for Teaching and Learning.

Leveraging Data Analytic Power with

Formation of Strategic Information Council

IUPUI

Vice Chancellor for Student

Affairs

Vice Chancellor for Finance and Administration

Leader of Institutional Research &

Decision Support

Chief Academic Officer

Leader of Enrollment

Management

Leader of Undergraduate

Education

Leader of University College

Leader of Campus Career

& Advising Services

Strategic Information Council

IUPUI

Strategic Information

Council

Deans Council

Advising Council

Student Experience

Council

Cabinet

Data Inquiry Group

University College Councils

IUPUI

Data Inquiry Group (DIG)

DIG primary purpose is to leverage the data and analytic tools available

across IUPUI. Group members are composed of key data users/analysts

in schools and other units as appointed by Deans from each school or

Vice Chancellors of units. About 43 members. Monthly meetings.

Charge: Serve as data coaches for IUPUI in collaboration with the

office of Institutional Research and Decision Support. As such, group

members engage in dialogue around data-driven decision-making,

serve as advisors around types of data needed for decision-making

and provide input on issues such as data quality, literacy, use,

capacity, and transparency.

INDIANA UNIVERSITY–PURDUE UNIVERSITY INDIANAPOLIS

JagsSpeak

● What: Implemented a student survey process to ensure equitable student

voice in key institutional decisions.

● Why: Desired more student input/better response rates.

● How: Designed short, action-oriented, topical surveys on relevant student

issues.

● How help: Obtained student input on survey topics and successfully

implemented iterative, ongoing approach to capture and act on student

voices. 17

Self-Assessment of

Progress Made Toward

Data-Informed Decision

Culture

IUPUI

IRDS Retreat and AIR Data Rubric

Primary Areas of Focus:

• AIR Data Rubric Results and Activity

• Strategic Planning Around Statement of Aspirational Practice

• IRDS Vision Statement

IUPUI

AIR Data Rubric 12 Areas (DRAFT)1. Culture of Data Use

2. Use of Data

3. Student Context

4. Student Input

5. Culture of Inquiry

6. Data Quality

7. Data Literacy

8. Professional Development

9. Data Production

10. Decision Makers Expectations

11. Decision Makers Access

12. Data Policy

Outreach to users

IUPUI

Redesigned Website with

Accessible Self-Service Data

Tools

Institutional Research and

Decision Support

https://irds.iupui.edu/

Contains highly interactive

dashboards allow users to drill

down and filter to allow

detailed exploration of key

indicators associated with the

IUPUI Strategic Plan.

.

IUPUI

IRDS Data Workshops

IUPUI

Feedback & Results

1. Most useful aspect of workshop?

– “Watching the presenters navigate through the different reports”

– “Learning about the sheer breadth and depth of the information available from IRDS, and how user friendly the dashboards, filters, and functions are”

2. How to improve workshop?

– “Break into faculty based data and student programming/success data. Then you could attend both, but could pick the one most relevant.”

– “More real-world applicability is nice with any data discussion.”

– “Navigation was difficult to keep up with”

– “Shorter, topic driven workshops”

– “More specific, in-depth workshops”

IUPUI

Next Steps

1. Specific topic sessions

– I Heart Surveys: Where to find IUPUI survey results and best practices for crafting your own.

– Faculty and Staff Data

– Understanding course enrollment trends and information

– Admissions and Enrollment Point-in-Cycle

– Using Blue Online Course Evaluations at IUPUI: Benefits, Challenges, and Lessons Learned for Improving Teaching and Learning\

2. Continue with new topics in Fall

3. Solicit DIG group for topics

IUPUI

Challenges and Potential Obstacles

• Students as decision makers (how do we know what they will use/need without overwhelming them, what formats effective).

• Unprecedented access to data does not equal to clear understanding, effective interpretation, and use.

• Facilitating easy navigation of IR reports still a work in progress.

• Decision makers do not always know how to use data for effective decision making.

• Difficult to expect very busy directors in student-facing roles to be expert data analysts, evaluators, and assessment specialists.

• Data governance not owned by IR at IU - IT responsible for data warehouse, data stewardship, and governance.

Questions?