Building bridges to constituents: Building bridges to constituents: Communicating through performance Communicating through performance

indicators and electronic portfoliosindicators and electronic portfoliosTrudy Banta, Victor Borden, & Susan Trudy Banta, Victor Borden, & Susan

KahnKahn

Sharon Hamilton Policy and governance issues for electronic student portfolios

OverviewOverview

Planning for improvement PIs @ IUPUI

The tip of the assessment iceberg

iPort: The institutional portfolio Placing PIs in context

ePort: The electronic student portfolio Demonstrating student learning

Completing the cycle Linking accountability to improvement

Planning for ImprovementPlanning for Improvement

Planning & Budgeting 1. Mission, Vision, Goals developed 2. Unit goals aligned 3. Annual reports on web 4. Programs based on assessable goals,

with performance indicators 5. Biennial planning/budgeting hearings

conducted

Implementation

(Everyone on campus implements goals)

Evaluation

1. Academic and administrative program reviews

2. Evaluation of process effectiveness 3. Assessment of learning outcomes

in major in general education

4. Course evaluations 5. Student assessment 6. Constituent surveys 7. Management information and analysis 8. Program cost analysis 9. Web-based evaluation tools 10. Annual campus performance report 11. NCA accreditation

Assessable

Outcomes

Instrumentation

Data Collection

Analysis

Application

of FindingsCulture

OfEvidence

Improvement 1. Reporting to internal constituents

2. Demonstrating accountability to external stakeholders

3. Applying findings in campus improvement initiatives

4. Proposing improvement initiatives

5. Improving assessment methods Web-based

data

Electronic portfolios

Planning and BudgetingPlanning and Budgeting

1. Mission, Vision, Goals developed2. Unit goals aligned3. Annual reports on web4. Programs based on assessable

goals, with performance indicators5. Biennial planning/budgeting hearings

conducted

ImplementationImplementation

Everyone on campus implements goals

EvaluationEvaluation

1. Academic and administrative program reviews2. Evaluation of process effectiveness3. Assessment of learning outcomes

in major in general education

4. Course evaluations5. Student assessment6. Constituent surveys7. Management information and analysis8. Program cost analysis9. Web-based evaluation tools10.Annual campus performance report11.NCA accreditation

ImprovementImprovement

1. Reporting to internal constituents2. Demonstrating accountability to

external stakeholders3. Applying findings in campus

improvement initiatives4. Proposing improvement initiatives 5. Improving assessment methods

• Web-based data• Electronic portfolios

Planning for ImprovementPlanning for Improvement

Planning & Budgeting 1. Mission, Vision, Goals developed 2. Unit goals aligned 3. Annual reports on web 4. Programs based on assessable goals,

with performance indicators 5. Biennial planning/budgeting hearings

conducted

Implementation

(Everyone on campus implements goals)

Evaluation

1. Academic and administrative program reviews

2. Evaluation of process effectiveness 3. Assessment of learning outcomes

in major in general education

4. Course evaluations 5. Student assessment 6. Constituent surveys 7. Management information and analysis 8. Program cost analysis 9. Web-based evaluation tools 10. Annual campus performance report 11. NCA accreditation

Assessable

Outcomes

Instrumentation

Data Collection

Analysis

Application

of FindingsCulture

OfEvidence

Improvement 1. Reporting to internal constituents

2. Demonstrating accountability to external stakeholders

3. Applying findings in campus improvement initiatives

4. Proposing improvement initiatives

5. Improving assessment methods Web-based

data

Electronic portfolios

PIs @ IUPUIPIs @ IUPUI

Performance Indicators as the tip of the assessment iceberg

PIs @ IUPUI: BackgroundPIs @ IUPUI: Background

Status of PIs in the early 1990s Kells (OECD)

Failure of PIs for national accountability in Europe and elsewhere. Movement toward quality assurance

Borden & Banta (NDIR 82) Greater possibilities for PIs if they emerge from

comprehensive planning efforts Gaither, Nedweck, & Neal (ASHE ERIC)

Increasing demand for accountability and continuing inability of HEIs to communicate about performance to external audiences

The Rise of CSFs, KSFs, KPIs Rockhart (1979) Critical Success Factors (CSFs) NACUBO/ Peat Marwick – 67 Indicators Dolence (1989) KPIs for enrollment management

Balanced Score Card (BSC)Balanced Score Card (BSC)

Kaplan & Norton propose business model Financial performance Customer service and satisfaction Process effectiveness and efficiency Organizational learning

Ruben (1999) applies BSC to HEIs Teaching/Learning - Programs/Courses, Student Outcomes Service/Outreach - University, profession, alumns, state,

prospective students, families employers Scholarship/Research - Productivity/Impact Workplace satisfaction - Faculty/staff Financial - Revenues/expenditures

Problems with Current Problems with Current SystemsSystems

Despite Ruben’s challenge to focus on mission critical activities, most focus on administrative functions

There is a disconnect between performance indicators and core performance objectives

What is the organizational link between the accountability focus of campus executives and the performance of mission critical functions?

Making the ConnectionMaking the Connection

Doing assessment right (Banta, 2002) Participative planning

Assessment must be explicitly tied to institutional functions and be associated with goals that faculty and administrators value

Implementation Assessment is most powerful at the unit level where

changes will most immediately affect the faculty and students

Improving and sustaining while the results of assessment may be used to respond

to various external accrediting and stakeholder constituents, it will first and foremost provide the means by which the organization can continuously assess and improve itself

PIs @ IUPUI: Performance PIs @ IUPUI: Performance ObjectivesObjectives

Excellence in Teaching and Learning Attract and support a better prepared, more diverse student

population. Support and enhance effective teaching Enhance undergraduate student learning and success Provide effective professional and graduate programs and support

for graduate students and post-doctoral fellows. Excellence in Research, Scholarship, and Creative Activity

Conduct world-class research and creative activities relevant to Indianapolis, the state, and beyond.

Provide support for scholarly activity and external funding. Enhance infrastructure for research and creative activities.

Excellence in Civic Engagement, Locally, Nationally, and Globally Enhance capacity for civic engagement. Enhance civic activities, partnerships, and patient and client

services. Intensify commitment and accountability to Indianapolis, Central

Indiana, and the state.

Co

llab

ora

tio

n –

Div

ers

ity

– B

es

t P

rac

tic

es

PI Development ProcessPI Development Process

Vested interested groups (handout) Brainstorming session

What evidence would convince a (neighbor, prospective student, legislator) that you we were doing well?

What evidence do you (or could you) use in your unit to assess performance?

3-4 performance indicators per objective

Three level rubric (green, yellow, red) Adapted Nominal Group Technique for voting

Status and ResultsStatus and Results

In Handout Summary of performance objectives and

performance indicators, with measures shown for one set of indicators

Published in… Campus annual performance report Chancellors State of Diversity address Institutional portfolio

iPortiPort

The Electronic Institutional Portfolio

““Institutional Portfolio”Institutional Portfolio”

A focused selection of authentic institutional work, combined with narrative analysis, that demonstrates specific institutional achievements and shows learning and improvement over time—i.e., “institutional effectiveness”

IUPUI’s iPortIUPUI’s iPort

Begun in 1998 as part of national grant-funded project

Organized around main components of institutional mission and major strategic priorities

Intended for purposes of both internal improvement and accountability to multiple stakeholders

Used as platform for accreditation self-study in 2002

Is on the Web—available to all

What do we gain from work onWhat do we gain from work onan electronic institutional portfolio?an electronic institutional portfolio?

Occasions for focused, campus-wide discussions of areas needing improvement

A means of demonstrating institution’s commitment to accountability and effectiveness

An easy, accessible medium for communicating with stakeholders about accomplishments and effectiveness

Next StepsNext Steps

Develop new iteration/update as annual performance report

Update the design and technological infrastructure

Foreground links to other relevant campus Web sites

Incorporate more direct evidence of student learning through interface with student electronic portfolio

ePortfolioePortfolio

The Electronic Student Portfolio

Authn/Authz Security WorkflowComm.Tools

Storage

“Unbundling”

In Touch

Syllabus

Testing

Gradebooks

Schedule

EreservesDigital content

Full Text articles

Federated Searching

Learning ProfileLearning Matrix

Advising

Resume/Vita BuilderResearch Manager

Content Manager

Knowledge Mapper

Authn/Authz Security WorkflowCommTools.

StorageOther

ServicesOther

Services

ePortfolio Goals

Support students' learning of and engagement with the PULs over their entire undergraduate experience at IUPUI beginning in the freshman learning community and culminating in the capstone experience

Assist both faculty and students to reach a clearer, more coherent understanding of how various aspects of the curriculum support students' increasing mastery of the PULs

Contribute to assessment of student learning of the PULs at the levels of the individual student, the course, program, and institution

ePortfolio and the PULs

The ePortfolio is organized around IUPUI's Principles of Undergraduate Learning

1. Core communication and quantitative skills

2. Critical thinking

3. Integration and application of knowledge

4. Intellectual depth, breadth, and adaptiveness

5. Understanding society and culture

6. Values and ethics

ePortfolio Levels of Competence

Introductory: What all undergraduate students at IUPUI should know and be able to do in relation to the PULs within the first 26 credit hours

Intermediate: What all undergraduate students at IUPUI should know and be able to do in relation to the PULs within the first 56 credit hours

Advanced: What all baccalaureate students at IUPUI should know and be able to do in relation to PULs in their major or profession or academic program

Learning MatrixLearning Matrix

Planning for ImprovementPlanning for Improvement

Planning & Budgeting 1. Mission, Vision, Goals developed 2. Unit goals aligned 3. Annual reports on web 4. Programs based on assessable goals,

with performance indicators 5. Biennial planning/budgeting hearings

conducted

Implementation

(Everyone on campus implements goals)

Evaluation

1. Academic and administrative program reviews

2. Evaluation of process effectiveness 3. Assessment of learning outcomes

in major in general education

4. Course evaluations 5. Student assessment 6. Constituent surveys 7. Management information and analysis 8. Program cost analysis 9. Web-based evaluation tools 10. Annual campus performance report 11. NCA accreditation

Assessable

Outcomes

Instrumentation

Data Collection

Analysis

Application

of FindingsCulture

OfEvidence

Improvement 1. Reporting to internal constituents

2. Demonstrating accountability to external stakeholders

3. Applying findings in campus improvement initiatives

4. Proposing improvement initiatives

5. Improving assessment methods Web-based

data

Electronic portfolios