Overview of Institutional Overview of Institutional Effectiveness Concepts and Effectiveness Concepts and

MeasuresMeasures

Andreea M. Serban, Ph.D.

Solano Community College

July 19, 2006

1

DefinitionsDefinitions

2

Institutional Effectiveness

“The ability of an institution to match its performance to established purposes as stated in its mission” (Alfred et al, p.6)

Expression of institutional values and priorities

“An internal process of planning and evaluation that is intended to ensure that the college’s performance matches its purposes. A global process that uses assessment strategies to demonstrate accountability” (Midlands Technical College, p.ix).

  

PurposesPurposes

3

Clarification of Mission

Improvement

Identification of Priorities

Working Document

Planning and Decision Making

Enhance Reputation

Historical Record

Accreditation Requirement

Information for Prospective and Current Students

Core ConceptsCore Concepts

4

Stakeholder Needs

Mission Results

Resources

Costs

Institutional Effectiveness and PlanningInstitutional Effectiveness and Planning

5

•Institutional effectiveness measures are part of the annual evaluation of the strategic plan

•The information is used to determine progress towards achievement of stated goals in the strategic plan

•Establish a structure to monitor, report and take corrective actions based on institutional effectiveness measures

•Departmental effectiveness measures are linked to and, as appropriate, aggregate into institutional effectiveness measures

•The college needs to have a systematic approach to communicating the data from the institutional effectiveness to internal and external constituents on a regular cycle (annual basis suggested)

Modeling and Measuring EffectivenessModeling and Measuring Effectiveness

6

Outcomes serve as benchmarks for the college mission by comparing results with purpose, within the limits of costs and available resources

When outcomes are compared with an institution’s mission and goals, the result is a composite picture of an institution’s effectivenessAn indicator that helps create this composite picture describes a condition or result that the community college can produce regularly, report publicly, and use systematically in decision making

Source: AACC

A core indicator is a measure that describes a critical, widely recognized outcome of mission – one that is clearly responsive to key constituent groups and is produced regularly.

1999 AACC Model1999 AACC ModelMission and Core IndicatorsMission and Core Indicators

7

Student Progress Student Goal AttainmentPersistence (Fall to Fall)Degree Completion Rate

Workforce DevelopmentPlacement Rate in the WorkforceEmployer Assessment of StudentsLicensure/Certification Pass RatesClient Assessment of Programs and Services

General Education Demonstration of Critical Literacy SkillsDemonstration of Citizenship Skills

1999 AACC Model1999 AACC ModelMission and Core Indicators (continued)Mission and Core Indicators (continued)

8

Transfer Preparation Number and Rate Who TransferPerformance After Transfer

Developmental Skills Success in Subsequent, Related Coursework

Outreach Participation Rate in Service AreaResponsiveness to Community Needs

Feasibility of the AACC ModelFeasibility of the AACC Model

9

Core Indicators for which Data Are Available

Core Indicators for which Data Are Not Easily Available

Persistence (Fall to Fall)

Degree Completion Rates

Job Placement Rates (data have limitations)

Number and Rate Who Transfer

Success in Subsequent, Related Courses

Licensure/Pass Rates

Participation Rate in the Service Area

Student Goal Attainment

Employer Assessment of Students

Client Assessment of Programs and Services (at a minimum use student surveys)

Demonstration of Critical Literacy Skills

Demonstration of Citizenship Skills

Performance After Transfer

Responsiveness to Community Needs

Expanding on the AACC ModelExpanding on the AACC ModelExamples from Community CollegesExamples from Community Colleges

10

Santa Barbara City College – Criteria for selecting institutional effectiveness measures

•Related to mission and College Plan•Reflective of the changes in relevant literature•Reflective of the ability of the Office of Institutional Assessment, Research and Planning to obtain the information from the college student system, other college databases/sources or external sources•Quantifiable•Accepted by the college community as valid

Expanding on the AACC ModelExpanding on the AACC ModelExamples from Community CollegesExamples from Community Colleges

11

Santa Barbara City College (list of measures provided in separate document)

•Measures of institutional effectiveness developed for and linked to each major area of the college plan

•Discussion of institutional effectiveness trends and implications for the college included in the annual evaluation of the college plan

•Annual presentations and discussions with the Board of Trustees and shared governance groups (College Planning Council, Academic Senate)

Expanding on the AACC ModelExpanding on the AACC ModelExamples from Community CollegesExamples from Community Colleges

12

Long Beach City College (annual institutional effectiveness report and latest presentation to Board of Trustees provided in separate documents)

•Uses the AACC outcome dimensions tied to Astin’s assessment model which differentiates between inputs, process/environment and outcomes

•Measures of institutional effectiveness tied to the long range goals specified in the Educational Master Plan

Implications for Solano Community CollegeImplications for Solano Community College

13

•The college had previously developed performance indicators related to the goals included in the Strategic Plan as outlined in the “Review and Validation of Solano Community College District’s Strategic Plan” May 2000. Those performance indicators represent a solid base for creating an institutional effectiveness framework related to the Strategic Plan goals and objectives.

•Institutional effectiveness measures, once developed, should be monitored, reported, discussed and integrated into the annual evaluation of the Strategic Plan.

ReferencesReferences

14

Alfred, R., Ewell, P., Hudgins, J., and McClenney, K. (1999). Core Indicators of Effectiveness for Community Colleges, Second Edition. Washington, D C: Community College Press, American Association for Community Colleges.

Midlands Technical College. (1997). Managing Your Institution’s Effectiveness. A User Guide. Washington, D C: Community College Press, American Association for Community Colleges.

ResourcesResources

15

Banta, T. W., Borden, V.M.H. (1994). Performance Indicators for Accountability and Improvement. In V.M.H Borden and T. W. Banta (eds.), Using Performance Indicators to Guide Strategic Decision Making. New Directions for Institutional Research No. 82: San Francisco, CA: Jossey Bass.

Burke, J. C., Modarresi, S., Serban, A.M. (1999). Performance: Shouldn’t It Count for Something in State Budgeting?. Change, November/December, 31(6), 16-23.

Burke, J. C. (1998). Performance Funding Indicators: Concerns, Values, and Models for State Colleges and Universities. In J. C. Burke and A. M. Serban (eds), Performance Funding for Public Higher Education: Fad or Trend?. New Directions for Institutional Research No. 97. San Francisco, CA: Jossey Bass.

Cleary, T. S. (2001). Defining Quality Through the Eyes of Campus Stakeholders. Community College Journal, August/September, 41-47.

Council for Higher Education Accreditation. (2000). The Common Data Project. CHEA Occasional Paper, August. Washington: DC: CHEA.

ResourcesResources

16

Ewell, P. T. (1992). Outcomes Assessment, Institutional Effectiveness, and Accreditation: A Conceptual Exploration. Resource Paper for the Council on Postsecondary Accreditation Task Force on Institutional Effectiveness (ERIC Document No. 343513).

McLeod, M. W., Cotten, D. K. (1998). Essential Decisions in Institutional Effectiveness Assessment. Visions: The Journal of Applied Research for the Florida Association of Community Colleges, 2(1), 39-42.

Nichols, J. O. (1995). A Practitioner’s Handbook for Institutional Effectiveness and Student Outcomes Assessment Implementation. Bronx, NY: Agathon Press.

Ronco, S. L., Brown, S. G.. (2000). Finding the "Start Line" with an Institutional Effectiveness Inventory. Paper presented at the Annual Forum of the Association for Institutional Research, Cincinnati, OH, May 21-23, 2000.

Roueche, J. E., Roueche, S. D. E., Eileen E.. (2001). Pursuing Excellence: The Community College of Denver. Community College Journal of Research and Practice, 25(7), 517-537.

Roueche, J. E., Johnson, L. R., Roueche, S. D., & Associates. (1997). Embracing the Tiger: The Effectiveness Debate and the Community College. Washington, D.C.: The Community College Press.

ResourcesResources

17

Serban, A.M. (2001). Institutional Effectiveness and Performance Funding for Community Colleges: Where Do Institutional and State Values and Priorities Converge? Presented at the Annual Forum of the Association for Institutional Research. Long Beach, CA, June 4, 2001.

Sorensen, James E. (1998). An Investigative Study on the Systematic Application of Effectiveness Indicators for Institutional Improvement in Northwest Community Colleges.

Sullivan, M. M., Wilds, P. C. (2001). Institutional Effectiveness: More Than Measuring Objectives, More Than Student Assessment. Assessment Update, 13(9), 4-6.