Closing the Loop on Student Learning Assessment

Mohua Bose, Ph.D Mary Beth Hanner, Ph.D, RN, ANEF

Director of Outcomes Assessment Provost and Chief Academic Officer

Excelsior College, Albany, NY

Presentation at MSCHE, December 12, 2011

Purpose

Describe a framework that addresses assessment of student learning in a non-traditional degree completion institution.

Discuss improvements based on assessment results at the program and institutional level.

Describe the process of facilitating an assessment culture.

Background

Excelsior College founded in 1971 (previously Regents College).

1998 Chartered as a private, independent college.

Large student body (approximately 33,000 students).

Distance education, flexible learning format.

Philosophy:

“What you know is more important that where or how you learned it”

Background

Valuing prior learning is a central premise for adult learners (Green, 1998).

Acceptance of a variety of credit sources:Examples: Regionally accredited institutions ACE/PONSI approved courses Exams – CLEP, DANTE,EC EC portfolio assessment for prior learning Online courses at Excelsior and preferred providers

Residency requirement (presently minimal, historically none).

Background

Accreditation Middle States (all programs) NLNAC (specialized accreditation - Nursing) ABET (specialized accreditation - Technology) IACBE (specialized accreditation - Business)

Challenge:Who is a typical student?

Career oriented working adult

Average age: 39 year

College 1

College 4

College 3

College 2

College 5

Excelsior College

No first time, full time students.Credits transferred from other approved sources ranges from 25% to

75% of total credits required for the degree program.

At a distance

Accountability and Continuous Improvement

OAIR Steering Committee

OAIR Office

School ofBusiness

& Tech

School of Liberal

Arts

School of

Nursing

School of Health Sciences

Institutional Assessment Plan for Student Learning (IAPSL)

Institutional Assessment Plan for Student Learning (IAPSL)

Provides a framework for assessment of learning outcomes at course, program and institutional level.

Addresses the key question “How effectively do we deliver on our mission and make a distinctive impact relative to our resources?” (Collins, 2011).

IAPSL link: http://www.excelsior.edu/web/excelsior-the-institution

Establish clear learning

outcomes

Use the ResultsProvide learning

opportunities

Assess Student Learning

Assessment Process at Schools

Step 1: Establish Learning Outcomes

Ultimate responsibility to determine learning outcomes for a program rests with the faculty.

Instructional/Adjunct faculty receive an online faculty training that orients them to EC. The orientation includes an introduction to learning outcomes assessment.

Development of online courses requires

completion of a structured template that involves program, course and module outcomes.

Step 1: Establish Learning Outcomes (continued)

A raised awareness of the importance of learning outcomes assessment at the College due to OAIRSC and OAIR offices. Senior level personnel significantly involved (eg. Deans).

In addition to OAIR, an assessment director within the schools work with the faculty.

Continuous and evolving process in defining learning outcomes.

Step 2: Create Learning Opportunities

Monitoring the learning opportunities provided to students:

Course revisions based on:Centralized student course evaluationsFaculty evaluation of coursesFaculty curriculum committee

Curriculum reviews

Example: Curriculum Review

Committee of 3 reviewers (some members are completely external to the college).

2-3 days including two teleconferences. Reviewers examine curricular structure, topical

areas covered, course descriptions. Provide feedback on curriculum rigor,

curriculum currency, and program outcomes attainability and currency.

Feedback from the external curriculum review is taken back to program faculty.

Step 3: Assessing Student Learning

Direct assessments of program outcome measures for all programs using:oCapstones, Portfolios, or Exams (theory or

performance)o State licensing exams (e.g. NCLEX)

Direct assessments of general education outcomes using:o ETS Proficiency Profile & VALUE Rubrics

Step 3: Assessing Student Learning

Also have indirect assessments:

Alumni surveys

Supervisor surveys

Centralized for all degree programs

Discipline Specific Example on Closing the Loop

Associate Degree in Nursing

NCLEX pass rates Consistently at or above national average

over a decade. Dipped below for one quarter in national

and state pass rates by approximately 1%. Conducted analysis of subscales on the

NCLEX exam. Examined the CPNE exam.

Associate Degree in Nursing

Analysis of the data revealed that students scored lower on o Knowledge of adolescent growth and

developmentoCare of the ill adolescent

After faculty review – new curriculum requirements.

Growth and Development across the Lifespan.

Associate Degree in Nursing

After approximately 2 years• Scores exceeded both state and national

levels• NCLEX scores in overall exam and

subscales

Remained up for a number of years.

Continuing to monitor.

General Education Example

General Education

Assess WritingOutcome: Write with grammatical

competence, clarity, and coherence

Administered standardized assessment: ETS Proficiency Profile

Reading Writing Mathematics Critical thinking

General Education

ETS Proficiency Profile administered annually 2009, 2010, and 2011.

Results pertaining to Writing show that Excelsior students at the Associate’s and Bachelor’s level have performed at or above the ETS norms for writing.

General Education

College decided to use multiple measures to assess writing.

Use of VALUE rubrics in writing Capstone papers from Bachelor’s degree

programs across the College used. Independent panel of reviewers. Data collection completed, shows

variations in student performance. Closing the loop takes time.

Key Steps in Implementing Outcomes Assessment

Institutional support and commitment. Creation of a college-wide oversight

committee to set a relevant and strategic agenda for assessment of student learning and institutional performance.

A visionary and guiding document for implementation of outcomes assessment College-wide.

Key Steps in Implementing Outcomes Assessment

Training and implementation of an assessment management tool/database.

Centralized assessment model supported by a distributed model of assessment staff.

Showcase assessment results - Quarterly Dean’s presentations at Steering Committees.

Bi-annual assessment audits. Continuous and evolving process!