Measuring digital learning in the 21st century

Engaging Online Faculty and Administrators in the Assessment Process at the American Public University SystemMeasuring Digital Learning in the 21st Century - Data Warehouses, Dialects, and

Dashboards

INTRODUCTIONS

AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE

Dave BecherDirector, Academic Information Analysis

Phil Ice, Ph.D.Director, Course Design, Research, and

Development

Frank McCluskey, Ph.D.Executive Vice President and Provost

Jennifer Stephens Helm, Ph.D.Associate Vice-President, Dean of Assessment


WHO WE ARE

American Public University System has been dedicated to educating those who serve since our doors first opened in 1991. The American Public University System (APUS) is a fully online system, encompassing both American Military University and American Public University. We are regionally accredited by the Higher Learning Commission (HLC) of the North Central Association.

“Educating Those Who Serve”

EXCELLENCE IN EDUCATION


We face challenges that are well documented in the educational literature. We are an open enrollment college, as well as a fully online institution of higher education. APUS is committed to improving our learning environment. To continue and increase the success of our students we employ a rigorous assessment plan. Excellence in education is fostered through a cycle of data collection, analysis, remediation, and then further data collection to measure the learning again.

ASSESSMENT AT APUS


STRATEGY #1:


External Review

Analysis

Program Review

Meeting

Three Year Plan

Follow-Up

Data Collection

Promote Accountability Among Faculty and Administrators – APUS

Program Review Process

EFFECTIVE DOCUMENTATION


Promoting Accountability through the Documentation of Student Learning Outcomes Evidence

We begin with the assumption that our faculty are the experts. Our data collection assists them in translating their expertise into student learning. We want to help them free our students from fixed ideas, cloudy thinking and imprecise analysis. We want to liberate the learning already taking place.



• Course books

• Electronic resources

• Learning strategies

Library and Learning Resources

• Student learning outcomes

• Instructional strategies

• Evaluation procedures

• Academic rigor

Curriculum Assessment

• Expert reviewer report

• Industry Advisory Council report

External Reviewer Feedback

Data Collection and Analysis



• Analysis of faculty credentials and expertise to ensure breadth and diversity

Faculty

• Student demographic information

• Enrollment History

• Growth trends

Students

• Curricular Mapping

• Assessment measures

• Fact book

Learning Outcomes Assessment




• Program benchmarking with similar programs and institutions

Program Benchmarking

• Evaluation of findings

• Program recommendations

• Three year proposed strategic plan

Program Directory Summary

• Dean’s observations

• Meeting minutes

Review and Findings


STRATEGY #2


• One of fourteen charter member institutions

• College Choices for Adults website

http://www.collegechoicesforadults.org/

Transparency by Design Initiative

http://www.apus.edu/community-scholars/learning-outcomes-assessment/index.htm

APUS Learning Outcomes Assessment Website

• Meetings

• Faculty, staff, and student newsletters

• Faculty and student lounges

Internal Communications

Inform Internal and ExternalStakeholders on the Achievement

of Student Learning Outcomes






STRATEGY #3


Establish a Reciprocal Relationship among Stakeholders and the Institution on the Sharing of Data

• Regular dissemination of data to stakeholders

• Reciprocal relationship between data team, Deans, and Program Directors

• Assessment feedback loop

FACT BOOK DATA COLLECTION TIMELINE


STUDENT DEMOGRAPHICS


STUDENT COUNT


GRADES BY CORE COURSES (IN PROGRAM): %


STRATEGY #4


Incorporate Assessment Data intoDecision Making Processes

APUS utilizes a number of nationally benchmarked tests and validated instrumentsto measure learner engagement, learner success, and learning outcomes.

STRATEGY #4

National Survey of Student Engagement - NSSE

• Administered at several hundred colleges and universities in the United States and Canada

• Hundreds of thousands of freshman and seniors surveyed using a validated instrument recognized in the literature and nationally

• Survey uses more than 70 questions gathering self reported student perceptions

APUS is one of few fully online, for-profit institutions of higher education participating.


STRATEGY #4

The Educational Testing Service

• APUS uses the Proficiency Profile (formerly the MAPP) to measure skills of college program graduates.

• Presumption for open admission and fully online institutions to score below the national average.

• 2009 APUS scores rise above the national average with the sole exception of mathematics which is less than 1% below the norm (See Table 1 – 2009 MAPP Test Data).

• ETS performs a major field test for seven of our undergraduate majors upon graduation (See Table 2 – 2009 MFT CJ Data).


COMMUNITY OF INQUIRY


APUS participates in the Community ofInquiry (COI) End of Course survey.

The COI is a validated instrument based onthe research around Social, Cognitive, andTeaching Presence.

More than 500,000 learners have used this instrument and created a strong baseline forfurther research.

Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105

• Promote Accountability Among Faculty and Administrators

• Inform Internal and External Stakeholders on the Achievement of Student Learning Outcomes

• Establish a Reciprocal Relationship among Stakeholders and the Institution on the Sharing of Data

• Incorporate Assessment Data into Decision Making Processes

RECAP OF 4 STRATEGIES
