Upload
wcet
View
338
Download
0
Embed Size (px)
Citation preview
Evaluating Online Course Quality for Student Learning and Success
WCET 23rd Annual Conference, Denver, Colorado
• Who are we? Our mission and goals
• Community of Inquiry Framework and evolution at APUS
• Use of data for decision making, continuous improvement, ensuring student
learning and success
• Continuously improving the quality of courses and programs through quality
assurance processes
• Predictive analytics, semantic analyses, PAR framework
• Discussion/comments/questions
OUTLINE
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
WHO WE ARE
American Public University System has been dedicated to educating those who serve since our doors first opened in 1991. The American Public University System (APUS) is a fully online system, encompassing both American Military University and American Public University. We are regionally accredited by the Higher Learning Commission (HLC) of the North Central Association.
“Educating Those Who Serve”
• Founded as American Military University (AMU) in 1991 by James P. Etter, a Marine Corps Officer
• In 2002, AMU expanded into the American Public University System (APUS), adding American Public University (APU)
• Nationally Accredited (DETC) in 1995
• Regionally Accredited (NCA) in 2006
• Serving over 96,000 students worldwide
Mission: To provide quality higher education with emphasis on educating the nation’s military and public service communities by offering respected, relevant, accessible and affordable, student‐focused online programs, which prepare them for service and leadership in a diverse, global society.
STRATEGY:
• Focus on quality and affordability
• Graduate successful alumni who make important contributions to society and their professions
• Enhance America’s competitiveness by meaningfully increasing the population withcollege degrees
• Collaborate with other universities to create solutions to U.S. higher education challenges
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
2011: Celebrating 20 Years of Service and Leadership
5
An educational community of inquiry is a group of individuals who collaboratively engage in purposeful critical discourse and reflection to construct personal meaning and confirm mutual understanding.
The Community of Inquiry theoretical framework represents a process of creating a deep and meaningful (collaborative‐constructivist) learning experience through the development of three interdependent elements ‐social, cognitive and teaching presence.
Over 2 million students across the world have used this validated instrument.
COMMUNITY OF INQUIRY (COI)
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
“It is nothing short of a miracle that modern methods of instruction have not yet entirely strangled the holy curiosity of inquiry” – Albert Einstein
Social presence is “the ability of participants to identify with the community (e.g., course of study), communicate purposefully in a trusting environment, and develop inter‐personal relationships by way of projecting their individual personalities.” (Garrison, 2009)
Teaching Presence is the design, facilitation, and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes (Anderson, Rourke, Garrison, & Archer, 2001).
Cognitive Presence is the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse (Garrison, Anderson, & Archer, 2001).
Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text‐based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2‐3), 87‐105.
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
• Phil – sample of COI report
Data Driven Culture
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
“Quality begins on the inside . . . and then works its way out.” ‐Bob Moawad
• Internal and external benchmarking
• Data tools for stakeholders ‐dashboards, program and course level fact books, data warehouse, student learning assessment reports.
NATIONALLY BENCHMARKED INSTRUMENTS
9
National Survey of Student
Engagement
Proficiency Profile Tests
Major Field Tests
Community of Inquiry Survey
APUS utilizes a number of nationally benchmarked tests and validated instruments to measure student engagement, student satisfaction, and the achievement of student learning outcomes.
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
Direct and Indirect Measures of Assessment
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
“Experts often possess more data than judgment.” – Colin Powell
Regular Dissemination of Data to Stakeholders: Dashboards
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
“In the spirit of science, there really is no such thing as a "failed experiment." Any test that yields valid data is a valid test.” ‐ Adam Savage
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
Regular Dissemination of Data to Stakeholders: Course and Program Level Fact Books
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
Student Learning Assessment Report: Alignment of Student Learning Outcomes
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
Student Learning Assessment Report: Culminating Experience
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
Student Learning Assessment Report: Key Course Embedded Assessments
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
Student Learning Assessment Report: Assessment Data
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
Student Learning Assessment Report: Checklist
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
Student Learning Assessment Report: Changes Based on Data
PROGRAM REVIEW PROCESS
External ReviewExternal Review
AnalysisAnalysis
Program Review Meeting
Program Review Meeting
Three Year Plan
Three Year Plan
Follow‐UpFollow‐Up
Data Collection
Data Collection
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
“Quality is never an accident; it is always the result of high intention, sincere effort, intelligent direction and skillful execution; it represents the wise choice of many”William A. Foster
“One of the great mistakes is to judge policies and programs by their intentions rather than their results.” Milton Friedman
Data Aggregation & Analysis
• Course books
• Electronic resources
• Learning strategies
Library and Learning Resources
• Student learning outcomes
• Instructional strategies
• Evaluation procedures
• Academic rigor
Curriculum Assessment
• Expert reviewer report
• Industry Advisory Council report
External Reviewer Feedback
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
• Analysis of facultycredentials and expertiseto ensure breadth anddiversity
Faculty
• Student demographicinformation
• Enrollment History
• Growth trends
Students
• Curricular Mapping
• Assessment measures
• Fact books
• Student learning assessment reports
• reports
Learning Outcomes Assessment
Data Aggregation & Analysis
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
• Program benchmarkingwith similar programsand institutions
Program Benchmarking
• Evaluation of findings
• Program recommendations
• Three year proposedstrategic plan
Program Directory Summary
• Dean’s observations
• Meeting minutes
Review and Findings
Data Aggregation & Analysis
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE
COURSE REVIEW PROCESS
• Evolution of program review to course review process
• Response to regulation and oversight
• Evaluation of courses as a means to support student success, learning, and
engagement
• Systematic and comprehensive method to identify the strengths and areas for
enhancement or improvement for each course offered by APUS
• Aligned with the Community of Inquiry framework
“Everything can be improved.” ‐ C. W. Barron
“Quality is everyone's responsibility.”‐W. Edwards Deming
• Contact hours calculator ‐ Each class evaluated to ensure it had 45 contact hours of learning and an additional 80 – 100 hours of homework associated with the class for a total of 120+ hours for undergrad classes and 150+ hours for grad classes.
• Contact hour calculator emerged as a result of evaluation in the School of Public Safety and Health
• The template was adopted and refined for implementation across APUS schools. The calculator provides a tool for parties to calculate contact hours while establishing expectations.
ONLINE CONTACT HOURS CALCULATOR
ONLINE CONTACT HOURS CALCULATOR
ONLINE CONTACT HOURS CALCULATOR
ONLINE CONTACT HOURS CALCULATOR
Predictive Modeling
UI / UX
Semantic Analysis
GAP ANALYSIS
GAP ANALYSIS REPORT
Dr. Karan Powell, Provost and Executive Vice‐President
Dr. Phil Ice, Vice‐President, Research and Development
Dr. Jennifer Stephens Helm, Vice‐President, Institutional
Research and Assessment
Feel Free to Contact Us!
AMERICAN PUBLIC UNIVERSITY SYSTEM | EDUCATING THOSE WHO SERVE