View
93
Download
0
Category
Tags:
Preview:
DESCRIPTION
Evaluating the Quality of Online Programs. Diane Ruiz Cairns, Ed.S. Lawrence Technological University eLearning Services. Agenda. Overview Why monitor quality? Methods for monitoring quality of online programs Lawrence Tech, eLearning Services experience Next steps. Overview. - PowerPoint PPT Presentation
Citation preview
Evaluating the Quality of Online
ProgramsDiane Ruiz Cairns, Ed.S.
Lawrence Technological UniversityeLearning Services
Agenda
• Overview• Why monitor quality?• Methods for monitoring quality of online programs• Lawrence Tech, eLearning Services experience• Next steps
Overview
• Program Evaluation• Making data-driven decisions• Support of performance improvement• Alignment of resources, performance and strategic goals• Add measurable value
Overview
• Quality of online programs impacts student retention, enrollment and graduation rates
• Online environment includes:• Institutional Technology• Course Development• Course Structure• Teaching and Learning• Student and Faculty support• Methods of Evaluation and Assessment
Overview
• Measures of success include:• Enrollment Reports• Assessment of Learning• Student Evaluation Survey• Informal Feedback
Why Monitor Quality?
• Results based• Measurable results• Effective course content• Efficiencies in operation• Teaching effectiveness
Why Monitor Quality?
• Alignment of methods for measuring and assurance of quality• Stability of online programs• Impacting student satisfaction• Value of online programs
Evaluating Quality of Online Programs
• Views and methods for evaluating programs varies• Adoption of a comprehensive tool or methods brings
alignment• A validated tool recognized by industry can assist with
reliability• Requirements before adopting a tool
Evaluating Quality of Online Programs
• Repeatable data collection results in meaningful collection of data• Comprehensive approach• Multiple collection cycles supportive of reasonable and
responsible data
Evaluating Quality of Online Programs
• Requires careful planning• Data collection focus on mega, macro, micro levels• Systems approach• Data collected include:• Effective course content• Efficiencies in operation• Teaching effectiveness
Evaluating Quality of Online Programs• Mega
• Success of online program at meeting university enrollment goals• Support of teaching and learning goals
• Macro• Technological infrastructure• Individual courses support of teaching and learning guidelines• Faculty engagement
• Micro• Instructional design impact• Student, Faculty, Staff use of technology• Student and faculty support services
Evaluating Quality of Online Programs
• Planning• Timeline• Participation• Communication, Communication, Communication• Conduct the evaluation• Plan for interventions
Monitoring of Online Program
• Create Dashboard• Seven to Ten Metrics
• Method for reporting (communicating)• Data collection periods
Example of Data Collection Schedule
Building a Dashboard
• Elements of organizational performance• Enrollment Goals• Teaching and Learning Goals• Graduation rates• Employment outcomes• Technological metrics: uptime, type of support calls• Quality of teaching and learning• Faculty engagement• Faculty training, participation• Student evaluation survey data
Dashboard Examples
Lawrence Tech Experience• Sloan-C
• Evaluation of Online Program organization• Self assessment
• Baldrige Education Performance Excellence• Evaluation of education organization• Assessed by Baldrige evaluators
• Blackboard Exemplary Course Rubric• Evaluation of course development• Self assessment
• Quality Matters• Evaluation of course development• Assessed by qualified evaluators
Lawrence Tech Experience• Operation Quality• Course Quality• Course Delivery Quality• Documenting standards• Identify metric requirements• Adopting industry standards:• Sloan-C• Blackboard Exemplary Course Rubric• QM Course Design• Baldrige - future
Getting Started
• Why do this?• What will you do with the data?• Benchmarking• Building team• Confirming plan• Collecting data, what data• Reporting results• Engagement across campus services
Confirming
• Monitoring schedule• Reinforcement of quality measures• Integration • Policy and practices of monitor, evaluating, assessing• Managing, planning for change• Oversight
Change
• Be an agent of change• Lens of student, employers, accrediting bodies, stakeholders• Define critical success practices
Timeline
Conclusion
• Confirm metrics• Begin program evaluation• Sloan-C
• Develop Dashboard• Report• Refine• Apply intervention
Dashboard Data
Dashboard Data
Discussion
There is nothing wrong with change, if it is in the right direction.-- Winston Churchill
References
• Cokins, G. (2008, April 3). How are balanced scorecards and dashboards different? Information Management.com. Retrieved April 12, 2014, from http://www.information-management.com/news/10001076-1.html?zkPrintable=true
• Cowan, K. (2013, December 15). Higher education’s higher accountability. Accreditation and Standards, Winter(2014). Retrieved from http://www.acenet.edu/the-presidency/columns-and-features/Pages/Higher-Education%27s-Higher-Accountability.aspx
• Dessinger, J. C. & Moseley, J. L. (2004). Confirmative evaluation: Practical strategies for valuing continuous improvement. San Francisco, CA: John Wiley & Sons, Inc.• Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2004). Program Evaluation: Alternative approaches and practical guidelines. Boston, MA: Pearson Education.• Frigo, M. (2012). The balanced scorecard: 20 years and counting. Strategic Finance, p. 49-53.• Griggs, V., Blackburn, M., & Smith, J. (2012). The educational scorecard: The start of our journey. The Electronic Journal of Business Research Methods, 10(2), 121-131.• Guerra-López, I. (2007). Evaluating impact: Evaluation and continual improvement for performance improvement practitioners. Amherst, MA, HRD Press, Inc.• Hell, M., Vidačić, S., & Garača, Ž. (2009). Methodological approach to strategic performance optimization. Management, 14(2), 21-42.• Hughes, K. E., & Pate, G. R. (2013). Moving beyond student ratings: A balanced scorecard approach for evaluating teaching performance. American Accounting
Association, 28(1), 49-75.• Kaufman, R., Gurerra, I., & Platt,W. A. (2006). Practical evaluation for educators: Finding what works and what doesn’t. Thousand Oaks, CA: Corwin Press.• Kaufman, R., Oakley-Browne, H., Watkins, R., & Leigh, D. (2003). Strategic planning for success: Aligning people, performance, and payoffs. San Francisco, CA: Josey-
Bass/Pfeiffer.• Kesler, G., & Kates, A. (2011). Leading organization design: How to make organization design decisions to drive the results you want. San Francisco, CA: Josey-Bass.• Laureate Education, Inc. (Producer). (2011a). Assessment and accountability in education: Dashboards, part 1. Baltimore, MD: Author.• Laureate Education, Inc. (Producer). (2011b). Assessment and accountability in education: Dashboards, part 2. Baltimore, MD: Author.• Popham, W. J. (2008). Transformative assessment. Alexandria, VA: Association for Supervision and Curriculum Development.• Shelton, K. (2010). A quality scorecard for the administration of online education programs: A Delphi study. Journal of Asynchronous Learning Networks, 14(4), 36-62.• Shelton, K., & Saltsman, G. (2005). An administrator’s guide to online education. USDLA Book Series on Distance Learning.• The Sloan Consortium (2012). Changing course: Ten years of tracking online education in the United States (2013). Babson Survey Research Group and Quahog
Research Group. Retrieved from http://sloanconsortium.org/publications/survey/changing_course_2012• U.S. Department of Education, NCES (2011, October 5). Learning at a distance: Undergraduate enrollment in distance education courses and degree programs v. 154.
NCES: Author. Retrieved from http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2012154• United States Government Accountability Office. Higher Education (2011). Use of new data could help improve oversight of distance education. (GAO-12-39).
Retrieved from Retrieved from http://www.gao.gov/assets/590/586340.pdf• U. S. News World Report (2014, January 7). Online education. Retrieved from http://www.usnews.com/education/online-education
Recommended