S

Framework for using Learning Analytics for

Doctoral Program Assessment & Improvement

L. Roxanne Russell | All Rights Reserved

Doctoral Level Program Evaluation

Acknowledge that students are advanced level professionals- typical approaches to measuring student success or performance may not be appropriate Students are expected to succeed Pass/Fail program with substantive faculty feedback

Recognize the assessment value of student expectations and perceptions for their learning experiences in doctoral level graduate programs Credible judges Institutional impact (reputation, alumni, word of mouth)

Fulfilling Student Expectations

Caliber of faculty Originality of perspective- scholarship Personal exposure Charisma/personality

Prestige of institution Quality of environment (facilities/interface/tools) Quality of service & support Caliber of and access to peer network

Challenge of curriculum Relevance and level of content Worthwhile course activities Scaffolded project

Student Success

How are students likely to measure their success?

Self-actualization/Reflective practitioners Career competency/advancement Ability to serve their constituency

Student Attribution

What will students attribute their success in the program to? (perceptions of learning experience) Faculty interactions Peers Curriculum Content Manageability

Design features

Faculty Interacti

on{

Synchronous sessions

Asynchronous interactions

Office Hours

Campus Visits

Project Consultation

Design features

Peers {Group assignments

Breakout groups in synchronous sessions

Synchronous discussions

Asynchronous discussions

Design features

Curriculum

{Sequencing

Track

Advanced

Design features

Content {Multimedia

Reading methods

Lectures

Design features

Manageability

{Weekly structure

Calendar

Navigation

Workload

Project scaffolds

Student Behaviors

Run reports of design feature related behaviors with corresponding learning analytics

Student Perceptions

Student ranks level of perceived success in the program (personal success/learning)

Student ranks fulfillment of expectations for the program

Student ranks how these rankings could be attributed to design feature elements: Ranks level of satisfaction with element Ranks level of importance of element Ranks ease of use of primary tools Open-ended question space to elaborate on experiences with each

element

Analysis

Compare survey and questionnaire responses to actual behaviors Individual student data comparison E.g. student claims time management was a challenge/

student does not use time management design features like the planner/calendar

Conduct mid-term/mid-program learner analysis to offer chance for student reflection and change in behaviors; modification of design

Next

Long-term measures of student success

Faculty perceptions of student success

Framework for evaluating faculty experience/notions of success Preparation & development Course design, scaffolds, tools Facilitation & feedback Learning experience for students Project process