Upload
destiny-wilcox
View
45
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Multiple Perspectives on Directing Assessment in Student Affairs. 2008 International Assessment & Retention Conference Scottsdale, AZ June 13, 2008 Presented by:. A. Katherine Busby, PhD Director of Student Affairs Assessment and Planning University of Alabama. Ellen Meents-DeCaigny, PhD - PowerPoint PPT Presentation
Citation preview
Multiple Perspectives on Directing Assessment in
Student Affairs2008 International Assessment & Retention Conference
Scottsdale, AZJune 13, 2008
Presented by:
A. Katherine Busby, PhDDirector of Student Affairs Assessment and PlanningUniversity of Alabama
Ellen Meents-DeCaigny, PhDDirector, Student Affairs Assessment, Research and Communications DePaul University
Overview
In assessment one size does not fit all…
• Describe the context in which we work• Present and discuss 4 scenarios related to
assessment Culture Metrics Learning Outcomes Closing the Loop
• Offer insights based on our contexts and experiences
Student Affairs Assessment at University of Alabama
• Office formed in 2006• Staff includes director and one graduate assistant• Modest, director controlled budget• Director serves as “internal consultant”• Reports to Associate VP• Decentralized assessment across division• Leadership appreciates, but does not dictate
assessment• Active Student Affairs Assessment Council• Emerging partnerships with other assessment
professionals
UA Model of Assessment
Student Affairs Assessment at DePaul University
• Part-time Coordinator position created 2004• Full-time position created 2007 • 1 full-time staff member & 2 graduate
assistants report to Director • Budget has not yet been established • Director reports to VP• Division-wide approach to assessment • Student Affairs Assessment Committee• Strong partnerships with Institutional Planning
and Research and Teaching, Learning and Assessment
• Centralized approach to assessment at institution
DePaul’s Integrated Model of
Assessment
Success Factors: Programs, services & collaborations in support of mission
Department Key Activities Measures:CostMagnitudeSatisfactionLearning Outcomes
Measures: How are students
learning, engaged or involved?Department Assessment Question
Division of Student Affairs Mission
Culture: The Human Element
“Successful assessment is not primarily a question of technical skill but rather one of human will” (Angelo, 1999).
• Establish a support structure • Build shared trust• Build shared motivation• Develop a shared language• Establish key partners in assessment• Develop shared guidelines and
expectations
Building Culture at DePaul
“Assessment Initiatives flounder because they’re headed-up by people who lack the time and clout to accomplish what is necessary” (Cohen and Kotter, 2002).
• Establish a Support Structure:– Chief Student Affairs Officer: Vice President
of Student Affairs– Senior Level Sponsor: Associate Vice
President of Student Development– Assessment “Champion”: Director of
Assessment, Research and Communications
Building Culture at DePaul
• Build Shared Trust: – Hire a knowledgeable, trusted and well-
respected “champion” – Spend time gathering buy-in
• Divisional meetings• Department meetings• Individual meetings
• Build Shared Motivation:– Focus on the benefits of assessment– Meet individuals and departments at their level– Discuss ALL motivations related to assessment
Building Culture at DePaul
• Establish key partners in assessment:– Find institutional partners involved in
assessment and research– Select Assessment Committee members
interested in the initiative
• Develop shared guidelines and expectations:– Present the assessment model– Discuss the logic behind the assessment model– Distribute the model, timeline and report
templates in a timely fashion
Building Culture at DePaul
• Develop a shared language:– A challenging process!– Present terminology related to the
model– Define important terms
• learning outcomes• process vs. outcomes assessment
– Develop data definitions
Culture:What I do can’t be assessed.
Although most student affairs professionals realize that assessment is a part of their work life we still hear the cries of “what I do can’t be assessed.” Sometimes these cries come from someone who does not buy into assessment, but other times these cries come from those who fear the assessment unknown.
A few keys points about culture
• Helps to have a point person who is focused on assessment
• Helps to have VP and upper level leadership supporting the process and serving as the “champion”
• Make the process manageable and provide support for the process
• Demonstrate how the information will be used (and follow through!) and incorporate it into other areas of the division (annual reports)
• Determine which battles to fight
Learning Outcomes:More than just a well phrased
sentence.
Developing and utilizing learning outcomes is easier said than done. Some student affairs professionals cling to their “operational outcomes”, indicators, and measures of satisfaction (which are important in their own way) but fall short when it comes to learning.
Developing Learning Outcomes at UA
Audience
Behavior
Condition
Degree of Achievement (may be omitted)
R. Heinich, M. Molenda, J. Russell, S. Smaldino (2002). Instructional Media and
Technologies for Learning, 7th Edition. Englewood Cliffs: Prentice Hall, Inc.
Developing Learning Outcomes at UA
To write a learning outcome, follow the formula:
Condition Audience Behavior Degree
DePaul’s Outcome Evaluation Tool
Key Activity #1: (Name)
Learning Outcome
Related to key
activity?
Is the outcome meaningful?
What does it tell you about your
program?
How does the department
influence this outcome?
Phrased as an outcome?
Is it measurable?
How?
Suggestions/ Revised Outcome
Keeling and Associates, LLC, 2006
A few key points about learning outcomes
• Shoot, ready, aim OR ready, aim, shoot
• Understand and utilize different types of outcomes
• Embrace learning outcomes as an
iterative process
Metrics: What is the purpose?
Student affairs practitioners often want information particular to their program or office. However, a good assessment plan may call for that same information to be used at the unit/department or division level. Make the most out of your data by using carefully designed metrics.
DePaul’s Use of Metrics
• Metrics collected for departmental and divisional purposes
• Metrics enhance data collected through assessment projects
• Challenges: – Developing common terminology – Developing common methods of
measurement– Reporting metrics in ways that are
useful to external audiences
UA’s Use of Metrics
• Utilizing existing student data
• Beginning to identify metrics across units
A few key points about metrics
• Developing common terminology is key to examine data across the division
• Careful collection of data will allow stories to be told at programmatic level, unit level, and division level
• Appropriately used metrics affords the opportunity to get the right information to the right person for the right reason
Closing the Loop: How do we use the data?
A successful assessment does not mean that the results met or exceeded our expectations. Rather it means that we used the results to inform our work and improve the college experience for our students.
Closing the loop at DePaul
• Provide assistance with data analysis• Help make connections between departments and
across the division• Help connect results to institutional and national
data • Continue to find ways to share results:
Departmental and Divisional Annual Reports Assessment Celebration Presentations in and outside the division
• Share departmental changes that result from assessment
Closing the loop at UA
• Consultation after the analysis
• Distribution of reports to targeted audiences
A few keys points about closing the loop
• Assist staff through interpreting the results (even disappointing results)
• Maintain focus on improvement
• Demonstrate for staff how they can utilize results of national surveys
• Share information in multiple formats
Contact us
Katie BusbyDirector, Student Affairs Assessment and Planning
Box 870116Tuscaloosa, AL 35487-0116
205-348-7939205-348-2591
Ellen Meents-DeCaignyDirector, Student Affairs
Assessment, Research & Communications
25 E. Jackson Blvd, Suite 1400Chicago, IL 60604
312-362-7298312-362-5671