Upload
gary-powers
View
214
Download
0
Embed Size (px)
Citation preview
Using the Voluntary System of Accountability to Measure Learning Outcomes
Issues and Opportunities
AASCU Academic Affairs Summer Meeting
Portland Oregon ▪ July 28-31, 2011
Christine M Keller VSA Executive Director
VOLUNTARY SYSTEM OF ACCOUNTABILITY (VSA)
on the undergraduate student experience through a common web report – the College Portrait.
Initiative by public 4-year universities to supply comparable, transparent information
AASCU & APLUwww.colle
geportrai
ts.org
VSA GOALS• Provide a streamlined college search tool
for prospective students, families, and high school counselors
• Provide a mechanism for public institutions to demonstrate transparency and accountability
• Support institutions in the measurement and reporting of student learning outcomes through original research and by providing a forum for collaboration and exchange
SLO PILOT PROJECT• 2007-08: Directly measure, publicly
report student learning gains at institution level using a common method– Skills: critical thinking, analytic reasoning,
problem solving, written communication
– Tests: CAAP, CLA, ETS Proficiency Profile
– Reported: At, Above, Below Expected
• 2012: 4-year trial period ends
• 2011-12: Evaluate what works, what doesn’t work, next steps
EVALUATION FRAMEWORK• SLO measurement and reporting will
continue to be essential element of VSA
• How can the section be improved to better meet the needs of participating institutions and external stakeholders within the current environment?
EVALUATION ACTIVITIES• Interviews with key internal and
external stakeholders– National Institute for Learning Outcomes
Assessment (NILOA)
• Focus Groups, Conference Sessions• Surveys• Advisory panels, working groups
CLA N=80
ETS PP
N=18
CAAPN=11
• 34% or 109 College Portraits have published SLO results as of July 2011
SLO PUBLISHED ON CP BY TEST
VSA SLO SURVEY• Short survey to gauge progress,
gather feedback from VSA contacts
• ~200 responses in June/July
– 61% institutional research
– 16% academic affairs
– 13% assessment
– 10% other
SURVEY: TEST ADMINISTRATION• 83% administered at least one test
– 23% CAAP– 37% ETS PP– 51% CLA
• 60% posted results on College Portrait
• 92% plan to continue using one of tests
VALUE OF VSA APPROACHHelpful as overall indicator of student progress … powerful stimulus to other more local efforts
Benchmarks to compare skill levels of our students with other campuses
Good faith effort to be transparent
Selected Comments
VSA SLO Survey 7/5/2011
Dovetails with institutional priorities
VALUE OF VSA APPROACHLack of faculty buy in … small sample sizes provide limited value at operational level
Major limitation of VSA, please change this section … add AAC&U rubrics Selected Comments
VSA SLO Survey 7/5/2011
Testing protocol is negative because of cost (cash/staff time) with no useful results
Unfortunately I don’t think many people look at this section
SURVEY: OTHER ASSESSMENTSUsed Effective
AAC&U VALUE Rubrics 39% (52 of 135)
53% (43 of 80)
Electronic Portfolios 68% (105 of 155)
58% (71 of 122)
ETS Major Field Test 61% (90 of 147)
64% (68 of 102)
Professional Licensure/Certification
93% (161 of 173)
87% (135 of 155)
NEXT STEPS• Gather feedback on efficacy of the SLO
pilot study (Summer/Fall 2011)
• Assemble brainstorming group to consider learning outcomes in current context (Fall 2011)
• Convene technical workgroup to evaluate options (early Spring 2012)
• Work with senior advisory panel and VSA Board to recommend next steps and future directions (late Spring 2012)
QUESTIONS & DISCUSSION
• Questions about evaluation plan or current status of VSA/SLO
• Your input on strengths, weaknesses, recommendations for future for student learning measurement and reporting within the VSA
DISCUSSION• What are the benefits of the current
VSA approach to measuring/reporting SLO? – Has it been effective with external
stakeholder groups?
• What are some of the challenges or barriers of the current approach?
• How can the current approach be expanded or modified to better address current needs, challenges, external demands, etc?
MORE INFORMATION
Websites www.voluntarysystem.org
www.collegeportraits.org
Christine Keller, VSA Executive Director [email protected]
Wendell Hall, VSA Assistant Director [email protected]
Elspeth Payne, VSA Project Coordinator [email protected]