Upload
allen-griffith
View
218
Download
1
Tags:
Embed Size (px)
Citation preview
Closing the Loop Standards 7 and 14
R. Dean Foster, Ph.D.Director of Assessment
Joseph M. Thomas, Ph.D.Dean, Undergraduate Studies
Brian Buerke, Ph.D. Associate Professor of Physics
Albright College
• Small Private (1,650 traditional students)• Liberal Arts Mission• Interdisciplinary Emphasis• Primarily Residential• Reading PA (medium-sized urban setting)• Strong Faculty Governance
Goals of Presentation
Participants will • Better understand concept of Closing
the Loop• Be aware of several examples of Closing
the Loop onStandard 7Standard 14
Closing the Loop DefinedUsing assessment results to • Plan improvements• Advocate for allocation or reallocation of
budget and other resources (when those resources are needed to implement improvements)
• Implement improvements• Re-assess results (data and information)
Closing the Loop on Standard 7 Institutional Effectiveness
•Health Services•IT Support•Academic Support Services•Facilities - Classrooms and Labs
Closing the Loop on Standard 7 Institutional Effectiveness
Data Sources
• Higher Education Data Sharing (HEDS) Senior Survey • Association of Independent Colleges and
Universities of Pennsylvania End of First Year (Student) Survey (AICUP)
• National Survey of Student Engagement (NSSE)
Closing the Loop on Standard 7 Institutional Effectiveness
Initial Data
Selected areas of low student satisfaction–Health Services < 70% . . 2.7/5– IT Support < 80% . . 2.7/5–Academic Support . . < 3.0/5– Facilities – Classrooms and Labs . . 2.7/5
Qualitative feedback from students and faculty reinforced these concerns.
Closing the Loop on Standard 7
Health ServicesRatings < 70% . . 2.7/5 - Closing the Loop byDeveloped/administered the Gable Health Center
Satisfaction Survey. Student-users identified overall high satisfaction. Lower rating on location, hours.
Based on results, the College relocated and improved the Health Center, changed service hours, changed intake process. 100% satisfaction on location.
National survey results still not at benchmark, assuming non-user ratings, will survey broader student body Spring 2011.
Closing the Loop on Standard 7
IT Services
Ratings < 80% . . 2.7/5 - Closed the Loop byFollow-up surveys with students and discussions with
faculty identified areas of dissatisfaction which included
• Lack of wireless capability in residence halls.• Faculty concerns with technology support in
classrooms.
Closing the Loop on Standard 7
IT ServicesClosed the Loop by• $3/4 million for wireless residence halls and Campus
Center.• Move IT under provost. Classroom support was
restructured and improved. • $1 million over 5 years to renovate classrooms.Student satisfaction increased to 2.9-3.2 in 2009.Faculty acknowledged improvements.
Closing the Loop on Standard 7
Academic SupportRating < 3.0/5 - Closing the Loop by• Increased from 1 FT to 4.5 staff• Achieved CRLA certification of tutoring• Expanded skills workshops, tutoring times, and study
groups• Added services for students on academic probation• Improved disability servicesChanges too recent to be reflected in national surveys BUT . . .
Closing the Loop on Standard 7Impact also on Standard 14
Academic Support
Results of Increased Services for Students on Academic Probation (59 students)
• 27 fulfilled all requirements - 98% increased term GPA; 50% increased cumulative GPA
• 21 fulfilled some requirements - 60% increased term GPA; 21% increased cumulative GPA
• 13 fulfilled no requirements - 6% increased term GPA; no increase in cumulative GPA.
AND . . .
Closing the Loop on Standard 7Impact also on Standard 14
Academic Support
Retention of “at risk” students increased from • 32% in AY 2007-08 to • 56% in AY 2009 - 10
Closing the Loop on Standard 7
Facilities – Classrooms and Labs
Ratings < 70% . . 2.7/5 - Closing the Loop by• Completed a Campus Master Plan (facilities).• Making improvements in classroom technology as
already reported above.• Allocating $27 million for an addition to and
renovation of the Science Building.• Continuing to receive gifts (recent $5 million) to
support Strategic and Master Plans.• Continuing to implement the Master Plan
Closing the Loop on Standards 7 and 14
APR’s Cross-College Connections
APR and resulting academic improvements connect:
• Most institutional operations and goals• MSCHE Standards – 7, 11, 12, 14• Departmental planning to institutional
planning
Closing the Loop on Standards 7 and 14
APR Defined• Joint administrative-faculty process• Periodic (5-7 years), spans 2 years• Formalizes self-reflection • Department’s bully pulpit to describe
activities and needs• Vehicle for administration and other faculty
to evaluate support of mission
Closing the Loop on Standards 7 and 14
APR Process Overview
1. Self-study: analyzes data about faculty, curriculum, students, and resources
2. Site visit: provides analysis and consultation from experienced external peers given access
3. Action plan: specifies improvements in context of institutional goals and resources
All final documents available to Faculty and Board.
Closing the Loop on Standards 7 and 14
Sample APR Results
• BIO retention: + 8% from smaller sections, revised labs, formative assessments
• HIS curriculum gap: add World History• SOC rigor: add pre-reqs, theory requirement,
and quantitative capstone• Theater staffing, equipment, and facilities:
upgrades and planning cycle underway
Closing the Loop on Standards 7 and 14
APR Detail: First-Year Seminar
• Data: Low satisfaction (rated 2.3/5)• Analysis: Lack academic focus; non-credit• Responses: o2006-8 topical focus (rated 3.4/5)o2009 full credit seminar (rated 3.9/5)
• Faculty approval + Provost budget
Closing the Loop on Standards 7 and 14
APR Detail: Math Retention
• Data: Low grades in pre-Calc and Calc I• Analysis: Initial student preparation;
academic support; variable sections• Responses: placement; remedial course/SI;
tutor training; cross-section standards• Faculty approval + resources through
Provost’s budget and staffing cycle
Closing the Loop on Standards 7 and 14
APR Detail: Math Content and Mission
• Data: Curriculum gap; senior info?; low # of co-concentrators
• Analysis: Need “Real Analysis” course; more info on senior satisfaction and competency; more collaboration with other departments
• Responses: Create and require course; implement ETS test and senior survey; create new track with Business appeal
• Faculty approval + Provost’s budget cycle
Closing the Loop for Student Learning Outcomes (Standard 14)
• Course levelModern Astronomy (PHY 102)
• Conceptual learning
• Objective assessmentImpact of changes
Closing the Loop (Standard 14): Course
PHY 102 is a survey lab course with a standard syllabus
- Introduction/ancient astronomy - Planetary motion and gravity- Solar system - Characteristics and evolution of stars- Stellar explosions and remnants- Structure and evolution of the
Universe
Closing the Loop (Standard 14): Course
Lab includes project-based and observational activities
- Building a quadrant - Galileo’s gravity experiment - Building a telescope - Observing the sun - Nighttime viewings
A team of students tests the quadrant they designed and built
Closing the Loop (Standard 14): Conceptual Learning
One student learning outcome is to improve conceptual understanding
• Provide framework for deep learning
• Misconceptions- are common- hard to overcome- prevent effective learning
Closing the Loop (Standard 14): Objective Assessment
Conceptual knowledge is measured by a concept test (LST)1
• Distractor based• Nationally validated• Given pre- and post-semester• Direct measure of knowledge gains (g)
post preg =
100% pre
1. Alan Lightman and Philip Sadler, The Physics Teacher 31, 162 – 167 (1993).
Closing the Loop (Standard 14)
Dissatisfaction with conceptual learning inspired changes
• Fall 2000: Average gain (g) = 0.19
• Change to interactive lecture notes- improve student focus- reinforce mental engagement- increase opportunities for discussion
Closing the Loop (Standard 14): Assessment of Changes
Gains doubled after changes
0 1 2 3 4 5 6 7 8 90
0.1
0.2
0.3
0.4
0.5
0.6
LST concept test
Semesters since spring 2000
Aver
age
norm
aliz
ed g
ain
Interactive lecture notes introduced
Avg. Gain(7 semesters):
g = 0.44 ± 0.05
Closing the Loop Standards 7 and 14
R. Dean Foster, Ph.D.Director of Assessment
Joseph M. Thomas, Ph.D.Dean, Undergraduate Studies
Brian Buerke, Ph.D. Associate Professor of Physics