Assessment Management:Software and Surprises

(mostly good)TAIR 2008

Paul Orser, Ph.D.Associate VP for Planning and EffectivenessTexas A&M University-Corpus Christi

Assessment Management Issues We Faced

• We had moved from paper to a relatively primitive digital system

• But, assessment “bookkeeping” chores were still burning up far too much professional staff time

• Our home grown system worked OK for individual units, but was almost impossible to integrate and consolidate

• Difficult to demonstrate how the “sum of the parts” contributed to the “whole” University strategic direction

• Difficult to demonstrate and document linkages between planning, assessment, budgeting, and continuous improvement

• We needed a better way to prepare for SACS off-site compliance review

What we were looking for in an assessment management system

• A system that would streamline the assessment documentation process

• A system that would free our assessment professional staff to serve as assessment coaches and consultants

• A system that was intuitive and easy to use, so that the software did not “get in the way” of the job we needed to do

• A system that used similar logic to our “home grown” approach, so transition and training would be relatively easy

• A system that was web based and imposed little or no additional hardware and software IT load on the campus, so we could manage it ourselves without depending on our already overworked campus “geeks”

• A system we could afford, both to purchase and maintain, and which would help us manage costs given increasing demands for our services (good deeds rarely go unpunished!)

Expectations and Surprises• We knew that WEAVEonline could solve the organizational problem of

managing more than 140 individual unit annual assessment reports• Good surprises:

– Transition was relatively painless (thanks to some “back office” magic and very talented and highly credible assessment professional staff)

– Participation is nearly 100% (currently 146 of 147 units are complete)

– Built-in, step by step, logical structure has helped improve the quality of unit level assessment

• by making the logic of the relationships between mission, outcomes, measures, expected results, findings, and improvement plans explicit

– Units are discovering for themselves the gaps and discontinuities in their thinking, and are becoming much more aware of the differences between assumptions and evidence.

– Administrative continuity value…department chair personnel changes

Could what worked at the unit level also work at the university strategic plan level?

• How could we use WEAVEonline to document and measure progress on the University strategic plan?

• Once we realized that we could set up the university mission and strategic objectives as a “unit” of its own, inside WEAVE, things began to fall into place

• WEAVE does not use conventional planning terminology like “goals” and “strategies,” so must be careful about definitions and have a willingness to adapt

• We used the WEAVE structure to guide a strategic plan review this past summer

Measuring Progress and Documenting Achievement:Facts and Figures AND Painting the Picture…The power of Annual Reports

Built-in section headers need additional institutional instructions for use in order to get consistent reports that discuss the right things in the right places

Issues and Next Steps• Currently WEAVE captures findings electronically, but

does not store the evidence electronically

• WEAVE currently does not have a good way of documenting assessment-based budget requests and actions

• WEAVE currently does not have the capability for “sign-off and comment” on unit-level reports by the higher level adminstrators to whom the units report

• WEAVEonline has assured us that most of these capabilities are in the works, but development on the new version is running significantly behind the original release timetable

http://pie.tamucc.edu/