Upload
others
View
11
Download
0
Embed Size (px)
Citation preview
Annual Report on Annual Report on Institutional Effectiveness:Institutional Effectiveness:Pathway to ExcellencePathway to Excellence
Contributing ColleaguesDr. Julia Pet-Armacost, IAA
Dr. Taylor Ellis, UACDr. Michael Johnson, UAC
Dr. Paula Krist, OEASDr. Angela Albert, OEAS
Uday Nair, OEAS
August 30, 2004
UAC Annual Report 2August 30, 2004
Overview
institutional effectiveness at UCFenhancements made this yearstatus report on program assessmentnational visibilitynext steps
UAC Annual Report 3August 30, 2004
UCF’s Integrated Approach to Institutional Effectiveness
ProgramAssessment
Unit and Program Reviews
StrategicPlanning
linkages•share information•inform budget process
differences•different cycles•additional data elements•different purposes
•continuous improvement•evaluation•planning
UAC Annual Report 4August 30, 2004
Program Assessmentformative evaluation process designed to support improvementcontinuous
focused on improvingstudent learningstudent developmentservices and operations
Do
Act
Plan Check
UAC Annual Report 5August 30, 2004
Brief History of Assessmentat UCF
SACS reaffirmation visit, little evidence of assessment
Nichol’s modelIs adopted
Programs asked to conduct assessment
annually and submit for review every three years
OEAS Office isestablished and
President announcesannual review cycle
Web-based submission and review, Divisional Review Committees,
Web access to surveys
UAC is established and SACS accepts
UCF report
1995 1996 1997 19981998 19991999 20002000 20012001 20022002 20032003 20042004
UAC Annual Report 6August 30, 2004
This Year’s Enhancements
one-time submissionassessment handbooksUAC & DRC policies and proceduresenhanced focus on student learning outcomesnew partnership with FCTLNSSE and FSSEenhanced access to surveys
UAC Annual Report 7August 30, 2004
Status Reportsubmissionsmethods of assessmentstudent learning outcomeschanges made on the basis of assessment
academic programsadministrative units
UAC Annual Report 8August 30, 2004
Number of Programs and Units Conducting Assessment
202
254278 296 310
0
50
100
150
200
250
300
350
2000 2001 2002 2003 2004
Year
UAC Annual Report 9August 30, 2004
Submission Rates
Submission of Assessment Plans
81.0%92.0% 97.5% 99.0%
98.3%
0%
20%
40%
60%
80%
100%
1999 2000 2001 2002 2003
Year
Subm
issi
on R
ate
UAC Annual Report 10August 30, 2004
Assessment Tools Used in 2004
3.3%
11.3%
12.7%
18.3%
21.1%
21.1%
22.1%
61.5%
0% 10% 20% 30% 40% 50% 60% 70%
Scoring Essays
Other
Capstone Project or Performance Evaluations
Standardized (state normed) Exams
Standardized (nationally normed) Exams
Evaluation of Portfolios
Other Evaluations Using Scoring Rubrics
Locally Developed Tests
Percent UsageAcademic Administrative
UAC Annual Report 11August 30, 2004
Surveys Used in 2004
0.0%
9.4%
11.3%
55.4%
57.3%
0% 10% 20% 30% 40% 50% 60% 70%
National Surveys
Employer Surveys
Alumni Surveys
Other Locally (program or unit)Developed Surveys
UCF's Graduating Student Surveys
Percent UsageAcademic Administrative
UAC Annual Report 12August 30, 2004
Percent of Programs with 1 to 9 Program Outcomes
74.4%
37.5%
70.0%
7.1%
30.8%
52.3%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
CAS CBA CED CECS COHPA UNIVERSITY
1 2 3 4 5 6 7 8 9
UAC Annual Report 13August 30, 2004
Percent of Undergraduate Programs Addressing Particular Student Learning Outcomes
0%10%20%30%40%50%60%70%80%90%
100%
Discipline Knowledge,Skills, Attitudes, &
Behaviors
Communication Skills Critical Thinking Skills
CAS CBA CED CECS COHPA
UAC Annual Report 14August 30, 2004
Percent of Undergraduate Programs Addressing 0,1,2, or 3 Student Learning Outcomes (SLOs)
0% 20% 40% 60% 80% 100%
CAS
CBA
CED
CECS
COHPA
None 1 SLO addressed 2 SLOs addressed 3 SLOs addressed
UAC Annual Report 15August 30, 2004
Curricular Changes Made to Academic Programs
26%
24%
23%
12%
0% 10% 20% 30% 40%
Revise and/or EnforcePrerequisites or Course
Sequence
Revise Course Content
Add Course(s)
Delete Course(s)
2003-04 2002-03
UAC Annual Report 16August 30, 2004
Process Changes Made to Academic Programs
15%
13%
16%
6%
12%
9%
0% 10% 20% 30% 40% 50%
Modify Frequency or Schedule ofCourse Offerings
Made Technology RelatedImprovements
Made Personnel Related Changes
Implement Additional Training
Revise Advising Standards orProcesses
Revise Admission Criteria
2003-04 2002-03
UAC Annual Report 17August 30, 2004
Changes Made by Academic Programs to Assessment Plans
21%
28%
18%
26%
0% 20% 40% 60% 80% 100%
Revise StudentOutcome Statement(s)
Revise MeasurementApproach(es)
Collect and AnalyzeAdditional Data and
Information
Change Method(s) ofData Collection
2003-04 2002-03
UAC Annual Report 18August 30, 2004
Process Changes Made by Administrative Units
43%
57%
36%
43%
36%
9%
21%
0% 10% 20% 30% 40% 50% 60% 70%
Revamp Services
Make Technology RelatedImprovements
Make Personnel Related Changes
Implement Additional Training
Add New Service(s)
Delete Service(s)
Other Change(s)
2003-04 2002-03
UAC Annual Report 19August 30, 2004
Changes Made by Administrative Units to Assessment Plans
40%
43%
23%
37%
0% 10% 20% 30% 40% 50%
RestructureObjective
Statement(s)
Revise MeasurementApproach(es)
Collect and AnalyzeAdditional Data and
Information
Change Method(s) ofData Collection
2003-04 2002-03
UAC Annual Report 20August 30, 2004
Examples of Quality Improvements (Academic Programs)
implemented curricular changes that enhanced students’ writing skills (CAS, Chemistry, BA)
increased course offerings; also increased student enrollment (COHPA, Health Information Management Program, BS)
achieved 100% pass rate among program completers on the Florida Teacher Certification Exam in this specialty; also activated graduate certificate program (CED, Instructional Technology/Media, Educational Media, MEd (Online)
UAC Annual Report 21August 30, 2004
Examples of Quality Improvements (Academic Programs)
making changes to improve critical thinking, problem solving, presentation, and writing skills(CECS Industrial Engineering and Management Systems)
intensified effort on developing student learning outcomes; also implemented balanced scorecard that resulted in creation of stronger collaborative environment (CBA, Academic Programs & Undergraduate Student Services)
UAC Annual Report 22August 30, 2004
Examples of Quality Improvements(Administrative Units)
developed web application that improves access and process turn-around time (A&F: Facilities Planning)
improved training and customer service (SDES: Student Academic Resource Center)
enhanced faculty perception of the value of diversity in the curriculum (President’s Division: Diversity Initiatives)
enhanced capability to provide services (Academic Affairs: University Analysis and Planning Support)
increased client base, quality, awareness, and involvement (Sponsored Research: Technology Incubator)
UAC Annual Report 23August 30, 2004
Examples of Quality Improvements(Administrative Units)
shifted focus to outcomes related to increasing number and types of employers recruiting UCF students (SDES: Career Resource Center)
focused on new ways of educating clients by offering workshops on how to use services; also enhanced communication with clients (SDES: Student Union)
increased student satisfaction regarding safety in their living environments (SDES: Housing)
UAC Annual Report 24August 30, 2004
Increasing Our National Prominence in Assessment
conference presentations 28 presentations (15 last year)
1 invited plenary1 invited paper1 selected as “best paper”
at 18 different conferences (7 last year)primarily by 3 colleges and IAA staff
workshops presented6 workshops (5 last year)
1 paper published in J. of General Education
UAC Annual Report 25August 30, 2004
28 Conference Presentations
AACTE (1)AACSB International(1)AACSB Undergraduate (1)AAHE Assessment (1)AIR (4)FAIR (3)FATE (1)FIE (1)LiveText (2)
NCCI (1)NCSU Symposium (1) invitedSACCR (1) invited plenarySBA(1)SACS (2)SAIR (3)SCUP National (1)SCUP Regional (1): selected as best paperSESUG (2)
UAC Annual Report 26August 30, 2004
Workshops Conducted
This Year = 6AIR (2) BAP VI (1) invitedSACS (2)SAIR (1)
Accepted for Next YearSAIR (1)SACS (2)
UAC Annual Report 27August 30, 2004
External Consultations andSite Visits
Consultations and AssistanceOur Lady of the LakeNorthwestern State UniversityEmbry RiddleDBCCUSFFAUBOG and DCU
SACS Visiting Team 2004Clayton College
Planned ConsultationsFAMU UNF UWF
SACS Visiting Team 2005Texas Tech
UAC Annual Report 28August 30, 2004
Planned Enhancements for Next Year
develop stronger partnership with FCTLincrease emphasis on learning outcomes in assessment plansimplement Academic Learning Compactsrevise alumni survey with program-level dataimplement recognition for best practicesexpand national reputation
increased participation in conference presentationspeer reviewed assessment-related publications
UAC Annual Report 29August 30, 2004
UCF is Moving Toward Excellence!
SACS reaffirmation visit, little evidence of assessment
Nichol’s modelIs adopted
Programs asked to conduct assessment
annually and submit for review every three years
OEAS Office isestablished and
President announcesannual review cycle
Web-based submission and review, Divisional Review Committees,
Web access to surveys
UAC is established and SACS accepts
UCF report
1995 1996 1997 19981998 19991999 20002000 20012001 20022002 20032003 20042004
Focus on student learning
Partner w/FCTL
Annual Report on Annual Report on Institutional Effectiveness:Institutional Effectiveness:Pathway to ExcellencePathway to Excellence
Contributing ColleaguesDr. Julia Pet-Armacost, IAA
Dr. Taylor Ellis, UACDr. Michael Johnson, UAC
Dr. Paula Krist, OEASDr. Angela Albert, OEAS
Uday Nair, OEAS
August 30, 2004