Program Review

Steve WilsonDirector of Academic Assessment

UTB Definition

A program review provides the program an opportunity to review, evaluate, and plan. It forms the basis for evaluating the alignment of the program with institutional and college missions, goals and strategic directions..

Program reviews can assist in determining the adequacy of program resources and identifying requirements for improvements in program quality and effectiveness

Program reviews are done every 5 years, based on a schedule prescribed by The THECB

This year BS Biology, BS Math, BS Chemistry, BS Physics, BAT Health Services Tech, BSN Nursing, and BA Government are required to conduct program reviews

• Chairs should appoint an Assessment Liaison (preferably the same Liaison selected for the Program SLO Analysis). Given the large amount of work required to complete a program review, it is recommended that the Assessment Liaison also chair a departmental level assessment committee so that the work is manageable.

Committees Related to Program Review

• School/College Assessment Committee: Each department Assessment Liaison will be part of a college level assessment committee. The Associate Dean will chair the committee. The committee will meet once a month to discuss assessment related activities, including program reviews, to ensure sufficient progress is being made to meet the appropriate deadlines, and to report any issues or problems to the relevant officers, like the Dean of the college, Director of Academic Assessment or the Executive Director of the Office of Institutional Research, Planning and Effectiveness (OIRPE).

The Report

The Review is divided into several sections1. The Review2. Introduction3. THECB Measures4. The Curriculum5. Faculty and Support Staff6. Facilities and Budget7. Quality and Effectiveness8. Summary and Recommendation

The Review

The Review Fields

The Programs Mission Statement

Report all the programs options or tracks

Include an organizational chart of the department, school or college

Introduction

Description

The Introduction should provide a strong description of how the program mission is aligned to the goals of the school/college and/or the institution.

Do not use minimum or vague reflective language when describing the alignment.

Fields

Provide a description of the programs history or background to establish the review context

Describe how the mission of the program supports the mission of the school or college and the mission of the institution

THECB Measures

Description

•The Texas Higher Education Coordinating Board keeps track of data that measure institutional performance, like enrollment counts, fall course completion rates, number of graduates, graduation rates, fall-to-fall persistence rates, and average course size by department. This section should thoroughly describe the trends in the data over the past five years, the program’s barriers and/or limitations that contributed to the performance trends (inclines or declines) and plans to improve THECB performance outcomes. This section should also outline the program’s THECB performance targets for the next five years was well as plans to meet those targets.

Do not use minimum or vague reflective analysis of THECB measures or statements that do not adequately describe plans for improvement and/or targets for success.

Fields

• Reflect on the data: Describe strengths and accomplishments of the program in meeting THECB measures revealed by the data. Also, identify and describe any contributing program limitations or weaknesses where THECB performance outcomes did not meet proper expectations or showed decline.

• Compare THECB measures to peer institutions outcomes.• Plan: Describe how the program of study intends to improve

program effectiveness to meet those THECB performance targets over the next five years.

• Benchmarks: Describe the performance targets (or benchmarks) the program will strive to obtain for each THECB measure for the next five years.

The Curriculum

Description

• Program syllabi and degree requirements should be well defined, share a consistent design, and published. The program should demonstrate opportunities for student engagement in scholarly and service activities. Strategies for helping students achieve on-time graduation should be clearly articulated; including expectations for faculty advising roles. The curriculum should be updated and responsive to student regional needs, trends in the discipline, and/or in response to the knowledge and skill sets required by graduates.

• Do not use minimum or vague reflective analysis of the curriculum or statements that do not adequately describe plans for improvement. Support your analysis with data and examples if possible.

Fields

• Provide a copy of one syllabus per course offered for the last two years

• Provide links or electronic file for student handbooks, websites, etc., that publicize program options and requirements

• Describe any changes or improvements made to program curriculum in the last five years and the rationale for these changes

• Reflect: Evaluate how well semester-by-semester plan are helping achieve on-time graduation

• Plan: Identify what improvements are planned for the program over the next five years (innovation, use of technology, online courses, expansion to offsite campus, etc.).

Faculty and Support Staff

Description

• Thoroughly describe how faculty course evaluations are used for systematic, formal review of instructional effectiveness of all faculty and how they are integrated into professional plans and course level improvements. This section should also clearly describe how all faculty in the program have required credential of SACS COC, the distribution of faculty workloads is equitable and sufficient for ensuring program quality, the number, and the number of support staff is adequate, sufficient full-time faculty are available to support the program’s effectiveness and meeting the needs of program students.

Do not use minimum or vague reflective analysis of the faculty course evaluations use, faculty credentials, distribution of faculty workloads, the number of faculty and support staff. Support your analysis with data and examples if possible.

Fields

Identify the program coordinator

Describe what qualifies this person to be the program coordinator

Identify the roles and responsibilities of the program coordinator

Describe how has the program coordinator has contributed to the success of the program

• List the courseloads of every faculty member. Courseloads are available through the Office of Institutional Research, Planning, and Effectiveness

• Provide a copy of the most current CV for all faculty listed in the courseloads.

• Summarize the extent to which the program has historically used part-time faculty and how this may or may not contribute to the quality of the program

• Justify the extent to which the program has sufficient full-time faculty to accomplish its mission

• Provide a list of department support staff along with his or her position, title, and brief description of duties.

• Summarize how the program staff contribute to the quality and effectiveness of the program

• Describe the distribution of faculty workload including any additional responsibilities such as support services, research, special assignments, etc.

• Identify how the distribution of faculty workload is equitable and how it contributes to the quality of the program

• Give specific examples of how faculty have used course evaluation for improvement at the individual course level for the program as a whole.

Facilities and Budget

Description

Thoroughly describe how the financial, space, and other faculty resources are appropriate to meeting the needs of the program, how the library resources are current, diverse, and sufficient for program students and how the program plans to maintain library resources over the next five year. Also describe how financial support and scholarships that are available to students is sufficient. Demonstrate evidence of positive gains over time to increase financial support for students.

Do not use minimum or vague reflective analysis of the budget, library resources, and financial support for students. Support your analysis with data and examples if possible.

Fields

• Summarize the program’s financial budget. The budget and other financial resources (documents) are available from the Department Chairs. It is recommended that questions pertaining to the sufficiency of the budget and other financial resources be completed in collaboration with the chairs – as this needs to be considered and reflected upon at the program (not department) level.

• Describe the extent to which financial resources are sufficient to support the program

• Describe available work space for all program employees (for example, space dedicated to the program, like office space and labs).

• Explain the extent to which dedicated space for the program is or is not sufficient for maximum effectiveness and efficiency.

• Describe classroom, labs, workrooms, conference rooms, and other institutional facilities assigned to the program and whether the space is or is not sufficient for maximum effectiveness and efficiency.

• Describe whether current library resources are or are not adequate to meet program needs and the efforts to keep them current. Information about the library resources is available from the library. It is recommended that questions pertaining to library recourses be completed in collaboration with the library.

• Describe efforts to keep dedicated program library resources current over the past five years.

• Describe plans for the next five years to maintain current library resources.

• Summarize the distribution of scholarships and financial support given to students in the program. Data on scholarship and financial aid available through the Office of Institutional Research, Planning, and Effectiveness.

• Describe the efforts the program has made to increase financial support to students and the impact these efforts had on program completion rates.

• Identify any opportunities or challenges the program faces with regard to financial, physical, staffing, library, or other resources needed to effectively operate the program.

• Reflect on how the program facilities and budget does or does not support operational efficiency or other learning goals.

Program Quality and Effectiveness

Description

Quality:Thoroughly provide evidence and reflective analysis of the success of current students and graduates, as well as rates of satisfaction with collegiate experiences. Identify implications the data have on program excellence and equality

EffectivenessProvide evidence of systematic and intentional assessment of student learning, including curriculum maps that demonstrate developmental sequences of competency building across the curriculum, clear and sound methodologies for assessing student learning outcomes, proficiency benchmarks for success, results, and improvement plans, and closing the loop.

Do not use minimum or vague reflective analysis of program quality and assessment. Support your analysis with data and examples if possible.

Fields

• Summarize student characteristics from enrollment data provided by the Office of Institutional Research, Planning, and Effectiveness

• Describe admission requirements of the program of study, if applicable

• Describe trends over the last 5 years of prospective students and the actual number admitted to the program from data provided by the Office of Institutional Research, Planning, and Effectiveness

• Summarize student honors, awards, and scholarships over the past 3-5 years, including presentations, publications, exhibits, performances, etc.

• Describe other student related activities and achievements specific to program objectives (service learning, educational experiences, etc.).

• Reflect on the Data, identify implications the data and student characteristics described above have on program excellence and equality.

• Summarize the program SLO assessment reports from the past five years

• Describe whether and to what degree SLOs were achieved and the criteria used to evaluate and score the data. If any SLO data are missing, provide justification where appropriate.

• Identify how the program utilized results specific to planning and implementing improvement to achieve objectives

• Provide curriculum maps and grading rubrics

Summary and Recommendations

Description

Outline priorities and a comprehensive plan for enhancing the quality and effectiveness of the program. Articulate the programs limitations and concerns. Establish well-defined expectations and priorities for the next five years and link those expectations to findings and lessons learned from the program review.

AY 2014-2015 Exception

Because the Merger will take place in 8/2015, developing an action plan is not necessary. However, to wrap UTB assessment up into a nice package, we want departments to write a “lessons learned” section, based on reflections of the 2014-2015 program review findings. The fields will be available on SharePoint Site soon.

What happens when the report is finished

The Internal Program Review Committee: Once completed, the program review reports shall be forwarded to a Review Committee of faculty members external to the college or school. The Review Committee shall review the reports and then meet with individual department Chairs and Assessment Liaisons to report their finding and any recommendations they have to improve the report.

Reponses to the Review Committee: After meeting with the Internal Review Committee, departments will write their responses, which will address the review committees’ recommendations by either changing the report appropriately, or writing a rebuttal.

Administrative Review: The Associate Vice President for Academic Affairs and/or the Dean of Graduate Studies, the college/school dean, and the chair of the Review Committee shall meet together with the Department Chair and Liaison to review the program review and finalize the report.

Done!!

See Calendar Handout