Upload
letitia-manning
View
221
Download
2
Tags:
Embed Size (px)
Citation preview
Cyclical Program Review2014-2015
John ShepherdVice-Provost and Associate Vice-President (Academic)
Office of Quality Assurance
The Context – QA in Ontario
Ontario Universities’ Council on Quality Assurance (Quality Council) was established July 1, 2010
No more OCGS appraisals; No more UPR
The new regime covers the approval and review of all new and existing undergraduate and graduate programs.
The Context – QA in Ontario
Each university in Ontario has been required to develop its own Institutional Quality Assurance Process (IQAP) for ratification by the Quality Council.
Carleton’s Institutional Quality Assurance Process (IQAP):│ Approved by Senate June, 2010│ Ratified by Quality Council March, 2011│ Revised to include DUC – Approved by Senate February, 2012│ Ratified by Quality Council May, 2012│ Joint Carleton-University of Ottawa IQAP approved by Senate January, 2012│ Ratified by Quality Council May, 2012
QA@Carleton – What’s the point?
• To constantly improve programs that are already of good quality
• To have units feel the exercise was worthwhile and beneficial
• To constantly strengthen Carleton’s academic planning processes
• To help position Carleton advantageously in the changing Provincial context
The Carleton Process:Who Administers It?
Carleton’s Office of Quality Assurance (Academic Programs) (OQA):| Vice-Provost| Manager| Quality Assurance Officer| Program Review Co-ordinator
The Carleton University Committee on Quality Assurance (CUCQA):| Vice-Provost (Chair)| Provost (ex officio)| Associate Dean (Programs and Awards), FGPA (ex officio)| Associate University Librarian| 7 senior faculty members| 1 Dean
OQA and CUCQA – What’s Our Objective?
Quality assurance can have two purposes:| Program improvement| Accountability
Our objective is program improvement We work with programs to support them We are not the QA police!!
Not just an exam to be passed and forgotten!
Cyclical Program Review
Cyclical Program Review should not be approached as a hurdle to be overcome.
Rather, it should be seen as an opportunity to strengthen programs that are currently of good quality or can achieve good quality as part of an
overall process of continuous improvement.
Cyclical Program Review
Occurs on an 8 year cycle
Simultaneous review of graduate and undergraduate programs
Self-assessment of a program’s strengths and areas for improvement
Consultative – informed by input from faculty, staff, students, associate dean(s), dean(s), and external reviewers
20-month process
Cyclical Program Review - Steps
Self-Study (3 volumes)
Site Visit
Reviewers’ Report
Final Assessment Report
Action Plan
OQA – Here to Help
Our objective is program improvement – we are here to help
Customized template for program reviewCoordination of data, reports and other materials to inform program reviewCompilation of data into the required tables. Customized data and reports may be available upon request. Electronic document sharing site - cuCollabMeetings with Review Team membersReview full and partial drafts of the self-study
Require other assistance? Just ask!
Volume I: The Self-Study
Critical, self-reflective, and program-centricProgram history – response from the last reviewProgram structure and deliveryDegree Level Expectations and Learning Outcomes and Learning ObjectivesStudents: Enrolment, retention, satisfactionFaculty, staff resourcesProgram improvements
Volume I: Template
Each review team will have an electronic workspace on cuCollab for document sharing
| OQA will post the template, data, reports, etc. on this site.| Review teams will also be able to post documents
Each Review Team will be provided with a customized template for Cyclical Program Review.
Model tables are included in the template| Data from OIRP and CURO will be provided. Tables will be populated by
OQA. | Customized tables can also be generated.
How to Write a Bad Self-Study
Ineffective self-studies are:| Descriptive rather than reflective, analytical, self-critical, and evaluative| Defensive or aimed at justifying the status quo| Focused on the academic unit rather than the program(s)| Does not address or only superficially addresses Degree Level Expectations
and Learning Outcomes| Raw data are attached as appendices, or only used in a descriptive manner| Written by a single faculty member without evidence of buy-in of faculty
and students
Adapted from: Quality Assurance Framework: Creating an Effective Self-Study for Program Reviews
Writing an Effective Self-Study
Effective Self-Studies are:| Reflective, analytical, self-critical, and evaluative| Aimed at quality improvement. Self-appraisal includes an analysis of strengths and
weaknesses, and outlines how improvements can be made| Focused on the programs under review| Curriculum is fully examined, with an eye to Degree Level Expectations, Learning
Outcomes, and to change and improvement.| Expresses Degree Level Expectations and learning objectives that operationally
drive admission requirements, curriculum content, modes of delivery, bases of evaluation of student performance and commitment of resources.
| Data are analyzed and contribute to the evaluation of strengths and weaknesses of the program.
Adapted from: Quality Assurance Framework: Creating an Effective Self-Study for Program Reviews
Participant Experiences
Dr. Stephen Godfrey
Department of Physics
Degree Level Expectations and Learning Outcomes
Degree Level Expectations
➊ Depth and breadth of knowledge ➋ Undergraduate: Knowledge of methodologies
Graduate: Research and scholarship ➌ Application of knowledge ➍ Communications skills ➎ Awareness of the limits of knowledge ➏ Autonomy and professional capacity
COU Report: Ensuring the Value of University Degrees in Ontario - November 2011
Learning Objectives and Outcomes
Learning Objectives| Learning Objectives are the instructional goals of the program.
They outline the program inputs or learning opportunities students will have over the course of the program
Learning Outcomes│ Learning Outcomes are the measurable skills, knowledge,
competencies and/or behaviours that students will have as a result of successfully completing the program
Learning Objectives and Outcomes
For QA, program-level learning outcomes apply to programs not to individual courses.
We need to look at the individual courses to get a “picture” of how the learning outcomes are addressed within the program.
Model Tables enable an account of how program level learning outcomes are achieved and Degree Level Expectations are met.
A summative narrative is required of how courses as a collective achieve the intended learning outcomes.
Support will be provided in evaluating learning outcomes.
Volume II: Faculty CVs
The CVs of all faculty associated with the program need to be included.
Flexibility in format: No more OCGS template!
All CVs must be in same format.| Format to be approved by OQA.
Volume III: External Reviewers
10 external academic reviewers.
(4 external professional reviewers.)
An internal reviewer will be nominated by OQA, in consultation with the unit and Dean(s)
Timeline & Milestones
September 2013Review team is established
October 2013Attend OQA workshopReview team begins developing student survey/focus group questions (template provided)Review team confirms faculty/instructor information with OQAReview team should begin work on Sections A-D of self-study, with particular focus on the development of program learning objectives/outcomesOQA conducts student surveys/focus groups as applicable
November/December 2013Review team receives library reports, research funding tables, space management report, and survey/focus group reports
Timeline & Milestones
January 2014Review team receives data tables from OIRP
February/March 2014Review team should be finalizing the self-studyFaculty, students, associate dean(s), dean(s), should be consulted on drafts
It is highly recommended that OQA review drafts of the brief and provide feedback before it is submitted.
April 1, 2014Volumes I, II, and III submitted to OQA
Timeline & Milestones
Once OQA confirms that the brief is ready, it is submitted to the Carleton University Committee on Quality Assurance (CUCQA):| Assigned a discussant, who prepares a report to CUCQA| Program Review Lead is invited to attend the CUCQA meeting at
which the program is discussed| Supplementary questions (dean consulted)| Review Committee selected (dean consulted)
Timeline & Milestones
Summer 2014In consultation with the unit, OQA will arrange the site visit for fall 2014.
Fall 2014Two day site visit: external reviewers meet with dean(s), faculty, staff, and studentsWithin one month of the site visit, the external reviewers submit a report to OQAThe Review team prepares a response to the report, in consultation with the dean(s)CUCQA receives the external reviewers report and the Review team ’s response.
Timeline & Milestones
CUCQA considers the brief, report and response:| Discussant recommendation report| CUCQA recommends a categorization:
• Good Quality with international or national presence• Good Quality• Good Quality with report• Conditional approval• Not approved to continue
│ Categorization sent to unit and dean(s)
Action plan requested
Timeline & Milestones
Winter 2015Action plan submitted to CUCQACUCQA: Final Assessment Report and Executive Summary
│ Submitted to the ProvostProvost considers Report and SummaryDocumentation to SAPC/Senate for approval
academic unit; dean(s); Board of Governors; Quality Council; Carleton website
Participant Experiences
Dr. Mitchell Frank
Institute of Comparative Studies in Art and Culture
Help!?
Office of Quality Assurance| Office of Institutional Research and Planning| Carleton University Research Office| Office of Space Management and Capital Planning| University Library
Undergraduate Programs: Faculty undergraduate associate deans
Graduate Programs: Associate Dean (Programs and Awards), FGPA
Contact
Office of Quality Assurance421 Tory Building
www.carleton.ca/oqa
Jessica DeVries, Manager (Interim) & Program Review [email protected] x 3231
Ann Clarke-Okah (OQA Consultant)[email protected]