The IR Role in The IR Role in Subscriber Subscriber Managed Course Managed Course ReviewsReviews

QM Institution Representative Training © MarylandOnline, Inc., 2010. All rights

reserved.

Agenda for Today’s Workshop•Types of course reviews and their requirements•Readiness for a review – the course review and the faculty developer•IR responsibilities and time frames in a course review•How to use the Peer Reviewer Search Tool•How to apply for a course review•How to set up a subscriber managed course review•Reporting to QM•Scoring and certification of courses•Re-certification of courses

8 Topics for QM IR Webinars1. The Role of the QM Institution Representative

2. Using the QM Website and MyQM

3. What IRs should know about the QM Rubric

4. The IR role in Internal (Unofficial) and QM-Managed Course Reviews

5. The IR Role in Subscriber-Managed Course reviews for QM Recognition6. Selecting and Registering for QM Training

7. Communication by, with, and about QM

8. The QM Implementation Plan

Course Review Process

Peer Course Review

Feedback

Course

Course Meets Quality Expectations

Course Revision

Instructional Designers

InstitutionsFaculty Course Developers National Standards &

Research Literature

Rubric

Faculty ReviewersTraining

Quality Matters:

Peer Course Review Process

Peer Course Review

Feedback

Course

Course Meets Quality Expectations

Course Revision

Instructional Designers

InstitutionsFaculty Course Developers National Standards &

Research Literature

Rubric

Faculty ReviewersTraining

Quality Matters:

Peer Course Review Process

Types of Course Reviews and Requirements

Official Course Reviews – Any course reviews, QM-managed or subscriber-managed, that are conducted in compliance with QM Course Review Standards with the intention of receiving QM recognition. QM-Managed Course Reviews – Course reviews that are

managed by Quality Matters (QM) staff and/or designees in compliance with QM Course Review Standards.  

Subscriber-Managed Course Reviews – Official course reviews, managed by a trained Subscriber representative, that adhere to the QM Course Review Standards and use the QM online tools. 

Internal Course Reviews - Unofficial course reviews managed by the subscribing institution. Courses reviewed in this way are not eligible for QM recognition.

QM Course Recertification – Courses older than 5 years that have previously received QM recognition are eligible for an expedited course recertification. See policy on QM Course Recertification (in QM IR Manual)

Steps in a Subscriber Managed Course Review

IR notifies faculty developer of course review IR submits course review application for all types of course reviews IR contacts peer reviewers: confirms availability and discusses

stipends IR approves course review application IR initiates completion of faculty developer worksheet IR assigns peer review team and opens up the rubric for peer review

team Chair sets up pre-review conference call with faculty developer, IR

and peer review team Review team reviews course (3-4 weeks) Review team conducts post review call if needed Chair submits final report and sends review outcome Faculty developer submits online faculty response form Faculty developer makes changes during amendment period if

needed Quality Matters awards QM recognition * (duration may vary, see next

slide)

How Long does Recognition Last?

Up to 5 years, as long as Any improvements to the course is made by

original instructor or design team Material added by other instructors do not alter

the basic structure and IR verifies compliance with standards (Master Course model)

Versions of a QM certified course offered by other instructors must undergo their own review

See Multi-Section (Master Course ) policy

Preparing for a Course Review

QM course reviews are designed for mature courses that have been taught previously and for courses that are fully developed online such that reviewers can see all evaluated components.   Note: Courses that have not yet been taught MAY be

reviewed only if all components have been fully developed

What kind of course access does the review team need?  Student-level access. Quality Matters recommends using a full-developed

course shell rather than a live course or the most recent archive, with steps taken to ensure student privacy rights.

Checklist for Review Readiness

The following checklist can help you determine if a course is ready to be submitted for review: _____ Has the faculty developer/ instructor had time to review the Quality Matters rubric and make modifications before the review? Faculty that are prepared for a Quality Matters review have better outcomes and get more out of the review process. _____ Are the course-level outcomes specified and are there module/unit objectives for each of the course module/units? The lack of course-level outcomes and module/unit-level objectives is among the most frequently missed standards. _____ Are all discussion board questions or topics posted for review? This might include samples faculty responses designed to facilitate the desired interaction. (If student responses are also provided, they should be stripped of identifying information.)  _____ If the course uses email in any part of the instruction, is this information made available to the review team? Examples of such email exchanges should be provided to the review team during its review of the course. _____ Are all course activities, including all audio-visual components, available to the review team? Sometimes instructors make assignments “not available” after a specified “due date.” All such assignments will need to be available to the review team. _____ Are all assessment tools available for review by the team? The review team will need to be able to access quizzes, exams, and tests, and it would be beneficial to the review team to also have access to the (blank) grade book as it is set-up for the “student view.” _____ Can the review team see and experience the course as a student would?

IR Responsibilities

Institution Representatives play an important role in the success of a review team. Keep in contact with the Course Developer to assist him/her in completing the faculty responsibilities. Contact the Team Chair if you have questions about the review in progress.

1. Keep in contact

2. Complete & Approve application

3. Set up Peer Review Team

4. Send & Approve Instructor Worksheet

5. Participate in a Pre-Review Team Discussion

6. Enroll Peer Course Review team members in your course

7. Send login directions

8. Return Faculty Response Form

9. Pay Reviewers

10. Post Review - Revise course (if needed)

Review Requirements

•Use of current and unmodified QM rubric standards•Review of online or hybrid course•Three person peer review team meeting the following criteria:

All reviewers are certified eligible peer reviewersAt least one reviewer is external to the institution submitting the

reviewAt least one reviewer is designated as a subject matter expertTeam chair is certified QM Master Reviewer

Process Requirements – The following are the QM process requirements that must be met on the reviews of all courses seeking QM recognition:

Instructor Worksheet, Final Report, and Course Amendment Form (if applicable) have been submitted online.Team chair evaluates amendments to courses.Pre-review communication with Course Developer and review team.Post-review survey completed by the institution representative, Course Developer, and all reviewers.No more than 20 weeks between the start of the review and the date documentation is submitted to Quality Matters for course recognition.

Scoring and Recognition

To meet standards and gain QM recognition a course must:

•Have followed all QM review requirements•Met all 17 essentials standards (3 point standards)•Have a score of at least 72 points out of 85•Complete all necessary paperwork such as faculty developer

worksheet and survey•If amendment process is needed: fill out course amendment form,

chair approval, completed within 20 week time frame

How to Apply for a Course Review

http://qmprogram.org/review/

The new Course Review Management System CRMS (online rubric tool) will officially launch November 15, 2010

All types of reviews will start with an application being submitted.

Info needed:

•who will serve as IR•faculty developer email address•course number•course name•discipline area

Peer Reviewer Search 1

Peer Reviewer Search 2

Peer Reviewer Search 3

Peer Reviewer Search 4

CRMS Homepage

Application

Institution selection

Select IR for review

Choose Review Type

Application information

Application information

Display Applications

Process Application

Send Faculty Developer Worksheet

Faculty Developer Worksheet

Status FD Worksheet

Approve Faculty Developer Wksht

FD Worksheet Upload

Upload Course Objectives

File Uploaded

Status FD Worksheet

Faculty Developer Worksheet

Add Reviewers

Add Reviewers

Add Reviewers

Review Team Verification

Mark Reviewer Selection Complete

Review Started

Quality Matters

Questions?

Thanks to YOU…Thanks to YOU…Quality Matters!Quality Matters!

Fall 2010© MarylandOnline, Inc., 2010. All rights

reserved.