Enhance Exam Integrity with Online Proctoring...Exam content needs to be updated more frequently (sometimes several times a year) Which means that…. Release cycles are getting quicker

Chloe Mendonça

UK Marketing Manager, Questionmark

Enhance Exam Integrity with Online Proctoring

Copyright © 1995-2016 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.

Agenda


What is online proctoring and how does it work?

What are the benefits of online proctoring?

Managing test security threats

Considerations & Best Practices

A bit about us…

• Founded in 1988

• Offices in US, UK, Canada; Distribution partners in 27 countries

Background

• Multilingual Assessment Management System

• Questionmark OnDemand (Cloud/SaaS)

• Questionmark Perception (Installable software)

• Assessment technologies and portal solutions that enable organisations to measure knowledge, skills, and attitudes securely for certification, regulatory compliance and learning

Products & Services

• Enterprise customers worldwide within the corporate, government; higher education; awarding body; test publisher sectors

Customers


Online Proctoring / Online Invigilation

• We used to use the term invigilation

• With the proliferation of US Companies providing Online Invigilation the industry appears to be accepting the term Online Proctoring


Online Proctoring

X

A webcam is used to authenticate the candidate’s identity

The proctor observes the candidate as they complete their exam


Candidates can take exams online from their home or office using their own laptop

Options for Proctoring Tests and Exams

Record & Review

Dedicated Kiosk with multiple cameras

Test Centers

Events (Classrooms or Conventions)

Remote real-time webcam

Un-Monitored


• Convenient, personable, consistent

• Proctor vulnerable to unfair influence

Events (Classrooms or Conventions)

• Personable, consistent

• Proctor vulnerable to unfair influenceTest Centers

• Proctor is less vulnerable

• New and not yet widespread

Dedicated Kiosk with multiple cameras

• Proctor is less vulnerable

• Easy to deploy for use at home

Remote real-time webcam

• More cost effective than remote real-time

• Nothing for content theftRecord & Review

• Works for employees

• Nothing for content theftUn-Monitored

Comparing the options

ID F

rau

d

Pro

tect

C

on

ten

t

Min

imiz

es

Ch

eat

ing

√ √ √

√ √ √

√ X

√ X X

√ √ √

√

√ √ √


Traditional Proctoring•Proctor observes test taker physically in

person / in the same roomOnline Proctoring• Proctor observing test taker live via webcam

• Internet connection must be continuous


Typical Candidate Experience

1Candidate login

and system-check

2Verification of

candidate ID and work space

3Candidate

begins exams

Initiates proctored

session

Exam released to candidate

4Candidate completesexam and proctored

session ends

Candidate minimizes

proctor window


Who is using it?


Higher Education and Certification Institutions


What is driving this need? – Explosion of MOOCs


“Students definitely prefer online proctoring. It’s more

convenient and affordable than going to a testing centre.”

- Chris Heather, Product Manager, Coursera

What is driving this need? – Reducing time-to-market


Release Cycles Happen More Frequently

Which means that….

Qualifications for credentials/certification need to keep pace

Exam content needs to be updated more frequently

(sometimes several times a year)

Which means that….

“Release cycles are getting quicker and quicker. For certification, this means that consultants and others have to show that they are always up to date and are certified on the latest release.

But of course if you are taking certification exams more often, certification costs have to be much lower. In some regions, people have to travel large distances to get to a test centre. Our aim is to allow certification anytime and anywhere using the cloud.” - Ralf Kirchgaessner, Manager of Global Certification, SAP


What is driving this need? – Enhancing Exam Integrity

Director Varinder Bajarh, Director of Studentway Education told the undercover test taker:

"Someone else will sit the exam for you. But you will have to have your photo taken there to prove you were present.“

The researcher was told a "guaranteed pass" would cost £500 - about three times the proper fee for the exam

Student visa system fraud exposed in BBC investigation . At least 29,000 exam results were invalid!

Benefits of using Online Proctoring

Professionally trained Technology Supported

Lower Costs

Independent Proctors

Fairness Candidate Satisfaction


How does online proctoring deal with exam security threats?

Impersonation. Cheating. Content Theft.


Key threats to consider

Candidate Impersonation Collusion Content Theft

Access to restricted aids Reliability and robustness Privacy & data protection


Dealing with impersonation / proxy test takers

Enabling proctor controls, e.g. pause stop etc.

Recording / storing session and time stamping

any incidents

REC SAVE

Authenticating candidates

Options for…


Dealing with cheating aids / collusion

Camera can see large portion of the room

Camera & microphone pick up sounds / talking

Secure browser to restrict access to online aids

Checking of area with reflective surface

Panning of room with webcam

Proctors can be prohibited from seeing

test taker screens


Mitigating risk of content theft

Secure browser to restrict right click, copy paste and other functions

Live monitoring throughout the exam


Considerations for Online Proctoring

Must have high speed internet connection

Reasonably comfortable with tech

Proctor language

Breaks are restricted Computers not standardised

Test program customisation


Online Proctoring Best PracticesSix Tips


#1 Develop & Define Policy

• Policies for technical problems

• Policies for accessibility accommodations

• Policies for following up on infractions

Anticipate problems and have a way to manage them


#2 Set Candidate Expectations

Prepare and educate candidates with training resources

Blog Articles

Online Information

Videos


https://vimeo.com/143718550




#3 Use a comprehensive lock-down program

Should allow prevention of:

PrintingScreen capture

Copy/ PasteLaunch of applications and

communication toolsTask switching

Browser control / navigationControl/system keys

Right-click / context menu optionsView source

Accidentally exiting exam

Should be compatible across multiple devices and operating systems:

Windows Mac iPad


#4 Allow communication between proctor and test-taker throughout session

Participants should be able to communicate via:

• Audio

• Video

• Chat


#5 Allow proctor to control test session

Proctor controls should include the ability to pause, suspend or cancel the test.


#6 Record and store exam sessions (including chat log)

• Record entire session including all communications

• Time stamp incidents to make them easy to review


Online Proctoring in the future

It’s only going to get better…


More options for candidates

Online Proctoring is broadening the horizon for candidates and test sponsors


Keep in touch!


Follow our blogblog.questionmark.com

Connect with us on social mediaYouTube, Twitter, Instagram, LinkedIn, Google+, Facebook

Thank you for attending!Watch out for our next session: An Introduction to Questionmark


Chloe MendoncaUK Marketing Manager

[email protected]

LINK

Documents

Enhance Exam Integrity with Online Proctoring...Exam content needs to be updated more frequently (sometimes several times a year) Which means that…. Release cycles are getting quicker