CENTRE FOR THE EVALUATION OF HEALTH PROFESSIONALS … · Expanding Our Reach CENTRE FOR THE EVALUATION OF HEALTH PROFESSIONALS EDUCATED ABROAD PROGRESS REPORT 2013–2014 ... volume

Expanding Our Reach CENTRE FOR THE EVALUATION OF HEALTH PROFESSIONALS EDUCATED ABROAD

PROGRESS REPORT 2013–2014

*If you would like to obtain a copy of our audited financial

statements for 2013/2014, please send your request to

[email protected]

Inside this Report

CHAIR’S MESSAGE 3

CEO’s MESSAGE 4

COMPETENCY ASSESSMENTS 6

COMMUNICATION COMPETENCY PROGRAM 16

EDUCATION PROGRAMS 20

BUILDING CAPACITY FOR RAPID GROWTH 24

INTRODUCING OUR TEAM 28

SHARING OUR EXPERTISE 30

[email protected]

CHAIR’S MESSAGEOn behalf of the Board of Directors of theCentre for the Evaluation of Health ProfessionalsEducated Abroad (CEHPEA), I am pleased tointroduce our 2013/2014 Progress Report.The report chronicles a year of growth in thevolume and scope of the services we provideas we expand our reach to supportinternationally educated health professionals(IEHPs) seeking to practice in Ontario.

This growth was envisioned in the five-yearStrategic Plan that the board of directorsfinalized in 2012. The plan was designed toensure the organization’s continued successand relevance as a trusted resource for theprovince’s health care system.

As part of the Strategic Plan, the board wentthrough a period of renewal to define thecompetencies we need from our directors toguide the organization into the future. Thisyear, we were pleased to welcome thefollowing new members on our board: BrendaZimmerman, director of the Health IndustryManagement Program at the Schulich Schoolof Business, York University; John McKinley,principal at JMcKinley Consulting andCloudIPHI; and George M. Thomson, seniordirector, International Programs at the NationalJudicial Institute. We have a small, hard-workingboard, and I would like to take this opportunityto thank our members for committing theirtime, energy and considerable expertise toinvest in the best interests of CEHPEA.

The board works in close collaboration withSten Ardal, the organization’s chief executiveofficer (CEO) and the excellent staff. Sten,who was appointed CEO in the spring of 2013,has been able to build on the accomplishmentsthat were already underway, while bringingfresh vigour to the strategic plan and itsimplementation. During this period of growth,he ensured high-quality staff and serviceswhile growing the business and expanding theorganization’s scope. I would like to commendSten for his leadership this past year andacknowledge CEHPEA’s outstanding staffmembers. They are knowledgeable, committedand collegial and the work they do is incredible.

We expect that our growth will continue as weleverage our expertise to ensure that othergroups of IEHPs seeking to practice in Ontariohave access to fair, transparent and timelyassessments and services. We remaincommitted to providing high-qualityassessments at a variety of levels to ensurethat candidates are qualified and competentto provide care to the citizens of Ontario.

EXPANDING OUR REACH 3

Dr. Kennneth A. Harris

CEO’S MESSAGEThe past year has been a period of greatgrowth and expansion for our organization aswe implement our Strategic Plan (2012–2017).I am pleased to update you on the work thathas taken place in the following major areas of focus:

• Exploring opportunities for expansion• Strengthening the organizational structure• Reviewing our brand identity

Our organization was originally created, aspart of the HealthForce Ontario strategy, todevelop and administer tests and transitionprograms for international medical graduates(IMGs). This is an area that has continued toshow substantial growth. Working closely with

the Medical Council of Canada (MCC), wehave been the largest test site in the countryfor their National Assessment CollaborationExam, and in 2014, our test volumes almostdoubled. We also offer innovative transitionprograms for IMGs accepted into residencytraining at Ontario’s medical schools and thisyear we launched the Canadian MedicinePrimer – a new, customized immersionprogram for sponsored visa trainees andclinical fellows.

Exploring Opportunities for Expansion

Over the years, it became apparent that otherhealth care professionals and regulatorycolleges could also benefit from the type ofexpertise that we have to offer. A strategicfocus for us this past year has beenleveraging our core expertise in competencyassessments for physicians to help otherregulatory colleges create a process that isfair, transparent and accessible forinternationally educated health professionals(IEHPs) that want to practice in Ontario.

In 2009, we created an assessment processfor IMGs interested in becoming physicianassistants. Two years later, we partnered withthe College of Nurses of Ontario (CNO) tocreate an objective structured clinicalexamination (OSCE) as part of a qualityassurance initiative for nurse practitioners.

Our most substantial expansion has been thecreation of a new assessment process forinternationally educated nurses (IENs) thatwant to write the national registration exam fornursing. The IEN Competency AssessmentProgram, developed with the CNO, waslaunched in March 2013, and has rapidlyexpanded with 1,200 registered nursecandidates taking the test in 2014, an increaseof 45% over expected volumes.

4 PROGRESS REPORT 2013/2014

Sten Ardal


In 2013, we contracted with the College ofOptometrists of Ontario (COO) to createCanada’s first evaluating examination forinternationally educated optometrists (IEOs).The assessment has generated a great dealof interest with several observers fromcolleges across the country attending the fallpilot test. The COO Evaluating Examinationlaunches in June 2015.

This year, we launched a CommunicationCompetency Program, recognizing theimportance of language proficiencymeasurement as part of the process forIEHPs. This is a major new area of focus forour organization, made possible with grantfunding from the Ontario Ministry ofCitizenship, Immigration and InternationalTrade (MCIIT). This program will serve as ahub for occupation-specific languageassessments and other related resources andservices for IEHPs and regulatory colleges.

An early success has been our recognition by the Centre for Canadian LanguageBenchmarks (CCLB) as the successful bidderin a national competition to become Canada’snew centre for its Canadian English LanguageBenchmark Assessment for Nurses (CELBAN).This language proficiency test is recognizedby nursing regulatory bodies and will beadministered through our CommunicationCompetency Program’s CELBAN Centre insites across Canada.

Strengthening the OrganizationalStructure

We have been building capacity andstrengthening organizational structures tosupport our rapid growth in existing programsand assessments and the new business areaswe have developed.

Investing in our human resources has been avery important priority as we strive to build astrong, unified internal team and provideopportunities for staff development. We havecreated a Standardized Patient Unit, an ExamUnit and expanded psychometric services toaddress higher test volumes and support thedevelopment of new assessments. We alsoundertook a major overhaul of our informationtechnology, audio visual and security systems.

This fall, our Standardized Patient Programcompleted the biggest recruitment campaignin its history. It is now the second largestprogram in Canada with more than 1,200standardized patients in its database, a 45%increase from 2013.

Reviewing Our Brand Identity

Given the evolution of our organization overthe years, the board approved a review andanalysis of our brand identity. As a result of thatexercise, we will be officially launching our newname and brand in January 2015, when wewill become Touchstone Institute: CompetencyEvaluation Experts. As Touchstone Institute,we look forward to expanding and enhancingour services, working with regulators, ourpartners and specialists to facilitate entry topractice for IEHPs.

Finally, I would like to express my appreciationfor the leadership provided by Dr. Ken Harrisand our Board of Directors, to the Ontariogovernment for recognizing our critical roleand providing the funding necessary for us torealize our mandate, to the clients who workwith us to ensure that the highest quality isattained and maintained, to the candidatesand learners who remind us of why we careso much about our work and to the staff whocommit their very best every day and make thisorganization so compassionate and so unique.

The demand for a reliable, valid assessment ofprofessional competencies continues toincrease across professions. To meet theseneeds, we have increased our capacity in testadministration, with larger cohorts of nursingand physician candidates passing through ourdoors every year.

We also continue to leverage our coreexpertise to support other professionalgroups in developing written and clinicalcompetency assessments. This year, we haveformed new partnerships, established strongnetworks and shared our expertise inassessment with a growing roster of healthcare organizations.

For us, this growth has meant an increasedneed for resources and training.

We’ve been building teams and networks ofdedicated professionals with the educationand experience necessary to deliver high-quality assessments to diverse candidates.

COMPETENCYASSESSMENTS

Since 2007, our organization has played a key role in providing reliable information to colleges about theprofessional competencies of their candidates. “Colleges for self-regulated health care professions areentrusted with protecting the public,” says Chief Executive Officer Sten Ardal. “Our assessments arehighly relevant, providing information about candidates’ performance in objective, structured clinicalexaminations that mirror real-life situations. This information has a great impact on colleges’ ability toassess the competency of candidates.”


For IEHPs, a high-stakes assessmentpresents an opportunity to demonstrate theirknowledge, skills and experience. It can alsobe a potentially stressful day for them. Ourassessments are designed to ease test-takers’ anxieties and provide them with thetools they need to proceed through theassessment with a full understanding of theguidelines and expectations.

Orientation

Before their assessments, candidates attendan orientation session that outlines theagenda for the day. They learn whatpaperwork they need to complete andreceive an overview of the exams and thetiming allotted for each of the day’s events.For written tests, they are given examples ofhow to correctly mark their answers; forOSCEs, they learn what equipment they willhave access to, the timing and steps of theclinical encounter, and what the expectationsare for each stage.

Communication

Candidates receive clear instructions andguidance throughout their assessment. Priorto beginning each OSCE station, they areallotted time to read through writteninstructions, while during the clinical encounterclear overhead announcements notify them ofthe time remaining at each stage and whereto proceed next. Group leaders and supportstaff are on hand to answer their questionsand guide candidates from station to station.

Logistics

Our OSCE stations are organized in a way thatallows for an easy transition from one stage ofthe assessment to the next. Candidates are

separated into groups and complete a seriesof clearly marked stations within a track.Confusion during an exam adds unnecessarystress, so our goal is to ensure everything isas clear and intuitive as possible.

Consistency

During OSCEs, candidates interact withstandardized patients (SPs), who are trainedfor each part of the clinical encounter toportray the role in a consistent andmeasurable way. No matter which SPinteracts with the candidate, the candidate willreceive the same information and responses,ensuring a fair and valid assessment.


The Candidate Experience

Our ResourcesTo administer our exams, we rely on corein-house personnel and a network ofsupportive staff, all with a firmcommitment to providing fair andaccessible assessments for candidates.We draw from a vast database of expertsto meet the unique staffing needs of eachassessment.

1,000 support staff

1,000 standardized patients

600 examiners

5 full-time exam coordinators, plus a unit manager

5 full-time SP trainers, plus a unit manager

COmPETENCy ASSESSmENTS

Developed by the National AssessmentCollaboration (NAC), this examination is anational, standardized objective structuredclinical examination (OSCE) that assesses thereadiness of an international medical graduate(IMG) for entrance into a Canadian residencyprogram.

Prior to 2015, the NAC Examination wasoptional for IMGs. However, Ontario medicalschools recently enacted a recommendationthat all IMGs be required to challenge theMedical Council of Canada’s NACExamination. In order to qualify for 2015residency positions, candidates were requiredto submit their NAC Examination scores toCaRMS by November 2014.

As a result of the new requirements, thenumber of candidates registered to take theexam in Canada almost doubled from 2013 to2014. In Ontario alone, well over 900candidates were registered to complete theexam in September, in preparation for theirfirst-year residency positions.

While the Medical Council of Canada servesas the central coordinating and administrativebody for the NAC, the exam is administeredregionally by provincial test centres. We arethe only NAC Examination site in Ontario andthe largest site in Canada.

To accommodate the increased number oftest-takers this year, two additional dates


Assessments for PhysiciansThe National Assessment Collaboration (NAC) Examination for International medical Graduates

were added, resulting in four assessmentsadministered within a nine-day period.

“We were happy to be able to accommodatea larger number of candidates,” says SuzanneBambrick, director of Human Resources andOperations and acting director of Finance andAdministration, “and offer more frequentexam options with Toronto as the assessmentlocation.”

Even with our unique ability to accommodatea large number of participants, additionalcapacity was needed. For the first time, weadministered assessments off-site, with twoadditional tracks at the Toronto MichenerInstitute. We also recruited and trained

additional standardized patients andexaminers. Staff worked 12-hour days onsuccessive weekends to administer theassessments, which they did with remarkablyfew challenges.

“CEHPEA rose to the challenge to increase itscapacity to deliver the NAC exam this year,”says Alexa Fotheringham, NAC Examinationmanager at the Medical Council of Canada.“They were already the largest test site inCanada, but almost doubled their capacity tomeet the recent demand. CEHPEA staffmembers are clearly committed to ensuringthat candidates are assessed properly andthat the exam process is standardized,transparent and fair.”



Canadian NAC Examination Volumes 2012–2014

“Our principles centre on setting defensible competency standards in ourevaluations that ensure fairness, objectivity, transparency and impartiality for allcandidates while ensuring public safety. Working in partnership with professionalcolleges and regulatory bodies, we create, develop and implement our assessmentsbased on global best-practice design procedures.”

Debra SibbaldDirector, Assessment Programs

BC AB MB ON QC NS TOTALS

69 173 75 492 122 — 931

134 149 108 678 139 15 1,223

215 285 155 1,161 341 60 2,217

2012

2013

2014


The Internationally Educated NursesCompetency Assessment Program (IENCAP),continues to grow to meet the needs of IENson their pathway to practice in Ontario. Ourcommitment to the program has beensolidified in an agreement to administer theassessment for the next three years.

This year, we increased our capacity toadminister the assessment, as the number ofcandidates referred for evaluation continuesto climb. In 2013, each assessment dayconsisted of two administrations, allowing atotal of 144 candidates to challenge the examon each day. In 2014, we added an extraadministration, resulting in 216 candidatestaking the IENCAP on each assessment day.

This required calling on a greater number ofstandardized patients and staff toaccommodate the 13-hour exam days.

During several assessments, we welcomedobservers from the Office of the FairnessCommissioner, the CNO and the CARECentre for Internationally Educated Nurses.From within dedicated viewing stations, theywere able to observe how candidates,standardized patients and examinersperformed in a clinical encounter. Viewing anOSCE helps observers gain an understandingof the assessment process, the candidate’sexperience and the logistics involved in ourlarge-scale exams.

Assessments for NursesOntario’s Competency Assessment Program for Internationally Educated Nurses

“It remains a highlight of my professional career to be a part of this initiative. Theeffort that has been put into the program to ensure quality and provide adequateassessment remains inspiring to me. The efforts of the staff and examiners tomaintain standardization, underline their commitment to this program and to thesafety of the public in Ontario.”

Tammy McParlanChief Nurse ExaminerAssistant Professor of Nursing, Nipissing University, North Bay

We were also joined by partners in theoptometry field, who gained insight into theassessment process, which helped provide afoundation for their contributions to thedevelopment of the College of Optometristsof Ontario Evaluating Exam (COO EE).

Assessing Diverse Candidates: A Regulatory Challenge

Candidates with diverse overseasqualifications need to be screened for eligibilityto write the Canadian Registered NursingExamination (CRNE), now the National CouncilLicensure Examination for Registered Nurses(NCLEX-RN). Launched in 2013, IENCAPassists in assessing the equivalency of thediverse international qualifications to Canadiannursing programs. It reliably assesses IENs’readiness to write the registration exam andprovides feedback on areas for furtherimprovement.

The assessment was developed with contentexperts and conforms to an entry-to-practicecompetency blueprint. Chief examiners pre-rated the items, with criterion-anchored toolsthat validated content and competencyscoring schemes. It consists of writtenmultiple choice clinical examination and aseries of OSCE stations that assess nursing



IENs Assessed To-Date

Years Post-Graduation33.0%

67.0%

≤ 10

>10

Institution by Country

Philippines

India

Other

Candidate Age Range

< 30

30-39

40+

38.5% 34.8%

26.7%

23.1%35.4%

45.3%

knowledge, skills, judgment, client interactions,language proficiency and comprehension.

IENCAP has proven to be a valuablecomplement to traditional credentialscreening for the licensing exam. The testdesign incorporates appropriate parametersand measures to validate competency-basedperformance assessment for eligibilitycompared to graduates from Canadianprograms. The result is improved recognitionof diverse overseas qualifications through adefensible and appropriate assessmentprocess that is fair, transparent, objective andimpartial.

Nurse Practitioner PracticeAssessment

In 2011, we worked with the College of Nursesof Ontario (CNO) to create a QualityAssurance Practice Objective StructuredClinical Exam for nurse practitioners in theprovince. With the launch of newcompetencies for nurse practitioners, we areworking with the College to create a newquality assurance assessment for 2015.



“It has been really exciting to have been part of this process from the beginning. The IENCAP ultimately helps the candidates achieve their goal of becomingnurses in Canada.”

Heather ScottChief Nurse ExaminerCare Coordinator, Community Care Access Centre, Hamilton General Hospital, Emergency Department

Facts & FiguresAn IENCAP assessment in 2014consisted of:

3 administrations per day

6 tracks

12 exam stations

10 viewing stations for inter-rater reliability

82 examiners

100 support staff

110 standardized patients

216 candidates assessed on each day

1,300 candidates are expected to be

assessed in 2014


In January 2014, we began working with theCollege of Optometrists of Ontario (COO) todevelop an exam for internationally educatedoptometrists (IEOs). The COO EvaluatingExam tests the knowledge, skills, attitudesand values of the candidates as compared toa recent graduate of a Canadian optometricprogram, and assesses their eligibility to writethe licensing exam.

Exam Development

A panel of Ontario optometrists and our in-house exam experts participated in ablueprinting exercise to articulate professionalcompetencies and general attributes for theevaluation. Key competencies were identifiedbased on the College of Optometrists of

Ontario Competency-Based PerformanceStandards for Canadian Standard Assessmentin Optometry (2005).

The resulting evaluation exam consists of twocomponents – a written test and aperformance evaluation. In order to evaluatethe candidate’s ability to conform to the scopeof practice for optometrists, the assessmentprocess includes physical eye examinations.Unlike our OSCEs for physicians, nurses andnurse practitioners, the optometry stationsinvolved more physical testing and theassessment team met these challenges inunderstanding and acquiring the appropriateequipment.

Assessments for OptometristsLaunching a New Examination for Internationally Educated Optometrists

Field Testing

A field test held in August 2014 ensured thatthe clinical stations and written exam werefeasible and acceptable logistically, withrespect to content relevance, presentation,timing and level of difficulty. Scoring stationsused a criterion-based validation tool, whileobservers and examiners provided qualitativefeedback to help fine-tune the clinical stationsand offer commentary on the written multiplechoice exam.

Exam Pilot

In November 2014, optometry registrars fromacross Canada observed the pilot test toassess the role the exam could play for theirjurisdictions. Our evaluating exam can bemade available to candidates seekinglicensure throughout English-speakingCanada.

Plan for Implementation

We continue to refine this evaluationassessment in preparation for the launch ofthe Optometry Evaluating Examination, whichis slated for June 2015.


Debra Sibbald leads a panel of Ontario optometriststhrough an assessment blueprinting exercise.

Development of an Evaluating Exam

Blueprinting Exercise

Workshops

Field Tests

Pilot Implementation

1

2

3

4

5




To provide optometry partners with insight into our exam processes, they observed dry run assessment activities andtraining of our standardized patients for their roles.

Suzanne Bambrick gives optometry experts an overview of logistics for the assessment blueprinting process.

COMMUNICATIONCOMPETENCY PROGRAM

The ability to communicate effectively is a critical asset for newcomers pursuing a professional registrationpathway. Health care professionals are expected to practise safely, with integrity and professionalism, andbe able to communicate across language, cultural, generational and situational barriers. This is describedas “communication competency.” In order to demonstrate communication competency, health careprofessionals must be able to use language effectively and appropriately to deliver quality patient care inOntario’s inter-professional, multicultural and client-centred health care environment.*

16 PROGRESS REPORT 2013/2014 * The OCECCA Project: Creating a Centre of Excellence for Communication Competency Assessment, 2013, p. 10

Recognizing the importance and impact oflanguage proficiency measurement as part ofthe professional registration pathway forinternationally educated health professionals(IEHPs) in Ontario, we have established a newunit dedicated to assessing communicationcompetency assessment. Funding for theprogram was provided by the Ontario Ministryof Citizenship Immigration and InternationalTrade (MCIIT).

The Communication Competency Programstarted in April 2014 with the appointment ofAndrea Strachan, a national expert inlanguage proficiency standards for regulators.

Our program draws extensively on the findingsof the 2013 Ontario Centre of Excellence forCommunication Competency Assessment(OCECCA) Project report. The reportrecommended the creation of an OCCECACentre to serve as a hub for occupation-specific languages assessments and otherrelated resources and services for IEHPs.

Devoted to fostering excellence inoccupation-specific language assessments,our Communication Competency Programstrives to:

• Develop alternative assessments oflanguage and communication competencyfor both individuals and groups acrosscareer stages

• Improve existing assessments and designnew tests to meet the evolvingrequirements of professional practice

• Administer secure and reliable health-specific language assessments that may beused to demonstrate communicationfluency at entry to practice

• Assist regulators in establishing fair anddefensible language proficiency standardsand accessible professional examinations


“Ultimately, we want to create a resource hub to support regulators across Canadain addressing the assessment of language and communication competencies in amanner that is relevant, fair and accessible. This is an area that regulators constantlygrapple with – especially in reference to guidelines from the Office of the FairnessCommissioner for relevant and fair assessments. Given the organization’s well-established expertise in competency assessments and training, we believe we cancontribute in the specific areas of language and communication skills.”

Andrea StrachanDirector, Communication Competency Program

COmmUNICATION COmPETENCy PROGRAm

Early Results

Our organization has a reputation forresponding quickly to changing needs inOntario’s health care system and creatinginnovative new programs from the ground up.The launch of our new CommunicationCompetency Program was no exception.Within six months of hiring the director, a teamof three has been recruited and a lot of workis underway. Here are some highlights ofprogress as of November 2014:

• Awarded a grant by MCIIT to implement theOCECCA Implementation and EvaluationProject through our CommunicationCompetency Program

• Created an interdisciplinary CommunicationCompetency Program Project AdvisoryCommittee (regulators, academics andcommunication specialist)

• Started liaising with health care regulatorybodies and employers to discuss needsand opportunities

• Began a CELBAN review process that willprovide a strong foundation to inform therenewal and development of CELBAN testforms. The review will include a languageproficiency review, nursing scope ofpractice review and technical review

• Began exploration of license agreementswith existing occupation-specific languageassessments

• Submitted an RFP and was successfulbidder to become the national administratorand manager of the Canadian EnglishLanguage Benchmark Assessment forNurses (CELBAN)

• Initiated a research project to evaluate theeffectiveness of CELBAN



Introducing the CELBAN Centre

CELBAN is a languageproficiency test that isaccepted by nursingregulatory bodies andthe preferred option formany internationally

educated nurses (IENs). The test, which isoccupation-specific, evaluates languageproficiency in speaking, listening, writing andreading English, using tasks that are set in anursing context. Occupation-specific tests,such as CELBAN, help candidates prepare forthe language demands of their professionalpractice and the workforce.

This test will be administered in Canadathrough our Communication CompetencyProgram’s CELBAN Centre. The Centre willfocus entirely on administration of the test andwill allow for the broader work with regulatorsin the area of communication competencyassessments to continue as the central focusof the program.

“Our initial priority is to scout out suitablelocations and assessors across Canada to runthe CELBAN assessment, and we hope toleverage those resources to build theinfrastructure to support other regulatorybodies in running assessments across thecountry,” says Andrea Strachan.



• Test review and evaluation

• Revise blueprint and create

new content

• Establish assessor training

and certification

• Preparation materials for

test takers

• Establish a quality

management framework

Re

ne

wa

l• Reinstate testing

• Address wait list through

"pop up" test sessions

• Review and update policies

and procedures

• Establish online registration

• Develop capacity

• Launch CELBAN Centre

website

Tra

nsi

tion • Increase number of test

sessions administered

nationally

• Access in areas previously

not served

• Establish feasibility of online

delivery

• Institute ongoing content

renewal

Gro

wth

CELBAN Summary Plan

EDUCATION PROGRAMSFOR PHYSICIANS EDUCATED ABROAD

We offer transition programs for international medical graduates (IMGs) who have been accepted intoOntario residency programs through the Canadian Resident Matching Service (CaRMS). The programsprovide IMGs (including Canadians educated abroad) with curriculum, orientation and services tofamiliarize them with Canadian health systems and practices before they begin their residency training.


Key Features of Our EducationPrograms

• Our education programs arecomprehensive, flexible and innovative.They were developed in consultation withprogram directors at Ontario’s faculties ofmedicine. Canadian physicians withextensive educational experience areinstructors, advisors, consultants andfacilitators.

• The Pre-Residency Program (PRP) and theOrientation to Training and Practice inCanada (OTPC) program start and end with“skills checks” through objective structuredclinical examinations (OSCEs) withstandardized patients and physicianexaminers. The checks provide candidateswith valuable feedback on their knowledgeand skills in a safe learning environment.

• Through the Academic Advising Program,physicians are able to provide guidance tostudents on setting individual goals andobjectives.

• Our courses offer a blend of classroom,simulation and online learning experiences.The integrated classroom sessions consistof interactive lectures and small groupsessions with standardized patients andphysician facilitators. Self-directed andfacilitated online modules are available forstudents prior to their arrival for classroomsessions and are sequenced in a way thatcomplement the classroom sessions.

• In 2015, we will be implementing a newmodular model and renaming our programs.Core courses, referred to as PRP, will berelevant to all specialties. PRP-FM will includethe core courses in PRP plus modulesspecific to family medicine.

EDUCATION PROGRAmS


EDUCATION PROGRAMS

At a GlanceOrientation to Training and Practice (OTPC)

The two-week OTPC was developed to help IMGs

prepare for a successful transition into specialty

residency training programs in Ontario. This

mandatory program covers key competency areas

as well as topics such as communication, law and

ethics, privacy, confidentiality and dealing with

challenging situations. The program includes

Advanced Cardiovascular Life Support (ACLS)

certification training.

Pre-Residency Program (PRP)

PRP is a mandatory program for all IMGs who are

accepted into an Ontario family medicine residency

program.

Phase 1 of PRP, which takes place at our Toronto

office, is a compressed, four-and-a half week

program that includes a blend of classroom,

simulation and online learning. The course includes

ACLS certification training and customized courses

that address key issues in family practice. Phase I

must be completed before beginning Phase 2 of

the program, which is conducted at the university

where the candidate is matched.

NEW Canadian Medicine Primer (CMP)

In 2014, we developed and piloted a new, custom

immersion program for sponsored visa trainees and

clinical fellows. The Canadian Medicine Primer is a

two-and-a-half week program that was created to

enhance the skills and knowledge of these trainees

and facilitate the successful transition to residency

training and postgraduate education in Ontario.

In addition to customized sessions to meet their

unique needs, the trainees and fellows are

integrated into PRP sessions. The primer was

launched in 2014 with an expansion of the program

tracking for 2015.

Evolving to Meet Changing Needs

In 2011, we embarked on a three-year periodof program renewal spurred by shiftingdemographics, emerging trends in medicaleducation and feedback from students,lecturers and program directors at Ontario’smedical schools. Here are some of thehighlights.

Creating Online Modules and BlendedPrograms

The revamp started with the launch of ahybrid program (classroom, experiential andonline learning modules). The blendedprogram greatly increased learner autonomyand the flexibility of scheduling and resulted ina shorter, integrated program. At the sametime, the curriculum was enhanced to focusmore on the CanMEDs framework and clinicaland system issues.

Increasing the Role of Physicians

The next series of changes greatly increasedthe role of physicians in programs. Academicconsultants were appointed to provide inputand guidance in developing our transitionprograms. The Academic Advising Programwas created to provide trainees with a facultypoint of contact and to allow them to receiveindividualized feedback on their performancein various assessments.

Developing Integrated Skills Checks

Skills checks were introduced to provide anassessment of candidates’ knowledge andskills at the beginning and end of theprograms. The checks are offered in an OSCE format using standardized patients and physician examiners.

As the transition programs expanded in depthand breadth, so did the skill checks. In 2013,skills check assessments moved beyondmedical, clinical and communication issues toinclude ancillary services and broader, legal,ethical and health system issues.

With assistance from our Exam Unit, theEducation Unit developed a rigorousassessment process and customizedtemplate to ensure that the data generatedby the skills check process are more valid andreliable. The improved skills checks providerealistic encounters and a more authenticbenchmark of learners’ skills at the beginningof the program so that any gaps can beaddressed through the curriculum.

Introducing Physician Facilitators

In 2014, our education programs achievedanother milestone by including physicianfacilitators in the skills checks and small groupsessions. In the small group sessions, three to

EDUCATION PROGRAmS


“We are very pleased with the program renewal milestones we have achieved so far.The evaluations have been very positive and our advisors report that learners are muchmore enthusiastic and engaged than in the past. I believe that I can speak for ouracademic consultants and partners when I say that it has been a very exciting journey.”

Marie Rocchi Director, Education Programs

six trainees have simulated encounters withstandardized patients several times per weekas part of the experiential component of thecourse. In the past, the sessions werefacilitated by standardized patients, but nowphysicians facilitate and lead the sessions,providing individualized feedback in asupportive environment. In another newdevelopment this year, education programsused patients from our own StandardizedPatient Program, rather than outsourcing thatservice.


EDUCATION PROGRAmS

WHAT OUR ACADEMIC CONSULTANTS

Are Saying: “CEHPEA’s orientation programs really help IMGs

make the transition to postgraduate education and

adapt to the many different environments they will

be training in.”

Dr. Fok-Han Leung

Staff Physician, St. Michael's Hospital

Assistant Professor, Department of Family and

Community Medicine

Associate Director, Assessment and Evaluation

Postgraduate Family Medicine Program

University of Toronto

“CEHPEA does not focus solely on medical, clinical

and communications issues. We also want the

trainees to think about related services and broader

legal, ethical and health system issues. We really

want to help them prepare for the real issues they

will face when they train and later practice

independently.”

Dr. Shobhan Vachhrajani

Senior Resident

Division of Neurosurgery

University of Toronto

“As program director and clinical preceptor for

Family Medicine at Western University, I have

experienced the benefits of CEHPEA’s transition

programs, both on an individual and organizational

level.”

Dr. Eric Wong

Associate Professor, Department of Family Medicine

Academic Director, Distributed Education Network

Schulich School of Medicine and Dentistry

Western University

Simon Nguyen, one of our exam coordinators, helps toprepare candidates for the education program skills check.

BUILDING CAPACITY FOR RAPID GROWTH

In 2013/2014, we focused on strengthening our organizational structures and building capacity tosupport the new business areas we created and the expansion of our existing programs and assessments.We continued to invest in human resources, expanded the Standardized Patient Program, updated ourtechnology and increased our psychometric services.

Assessment:

IENCAP

Assessment:

CELBAN

NewAssessment:

COO EE

New

Education:

CMP

Education:

OTPC

Education:

PRP

Assessment:

NAC ExamSP Program

Our Staff

+90%

+90%

+45%

New

Our Staff


In the past year, our competencyassessments experienced unprecedentedgrowth:

• The Internationally Educated NursesCompetency Assessment Program(IENCAP) increased from 610 candidates in2013 to 950 in 2014.

• The National Assessment Collaboration(NAC) examination volumes increased from678 in 2013 to 1,161 in 2014.

• A new examination for internationallyeducated optometrists was developed andpiloted with all the specialized training andequipment that entailed.

We introduced customized programming forsponsored visa trainees and clinical fellowsand continued to improve our transitionprograms for international medical graduates(IMGs).

A major focus in the past year has been onstrengthening the organizational structure,breaking down silos and continuing to invest inhuman resources to provide opportunities forstaff development, growth and advancement.We have put in place many other activities andprograms to build the organization’sinfrastructure and capacity, including:

• Expanding the Standardized Patient Program

• Updating our technology

• Increasing psychometric services

Expanding the Standardized PatientProgram

Over the last three years, our StandardizedPatient (SP) Program has tripled in size tobecome the second largest program inCanada. The program includes a diverse poolof approximately 1,200 standardized patients– men and women aged 14 to 75, of allphysical types, ethnic groups andbackgrounds – as well as five full-time trainersand a manager.

“CEHPEA staff members have a wealth ofknowledge in the area of communication skills and they are experts in the use of patient simulation – an area of expertise that is not available in many places in Ontario,"says Dr. Marcus Law, chief examiner atCEHPEA.

The SP Program is a key resource for ourorganization. As we develop newassessments and education courses, and see

BUILDING CAPACITy


WHAT ARE

StandardizedPatients?Standardized patients are people who have been

trained to portray patients in various medical

situations in a consistent, standardized manner.

While some are actors, many are not. They learn

cases based on real patients, simulating their

physical signs in simulated interviews and

examinations by health care professionals. Patient

simulations are a core component of Objective

Structured Clinical Examinations (OSCEs). They also

play a vital role in training programs by providing

candidates with feedback and the opportunity to

practice clinical skills in a safe training environment.

continued growth in our existing offerings, ourSPs and trainers have successfully adapted tomeet the needs of each program.

The new optometry assessment presentedsome unique challenges for our SP program.Assessment experts, trainers and SPs neededto understand and prepare for the physicalinteractions and extensive array of equipmentthat the exam requires. SPs also needed to bescreened in advance by practicingoptometrists to ensure their suitability toparticipate and to rule out actual diseases ofthe eye.

“The optometry exam is very different fromother assessments that we have previouslyrun,” says Fergus O’Connor, manager of theSP Program. “Optometry was a whole newfield. We had to learn about new medicalterminology, order extensive equipment andidentify and solve unique issues withstandardized patients.”

This year, instead of relying on outsourcedSPs, we drew on expertise of our own SPs tosupport the Education Program. Thispresented another learning curve for theteam. Rather than being “super standardized”as they are in assessments, SPs in oureducation programs observe and providefeedback for candidates, providing valuableinsight into each interaction and how it madethem feel. SPs and trainers relish theopportunity to work with physicians in aneducational rather than assessment setting.The learners also enjoy the experience, andthis aspect of the program received verypositive feedback from candidates.

“Although it has been a challenging year withlots of new programs, it has been worth it,”says Fergus O'Connor. “It certainly has been

very exciting to work here and rewarding tosee the direction we are going and how ourprogram has grown substantially.”

Increasing Psychometric Services

Our organization has always taken great careto evaluate and validate the results of thetests we design and administer. Ourassessment and psychometric teams worktogether to create new tests while constantlyfine-tuning existing examination content, testadministration and test results based onestablished evaluation best practices.

We developed and administered the firstcompetency-based tests for IMGs seekinglicences to practice in Ontario. Given ourexperience and expertise, we have had theopportunity to play a leadership role in thedesign of the blueprint and tests for theNational Assessment Collaboration (NAC), the national competency-based screeningexamination developed and administeredunder the auspices of the Medical Council of Canada (MCC). Arthur Rothman, ourpsychometrician consultant, was appointedas the first chairman of the NAC TestCommittee to lead this initiative and continuedto be actively involved as our representativeon the committee.

With the launch of the NAC examination and anew assessment for internationally educatednurses (IENs), our psychometric teamexpanded with the hiring of two additionalstaff psychometricians. The expanded teamwill assure the quality of the results of highertest volumes for the NAC and the College ofNurses of Ontario (CNO), and thedevelopment of assessments for new clients,including the College of Optometrists ofOntario (COO).


BUILDING CAPACITy

The psychometric team also plays a role in our orientation courses for IMGs enteringresidencies in Ontario. The team scores andanalyzes the results of skills checks thatassess candidates’ competency at thebeginning and end of the orientation,providing valuable feedback to the studentsand residency program directors.

Updating Our Technology

In 2013/2014, we embarked on a mission toconsolidate, simplify and upgrade internetservices, servers and Wi-Fi networks tosupport the organization’s expansion andachieve efficiencies. Our Wi-Fi networks cannow accommodate 250 to 300 concurrentconnections, ensuring sufficient bandwidth tosupport our learners and to enable ourmovement towards more automated datacapture during examinations.

As well, the switch from a cable internetconnection to fibre-based services has beeninstrumental in bringing increased speed andfunctionality to the organization.

We also completed a major upgrade of audio-visual systems. Given the high-stakes natureof the many assessments occurring at ourfacilities, our recording system is a crucialcomponent in monitoring and defending theintegrity of the assessment process for qualityassurance purposes. We updated fromanalog to digital, enabling rapid playback,download and analysis of video footage.Better placement of microphones andinstallation of new cameras in the clinicalexamination rooms has improved audio-visualquality. In addition, the control centre hasbeen upgraded with 12 new monitors, withthe capacity to view the 78 exam roomsthrough 156 cameras.


BUILDING CAPACITy

Unique FacilitiesSPECIALLY DESIGNED FOR HIGH-STAKES ASSESSMENTS

Located in Toronto, we have the largest

assessment centre of its kind in Canada. The

facilities have a fully secure environment, specifically

designed for test security. All entries are passkey

protected, the testing floor has the appropriate

sequestering rooms for candidates awaiting

interviews. At full capacity, we can assess up to 216

candidates in one day. The facility is equipped with

78 individual exam rooms and two large group test

rooms. Each group test room can hold up to 72

candidates comfortably for multiple choice question

exams. Over 2,000 internationally educated health

professionals (IEHPs) are assessed by our exam

experts every year.


INTRODUCINGOUR TEAM

From left to right: Steven Watts, partner, KPmG LLP; Dr. Rocco Gerace, registrar, College of Physicians and Surgeons of Ontario;Brenda Zimmerman, director, Health Industry management Program, Schulich School of Business, york University; Dr. Kenneth Harris,director, Office of Education, Royal College of Physicians and Surgeons of Canada; John mcKinley, principal, JmcKinley Consulting andCloudIPHI; Anne Coghlan, executive director and CEO, College of Nurses of Ontario; and Sten Ardal, CEO, CEHPEA. Absent: George m. Thomson, senior director, International Programs at the National Judicial Institute and mark Walton, assistant dean,Post-Graduate medical Education, mcmaster University.

Board of Directors

* As of June, 2014

Our team consists of many different groups that work together to facilitate entry to practice forinternationally trained health professionals. Our Board of Directors provides governance and strategicdirection, including the creation of our Strategic Plan. The Executive Team is responsible for overseeing ourday-to-day operations and implementing the Strategic Plan. Our in-house experts create and validate ourexams and training programs, recruit and train external experts to meet the unique needs of each exam,administer large-scale examination and education programs, manage the Standardized Patient Programand maintain the smooth operation of our facilities. Meet the team!*


Executive Team

From left to right: Adrian Frisina, Alyssia Tassone andAfsheen Hasan.

Administration and Program Support

From left to right: Darko Giacomini, Arthur Rothman andSandra monteiro.

Psychometric Team

From left to right: Fergus O’Connor, Laura Segal, KarenHuszar, Tami Karakasis and Amy myers.

Standardized Patient Unit

From left to right: Sarah Wood, Will Truong, Sara Cecchetto,Simon Nguyen, Alex Tran, Raffa De Fazio and meng To.

Exam Unit

From left to right: Bill Quinn, Daniel Grossutti and Arjun Puri.

Operations Team

From left to right: Dr. murray Urowitz, Andrea Strachan,Debra Sibbald, Suzanne Bambrick and Sten Ardal.

SHARINGOUR EXPERTISE

In 2013/2014, we travelled across Canada and abroad to connect with others in the field of competencyassessment and education. Our in-house team and physician consultants shared their learning andexpertise in the presentations that follow. Contributors included Dr. Fok-Han Leung, Dr. Eric Wong,Dr. Shobhan Vachhrajani, Marie Rocchi, Debra Sibbald, Dr. Murray Urowitz, Arthur Rothman, Fergus O’Connor, Sandra Monteiro and Sten Ardal.


Meeting the Challenge of Changing Learner

Demographics in an Orientation Program for

Internationally Educated Medical Graduates:

A Blended, Individualized, Physician-Led Learning

Approach. Canadian Conference on Medical Education,

Quebec City, April 2013

Academic Advising for International Medical

Graduates: A Systematic, Individualized Model.

Canadian Conference of Medical Education, Quebec,

April 2013

Validation of a Clinical Examination for

Internationally Educated Nurses (IENs).

Association for Medical Education in Europe, Prague,

August 2013

14th IMG Symposium (Display Booth). Canadian

Resident Matching Service, Toronto, January, 2014

Competence Assessments of Internationally

Educated Nurses (IENs) Successfully Evaluate

Eligibility for Canadian Licensure Examination.

Ottawa Conference, Ottawa, April 2014

The Impact of a Transition Program for International

Medical Graduates on Communication and Clinical

Skills. Ottawa Conference, Ottawa, April 2014

Effectiveness of an Individualized Academic Advising

Program for International Medical Graduates.

Canadian Conference on Medical Education, Ottawa,

April 2014

IEN Competency Assessment Program.

8th Annual Partners in Education and Integration of IENs

Conference, Toronto, May 2014

Competence Assessments Successfully Evaluate

Eligibility of Diverse Internationally Educated Nurses

(IENs) for Canadian Licensure Examination.

Association for Medical Education in Europe, Milan,

September 2014

Strengths and Challenges of OSCEs (Workshop).

Canadian Network of National Associations of

Regulators Conference, Montreal, October 2014

Perspectives in Competency Assessment.

The first annual Touchstone Institute symposium brings

together experts from a variety of backgrounds to

explore theories, challenges and future trends in

competency assessment. MaRS Collaboration Centre,

Toronto, February 2015

Our New Name

Since 2007, CEHPEA has been providing excellence in competency evaluation. In January2015, we will become Touchstone Institute, a name and identity that reflects our view to a

progressive future. As Touchstone Institute, we look forward to further expanding andenhancing our services, working with regulators, our partners and specialists to facilitate

entry to practice for internationally trained health professionals.

touch-stone : \ˈtəch-ˌstōn\

noun : a standard or criterion by which something is judged or recognized

synonyms : criterion, standard, yardstick, benchmark, barometer, bellwether, litmus test, measure, point of reference, norm, gauge, exemplar, model, pattern

80 BLOOR STREET WEST, SUITE 902 • TORONTO ON m5S 2V1 • P 416-924-8622 • F 416-924-8921

EmAIL • [email protected]

MAILTO:[email protected]

Documents

CENTRE FOR THE EVALUATION OF HEALTH PROFESSIONALS … · Expanding Our Reach CENTRE FOR THE EVALUATION OF HEALTH PROFESSIONALS EDUCATED ABROAD PROGRESS REPORT 2013–2014 ... volume