REFORMATION inSURGICAL EDUCATION &TRAINING

Stephen Tobin

DEAN of EDUCATION

Royal Australasian College of Surgeons

IMELF Calgary 2013

REFORMATION

● ‘… the action or process of reforming an institution or practice …’

● From ‘reformare’ = shape again

● The Reformation (the Church 1517)● Re-formation = the action or process of

forming again • * NEW OXFORD DICTIONARY OF ENGLISH

SURGICAL EDUCATION & TRAINING

● RACS commenced 1927-1930● ‘Excellence in surgical education &

practice’● Training in hospitals (employers)● Framework of SET principles● Collaboration with Speciality Societies● Fellowship exam (SET5-6)● Professional development role as well

The structure of the RACS program

● The College is accredited as the training provider

● The College provides curriculum resources, particularly for the non-technical competencies

● Each of the nine surgical specialties develop and deliver much of their curriculum in the regions and hospitals, and at specialty meetings

SET 2007…..

● Structured program from PGY 3-4● Effectively PGY3-6+● 5-year programs (5-7)● Absence of official surgical framework for

clinical years before SET program

● Workforce issues ● IMGs

The structure of the RACS program

● RACS as an organisation – diversity & number

Figures at the end of December 2012 Australia and New Zealand

Specialty Trainees Fellows

Cardiothoracic Surgery 34 182

General Surgery 419 1731

Neurosurgery 53 229

Otolaryngology Head and Neck Surgery 81 497

Orthopaedic Surgery 236 1419

Plastic and Reconstructive Surgery 83 440

Paediatric Surgery 22 104

Urology 111 410

Vascular Surgery 42 191

Totals 1081 5203

The structure of the RACS program

● RACS as an organisation – size and distribution

Issues with SET – reform?

● Medical students: surgical societies● Numbers of graduates● Lack of structure PGYs● Selection● Supervision – feedback, assessment● Fellowship exam success % less● Competent not confident Fellows

MEDICAL STUDENTS

● Well-developed ‘surgical societies’■ All 22 medical schools

● Articulate● Connections – locally, College● Up to 25% interest

● Courses, evenings, draft agreements….

GRADUATE NUMBERS

● 22 MEDICAL SCHOOLS● 55% graduate-entry● Nomenclature MD increasing● Numbers 1800 (2005) >> 3800 (2017)*● Vocational stream jobs less (est 2800 now)*

● Urban v. regional & rural● IMGs

• *MTRP 16TH REPORT , JULY 2013• Australian figures; NZ better match

J-Doc : New for early years

● Suite of resources / e-portfolio● Assist with pathway to surgery (SET)● Mapped to AustralianCurriculumFramework*

■ For young doctors in PGY1-2

● Develop to novice-intermediate levels of the 9 RACS Competencies

● Provide well-developed doctors for SET■ Or other (procedural) specialties

• *3RD EDITION, 2012, www.curriculum.cpmec.org.au

Bringing this together : First profile

12

Education across the continuum

● Developing expectations of behaviour and performance

Education across the continuum

Bringing it all together

Social tools/APPS

Web Linked Resources

• The PGY Curriculum• Passing the GSSE

Examination• Register for the GSSE

examination• Links to College

documents (policies/competencies)

• Self Assessment• Surgical SkillsBox• Practice Bank MCQs• Clinical Decision

Making• Goal Setting

e-Learning

PRE-VOCATIONAL LANDING PAGE

Presentations

External Web Links

• Register for Skills Training Courses

• Apply to SET

• How to pass the GSSE examinaton

• Video tips

Program resources…

Digital College Vision

Welcome John CitizenHome > My Page

Update My Details

Welcome John CitizenHome > My Page

Update My Details

News ePortfolio Self Assessment Progress

News feed, any upcoming events..

15%5%

30%

50% Courses completedThe Morbidity Audit and Logbook Tool(MALS)

My Learning Modules

Course Name Description Cost

Goal Setting

Clinical decision making

Examination Module

Surgical Skills tool box N/A

N/A

50

100

Text

Text

Text

Text

My Tools and Resources

Description Resource/Tool Description

Description

Text

Text

Text

Skills Training

External Anatomy Course

College Standard Documentation

Free Apps Text

View

View

View

My Subscriptions

Description Resource/Tool Description

Text

Text

Text

Text

Fax Mentis and Surgical News

Social Tools Yes No

Yes No

Yes No

Buy

Re-Attempt

Complete

Buy

Buy

ePortfolio

SET selection

● Curriculum vitae (15-30% weighting)■ Points■ Prerequisite jobs*

● References: 2 or more…6-8 (27-60%)■ Some closer to work-based assessment■ Features of MSF-360

● Interview (25-45%)■ Use of the above to rank for interview

• *some year of application (to come)• *others must have been completed

Summary of significant selection-assessment relationships

Examinations DOPS MiniCEX End of Term Assessments

SSE Gen-eric

SSE Spec-ialty

Spec-ific

Clin-ical

DOPS1

DOPS

2

DOPS

3

DOPS

4

DOPS Mean

Mini-CEX

1

Mini-CEX

2

Mini-CEX

3

Mini-CEX

4

Mini-CEX

Mean

ETA1

ETA2

ETA3

ETA4

ETA Mean

n 339 292 337 252 204 88 42 253 254 199 82 33 256 277 297 148 139 312

CV Neg Neg Neg

RR

Inter-view

                                     

Total Sel-

ection

P < 0.05 P < 0.01 P < 0.05

What about selection?

● British study compared 1990 cf late 90s● Compared regional vs.hospital/surgeon selection

● No difference between ‘performance’● Dropping-out; length of time; GMC

● Review BJS 2012*■ No discriminators■ Psychometric testing some role

● Not statistically significant• *Note Leading Articles series 2013 (100th)

SET PROGRAM - assessment

● Evaluation forms● LogBooks● MiniCEX used by 6 (usually early years)● DOPS used by 7 (as above)● MSF – 360 noted, not used● Purpose of assessment● Signing off for FEX exam.

Collins JP, ANZ J Surg June 2013

SUPERVISION

● “Good supervision is difficult”

● Need to improve quality, numbers

● Academy of Surgical Educators■ ‘professional development of educators’

● Foundation course for educators/supervisors

● Authentic ‘signing-off’● EPAs

What about the program?

● Academic enhancement (<10%)● RESEARCH – before or during SET● Masters programs – Australia-NZ, Uni Edinburgh

● Personal leave (2-5%)● Probation (2-5%)● Dismissal (1%)

RACS data 2009-2012

Comparable

● Time-expired may increase● As Fellowship pass rate has declined

Fellowship (FEX) examination

● Signing-off is vitally important● FEX tests :

■ Medical and technical expertise■ Decision-making and judgement

● Recent close-marking■ Documentation

● Recent decline in success rates■ 81% - 2010 : 63% - May 2013 ■ Variation across states & NZ■ Variation between specialties

Evaluation of SET – current survey

● Entry level varies● Need to develop WBA approach● Working hours● Presence of ‘Fellows’● Signing off?● FEX approach

■ Surveys of trainees, supervisors, new Fellows

New Fellows

● Competent level across competencies● What done this mean?● Need for support at transition

● Matching to positions● Overseas work● Post FRACS ‘Fellowships’● Work in hospital unit or (private) practice

Reformation…

● SET is grounded in the community● Hospitals, Specialty societies, College, Urban& Rural

● Excellence >> befit community needs● DISTRIBUTION OF FELLOWS

● College covers trainees and Fellows● EDUCATION & PROFESSIONAL DEVELOPMENT

● NEW J-Doc program starts the approach● RACS COMPETENCIES

Reformation could mean…

● Better quality applicants● Better selection● Better supervision & WBA● Appropriate “signing-off”

● For some, this could be earlier

● FEX matches competent practice● Support in workplace to start● Lifelong learning – e-portfolio/CPD (WBA)

Reformation needs…

● Evidence –base & evaluation● Good supervision● Recognition-support of supervisors● Motivated trainees● Competency assessment with meaning

■ EPAs , training to match profession (work)

● Leadership & awareness of surgical education■ International influences■ THANK-YOU