EBP meeting Dublin, 4th June 2016

Towards a European Diploma in general paediatrics

Prof. em. Jean-Christophe Mercier, Chair EBP

Dr. Robert Ross-Russell, Chair TCG

Agenda 1. Approval of the minutes of previous EBP meetings (Bratislava & Brussels).

– Should we assess completion of training in general paediatrics or specialised paediatrics?

– Should we limit the assessment to a single Knowledge-based MCQs exam?

– Three options were discussed in Bratislava:

• Sharing the RCPCH long lasting expertise

• Setting up a Working Group of National experts in education and assessment

• Or relying upon EAP volunteers?

2. How to improve lisibility of Paediatrics by UEMS CESMA?

3. How to implement training assessment in paediatrics?

4. Accreditation of new forms of CME-CPD activities by the EACCME

5. EAPS scientific meeting in Geneva, October 2016

Conclusions of the EBP meeting Bratislava, 29th May 2016

• There is an urgent need to organise a ‘European diploma in general paediatrics’.

• Three options: – To work along with the RCPCH? – To build up a WG of National Boards of Paediatrics? – To do it by ourselves? If 25 EAP delegates write 10 MCQs for

each of the 50 chapters of the Euromastercourse textbook...

• EAP/paediatric section of the UEMS should focus on education & assessment and play a growing role within the UEMS family.

European survey in 2014 – 26 exams

John Boorman – presented at UEMS/CESMA meeting in Innsbruck (A)

Written part hold in 54 centres 2,700 candidates – 12 different languages

Oral exams - ~800 examinees 28 years of experience

Agenda 1. Approval of the minutes of the previous EBP meeting (Bratislava).

– Should we assess completion of training in general paediatrics or specialised paediatrics?

– Should we limit the assessment to a single Knowledge-based MCQs exam?

– Three options were discussed in Bratislava:

• Sharing the RCPCH long lasting expertise

• Setting up a Working Group of National experts in education and assessment

• Or relying upon EAP volunteers?

2. How to improve lisibility of Paediatrics by UEMS CESMA?

‒ CESMA meeting at the Domus Medica Europaea in Brussels, 12th Dec., 2015

‒ CESMA meeting in Bucarest, 7th May, 2016

3. How to implement training assessment in paediatrics?

4. Accreditation of new forms of CME-CPD activities by the EACCME

5. EAPS scientific meeting in Geneva, October 2016

http://www.uems.eu/uems-activities/harmonization-of-medical-training/cesma

• Guiding principles: – Steering Committee composed not only

of medical specialists, but also of educational and assessment experts

– Committment and time frame (2 yrs)

– Assessment rules: • Detailed eligibility criteria

• Availability of a curriculum, textbook

• Description of assessment method

• Description of the score calculation

• Pass-fail decision criteria

• Appeal procedure

– Number of permitted attempts

– Assessment venue

– Duration of assessment • MCQs no more than 3 hours

• Other forms no more than 1 hour

– Archiving for at least 5 years

– Assessment fees

– Assesment size (from 50 to >150 candidates)

• Before the assessment: – Establishing a Steering Committee

– Producing a curriculum or syllabus

– Determining the format of the assessment

– Setting up eligibility criteria

– Communicating assessment rules

– Providing timely notification of examination venue

• During the assessment: – Registration of candidates, safety of examiners

– Availability of appropriate answer sheets, tablets

– Sufficient personnel for supervision

– Allow reliable score calculation

– Allow pass-mark setting

• After the assessment: – Dissemination pass-fail results to candidates

– Conuselling of failed candidates

– SWOT (Strengths, Weaknesses, Opportunities, Threats)-analysis

• UEMS-CESMA offer to appraise new Exams

http://www.uems.eu/uems-activities/harmonization-of-medical-training/cesma

• MCQ formats supported by UEMS-CESMA – Type A MCQs ‘single-best answer’ (negative

marking ill-advised)

– Type K MCQs ‘multiple independent true/false’ abandonned in many assessments

– Other types

• Advantages of using MCQ-based assessment of knowledge: – Electronic scoring

– Low costs

– Correct answers predetermined, not subjective

– Well-adapted for different locations and times

• Important points to consider: – MCQs and correct answers must be kept secure

– Hardware used in assessment should be reliable

– Question bank should be updated

• Features of a ‘good’ MCQ – A well-constructed MCQ consists of a

positively worded statement or ‘stem ’ followed by a clealy expressed question

– The ‘stem’ is derived from a specific item within the curriculum.

– The ‘stem’ is positively worded and focuses on a single concept.

– The correct answers and distractors should be of roughly equal text lenght

– Evidence should support correct and incorrect answers

– Each question should be self-contained

• Post-hoc analysis of performance

• Constructing the question bank

• Setting the pass mark

Writing good MCQs by Alfred Tenore – UEMS-CESMA workshop, Brussels 12th Dec 2015

European Diplomas Fees 2016 European Exams: the financial aspects • Intensive Care Medicine €1,100 • Plastic surgery €850 • Paediatric surgery €850 • Urology €800 • Orthopaedics €800 • Anaesthesia €790 • Neurosurgery €700 • Vascular surgery €700 • Hand surgery €600 • Neurology €600 • Nuclear medicine €500 • Pathology €400 • Physical rehabilitation €200 As compared with other Royal Colleges • Primary MCQs £315 • Primary OSCEs £580 Modulatory fees • ESCIM Members €300 – Non members €450 • General surgery €700 • But coloprotectomy +€700 • Ophtalmo surgery €400 • Glaucome speciality +€500

UEMS exams CESMA appraisal

Agenda 1. Approval of the minutes of the previous EBP meeting (Bratislava).

– Should we assess completion of training in general paediatrics or specialised paediatrics?

– Should we limit the assessment to a single Knowledge-based MCQs exam?

– Three options were discussed in Bratislava:

• Sharing the RCPCH long lasting expertise

• Setting up a consortium of National experts in education and assessment

• Relying upon EAP volunteers

2. How to improve lisibility of Paediatrics by UEMS CESMA?

3. How to implement training assessment in paediatrics?

4. Accreditation of new forms of CME-CPD activities by the EACCME

5. EAPS scientific meeting in Geneva, October 2016

How to implement training assessment in pediatrics?

EAP-RCPCH working group meeting on 19th Jan. 2016

RCPCH: • Prof. Neena Modi, President • Dr. Andrew Long, Vice-president (education) • Dr. Will Carroll, Chair, Written Examinations

Question Board Committee • Ms. Julia O’Sullivan, Director of Education

and Training

EAP/EBP • Prof. Jean-Christophe Mercier,

EBP chair • Dr. Robert Ross-Russell, TCG

chair

Summary notes of the RCPCH/EBP meeting London, 19th January, 2016

• The RCPCH feels it needs to evolve their current exam to include greater elements of primary care and mental health questions.

• The EAP would feel that a European exam might have a slightly different emphasis than the current RCPCH questions.

• Both, therefore, would want to develop any knowledge-based exam from the current model – which changes may well be in the same direction.

• Delegates at the EAP would need any exam to be distinct from the ‘MRCPCH’ for both political and practical reasons (i.e. membership of the RCPCH should not be mandatory!)

• Intellectula property in the current MRCPCH and the MCQs with the exam must remain with the RCPCH.

• Finally, we should look for any exam to be an electronic-based exam.

Robert Ross-Russell, 19th January, 2016

A guide to training recommendations in paediatrics

Medical qualification

Core training

• Common Trunk syllabus

Specialist training

• Primary care

• Acute Paediatrics

• Specialty Paediatrics

• ...

Completion of training

Assessment of competency Logbook maintenance

European Diploma

stefan.park@orzone.com

The RCPCH-EAP London meeting follow-up

To set-up a EBP MCQ examination steering committee? (JCM) • Jean-Christophe Mercier (F), EBP Chair &

links with UEMS-CESMA

• Robert Ross-Russell (UK), TCG Chair & links with the RCPCH and ERS MCQs Writing Committee

• Peter Hoyer (D), Link to the German education and GPEC, as an advisor

• Martin J White (IR), tertiary care

• Diego Van Esso (SP), primary care

• Corinne Wyder (CH), declined

• Who else?

To resume talks with the RCPCH Education Committee (RRR) • Andrew Long’s proposals (10th May, 2016)

• Possibility of a team of three attending the July Exam Boards in Kettering (?12th July)

• Sharing the Bob Dinwiddie’s guide to setting MCQ questions

• Neena Modi’s suggestion to keep ‘tight’ for the interim discussions in July

• To look for some dates in September/early October for a larger group from EAP and RCPCH to meet. Suggested dates: – Tuesday August 23 10:30-13:00 13:30-16:30 OR

– Wednesday August 31 10:30-13:00 13:30-16:30

• ?2018 for setting up an exam is ambitious, but potentially achievable

Agenda 1. Approval of the minutes of the previous EBP meeting (Bratislava).

– Should we assess completion of training in general paediatrics or specialised paediatrics?

– Should we limit the assessment to a single Knowledge-based MCQs exam?

– Three options were discussed in Bratislava:

• Sharing the RCPCH long lasting expertise

• Setting up a consortium of National experts in education and assessment

• Relying upon EAP volunteers

2. How to improve lisibility of Paediatrics by UEMS CESMA?

3. How to implement training assessment in paediatrics?

4. Accreditation of new forms of CME-CPD activities by the EACCME

5. EAPS scientific meeting in Geneva, October 2016

1 hr=1 AMA Credit point = 1 ECMEC

Agenda 1. Approval of the minutes of the previous EBP meeting (Bratislava).

– Should we assess completion of training in general paediatrics or specialised paediatrics?

– Should we limit the assessment to a single Knowledge-based MCQs exam?

– Three options were discussed in Bratislava:

• Sharing the RCPCH long lasting expertise

• Setting up a consortium of National experts in education and assessment

• Relying upon EAP volunteers

2. How to improve lisibility of Paediatrics by UEMS CESMA?

3. How to implement training assessment in paediatrics?

4. Accreditation of new forms of CME-CPD activities by the EACCME

5. EAPS scientific meeting in Geneva, October 2016

Conclusions • EBP priority is to organise paediatric training assessment in Europe,

– Because many other (28/43) UEMS section have already exams.

– It also helps unify national training programs and favour assessments.

• We are hopefully on the right track, – If we can benefit from both the UEMS-CESMA experience and RCPCH help.

– However, a huge investment both of time and financing is required.

• UEMS-CPE/CPD activity is enormously increasing: – The EAP/EBP role is to accredit every paediatric Live Educational Events.

– While full reciprocity between AMA, the Canadian, British, and European accreditation systems would ensure worldwide benefits.

• EAPS meetings are now reaching maturity (10 years anniversary!): – Despite a decrease in submitted abstracts, sessions and workshops are

well-maintained.

– Involvement of a growing number of European Scientific Societies?

– EAPS 2018 should take place in a very attractive European city!