MILESTONES, EPAS, NAS…AND OTHER ACGME JARGONCommittee on Graduate Medical Education

September 24, 2012

Sara LP Ross, MD

Objectives

To discuss the Next Accreditation System and what is known about how that will look in 2019

To discuss the system of trainee evaluation – Milestones and EPAs

NAS: Next Accreditation System

Goals Complete realization of the

Outcomes promise Free up “good programs”

to innovate Assist poor programs in

improving Reduce burden of

accreditation Establish and implement

milestones to better track program and institutional performance

Provide accountability to the public

July 2012: Seven initial core specialties/RRCs begin NAS training

July 2013: NAS officially begins; seven specialties “go live”; remaining specialties begin training

July 2014: All specialties/RRCs using NAS

NAS: Next Accreditation System

10-year self-study visit model: next visit 2019 for all Peds department programs Structure, Resources, Core processes, Detailed processes, Outcomes “What is your plan for the next 10 years to improve”

Residents will submit a confidential consensus list of five strengths and “opportunities for improvement” (OFIs) the residents wish to discuss Site visitor will share strengths, but will only share OFIs if residents give

permission (makes residents feel more connected to the site visit)

Annual program surveillance Performance indicators for each specialty developed by a “community of

educators” within the specialty Annual Resident Survey

Core elements of the competencies Levels of performance Core methods of assessment

Annual Faculty Survey Case Log Data

NAS: Next Accreditation System Institutional 18 month review Ongoing creation of Milestones Programs will get a letter annually stating

that they meet all “performance indicator thresholds”

May be placed on “accreditation with warning” at any point during the 10-year cycle May warrant an “immediate visit” If problems not fixed during a given window of

time, program may be placed on probation

Site Visits

Structure of the visit: 2 site visitors Brief meeting with PD Resident and faculty interviews Meeting with DIO Meeting with PD

PIF Elimination – YES (most likely) More focus on strengths of programs

Competencies

Competence: the ability to do something successfully (Oxford Dictionary of English)

Competencies: broad, general attributes of a good doctor With attempt at evaluation they get widdled

down to detailed skills/activities In the end don’t really reflect the original

meaning of the general competency Competence = Attribute Activity= Element of professional work

Entrustable Professional Activities(EPAs)

Units of work that may be awarded a more or less formal qualification at the moment when supervisors confirm the trainee is ready to assume responsibility for such activities

Entrustable Professional Activities (EPAs)

Which critical professional activities cover the relevant competencies of the profession?

How can supervisors learn when to entrust such activities to the trainee?

“Trust reflects a dimension of competence that reaches further than observed ability. It includes the real outcome of training – the quality of care”

Part of essential professional work

Require specific knowledge, skill and attitude

Lead to recognized output of professional labor

Confined to qualified personnel

Be independently executable within a timeframe

Be observable and measurable in its process and outcome (well done or not well done)

Reflect one or more competencies to be acquired

EPAsDomains ofCompetency

Competencies Milestones

EPA/Competency Matrix

Viewpoint: Competency-Based Postgraduate Training: Can We Bridge the Gap between Theory and Clinical Practice?ten Cate, Olle; Scheele, Fedde

Academic Medicine. 82(6):542-547, June 2007.DOI: 10.1097/ACM.0b013e31805559c7

EPAs

May be acknowledged formally as a “statement of awarded responsibility” (STAR)

Five levels of proficiency1. Has knowledge2. May act under full supervision3. May act under moderate supervision4. May act independently5. May act as a supervisor and instructor

Expected Levels of Confidence

Time to achieve STAR in a specific EPA dependent on:

The EPA

The working environment

The trainee

The clinical teacher

EPA Mapped to Competencies/ Subcompetencies

Milestones

Developmental roadmap for the competencies and subcompetencies

Observable developmental steps moving residents from novices to experts/masters

Means of restructuring competencies into a measurable rubric of six domains of clinical competency

EPAsDomains ofCompetency

Competencies Milestones

EPA Milestone Level I Level II Level III Level IV Level V

Serve as the primary admitting pediatrician for previously well children suffering from common acute problems

Patient Care: Gather essential and accurate information about the patient

Gathers too little info or exhaustively gathers info following a template regardless of patient's chief complaint. Recalls clinical info in the order elicited, with ability to gather, filter, prioritize, and connect pieces of info.

Clinical experience allows linkage of signs and symptoms of a current patient to those encountered in previous patients.

Advanced development of pattern recognition leads to the creation of illness scripts which allow information to be gathered while it is simultaneously filtered, prioritized and synthesized into specific diagnostic considerations.

Well-developed illness scripts allow essential and accurate info to be gathered and precise diagnoses to be reached with ease and efficiency when presented with most pediatric problems.

Robust illness scripts and instance scripts lead to unconscious gathering of essential and accurate info in a targeted and efficient manner when presented with all but the most complex or rare clinical problems.

Patient Care: Provide transfer of care that ensures seamless transitions

Demonstrates variability in transfer of info from one patient to the next. Frequent errors of both omission and commission.

Uses a standard template for the info provided during the handoff. Unable to deviate from that template to adapt to more complex situations. May have errors of omission or commission. Neither anticipates nor attends to the needs of the receiver of info.

Adapts and applies a standardized template, relevant to individual contexts, reliably and reproducibly, with minimal errors of omission or commission. Allows ample opportunity for clarification and questions.

Adapts and applies a standard template to increasingly complex situations in a broad variety of settings and disciplines. Ensures open communication, including but not limited to read-backs, repeat-backs and clarifying questions

Adapts and applies the template w/o error and regardless of setting or complexity. Internalizes the professional responsibility aspect of handoff communication.

Medical Knowledge: Demonstrate sufficient knowledge of the basic and clinically supportive sciences appropriate to pediatrics

Does not know or remember the basic content knowledge of common pediatric problems and illnesses

Understands the basic content knowledge of pediatrics, but is still learning to apply it to clinical situations

Understands the basic content knowledge of pediatric practice, and is able to synthesize and apply it in a clinical situation

Able to analyze and evaluate knowledge in a way that allows the generation of a meaningful differential diagnosis and can develop meaningful clinical management plans

Learns from experience; analyzes a situation, evaluates what worked well and what did not, and creates, adapts, or extrapolates info appropriately to new clinical situations

What milestone levels equate to different levels of proficiency?

Who determines you can practice the EPA independently going forward?

Minimum standards for advancement/graduation?

Where Are We in Pediatrics?

January 2012 The Pediatrics Milestone Project (51) September 2012 Selection of 21 Pediatric Milestones to

be

reported on semi-annually

- Requests for additional sites to study/ develop each of these milestones

Fall/Winter 2012 Program development of evaluation tool July 2013 Implementation of Milestone reporting

Where Are We in Pediatrics?

January 2012 The Pediatrics Milestone Project (51) September 2012 Selection of 21 Pediatric Milestones to

be

reported on semi-annually

- Requests for additional sites to study/ develop each of these milestones

Fall/Winter 2012 Program development of evaluation tool July 2013 Implementation of Milestone reporting

References

Carraccio C, Burke A. Beyond competencies and milestones: Adding meaning through context. J of Grad Med Ed. 2010;2(3):419-422.

ten Cate O. Trust, competence, and the supervisor’s role in postgraduate training. British Medical Journal. 2006;333:748-751.

ten Cate O, Scheele F. Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Acad Med. 2001;82:542-547.