Embed Size (px)
Citation preview
ABR Exams of the Future Examination Details
Impact on Training
Duane Mezwa MD FACR
ABR Trustee GI Section
Professor and Chair, Radiology
Oakland University William Beaumont SOM
Disclosure
The material presented today best
represents the current thoughts and
processes in developing the Core and
Certifying exams of the ABR.
What We’ve Done Until Now
Two comprehensive written examinations
Physics (first offered September 2nd year)
Diagnostic (first offered September 3rd year)
Candidates must pass both to qualify
One comprehensive oral examination
First offered in June of 4th year
11 categories examined
Written Examinations:
Details Physics
Diagnostic, Nuclear, Radiobiology
Recall and application of physics concepts
No associated images
Diagnostic ~20 MCQs in each category
Evaluated information recall
No associated images
Scored as a whole (not by category)
Written Examinations:
More Details Criterion-based exams
Theoretically, failure rate could be 0-100%
Failure rates (first-time takers):
Physics: 10-15%
Diagnostic: 8-10%
Both exams offered retake opportunities
Oral Examinations:
Details 25 minute oral examinations
Centered on specific case images
Minimal fact recall
Focus on observation, synthesis, management
Communication skills and judgment paramount
Specific diagnosis less important than
logical approach
Oral Examination:
More Details Criterion-based exam
Complex grading system Each candidate discussed in detail
Potential for raising marginally failing scores
Could fail 1-3 categories, but must re-test
Statistics for first-time takers: Pass: 85%
Fail: 5-7%
Condition: 8-10%
Criticisms of Examination Process:
Written Exams Physics examination better suited to physics
students, not radiologists
Rarely relevant to everyday practice
Diagnostic exam:
No images
Fact retrieval (trivia?) only
Not scored categorically—candidates could pass
knowing nothing about (fill in the blank)
Criticisms of Examination Process:
Oral Exams Subjective
No two tests the same
For some candidates, pathologically stressful
Exam reliability incalculable
Expensive
For ABR: ~400 examiners X 5 days
For candidates: single inconvenient location
Response by the ABR:
Changes in Exam Structure Qualifying (Core) exam:
Comprehensive: Covers all of diagnostic radiology Includes physics, patient safety, noninterpretive
Categorical: candidates must pass all sections Candidates and training programs receive data Opportunities for retakes: every six months
Knowledge level: basic to intermediate Appropriate for ending third year resident Timed for June of third year (except 2013) About 40% fact recall, 60% higher-level evaluation
R1 R2 R3 R4 Fellowship/
employment
Internship
12 mos 12 mos 12 mos 12 mos 12 mos 12 mos 3 mos
Core Exam Certifying
Exam
Core Exam in Diagnostic Radiology
Will be given September 30-October 4, 2013
Subsequent years will be offered in late June
Last full Oral exam in June, 2013
Covers all of diagnostic radiology
Comprehensive, categorical exam
Candidates must pass all categories
What Are These Categories?
Organ
systems
MSK, Thoracic, GI, Urinary, Neuro, Pediatrics, Cardiac,
Reproductive/Endocrine, Mammography, Vascular
Modalities CT, MRI, RF/Fluoro, Nucs, Interventional, Ultrasound
Fundamentals Physics, Safety
Exam Goals: Core Overarching goal: to protect the public by determining that
individual candidates have attained competence in basic diagnostic radiology
Specific goal: to create examinations that are relevant to current radiologic practice
Specific goal: to make a reliable pass/fail decision about candidate performance in each category For this high-stakes exam, reliability requires ~60 questions/category
Emphasis on unique importance of Physics content = 110 questions
Breast Cardiac GI MSK Neuro Peds Thorax
Repro /
Endo Urinary Vascular Q#
CT 60
IR 60
MR 60
NM/Molecular 60
Rad/Fluoro 60
US 60
Physics 110
Safety 60
Q# 60 60 60 60 60 60 60 60 60 60
Core Exam: Content
Image-rich (unlike current written)
~40% fact recall (like current written)
~60% higher level (like current oral)
Differential diagnosis
Management
Mostly MCQs, may include new question types
Example: Extended Matching For each patient whose clinical and imaging information is shown, select the most
appropriate diagnosis from the list below. Each option may be used once, more
than once, or not at all.
A. Focal nodular hyperplasia B. Liver cell adenoma
C. Cavernous hemangioma D. Inflammatory pseudotumor
E. Pyogenic abscess F. Fungal abscess
G. Nodular focal fat H. Biliary cystadenoma
I. Lymphoma J. Solitary metastasis
K. Intrahepatic cholangiocarcinoma L. Hepatocellular carcinoma
M. Fibrolamellar carcinoma
1: 35 year old woman who underwent sonographic evaluation for mild abdominal discomfort.
She was referred for MR imaging to characterize a solitary liver mass seen on that ultrasound.
Images are obtained 30 seconds (A), 70 seconds (B) and 1 hour (C) after administration of
gadobenate intravenously.
A B C
A. Focal nodular hyperplasia B. Liver cell adenoma
C. Cavernous hemangioma D. Inflammatory pseudotumor
E. Pyogenic abscess F. Fungal abscess
G. Nodular focal fat H. Biliary cystadenoma
I. Lymphoma J. Solitary metastasis
K. Intrahepatic cholangiocarcinoma L. Hepatocellular carcinoma
M. Fibrolamellar carcinoma
Key = A
2. 60 year old man with abdominal pain. Imaging performed elsewhere showed
a liver mass, and he is referred for CT imaging to characterize it. Images are
obtained before (A), 20 seconds after (B), 50 seconds after (C), and 5 minutes
after (D) intravenous administration of iodinated contrast material.
A. Focal nodular hyperplasia B. Liver cell adenoma
C. Cavernous hemangioma D. Inflammatory pseudotumor
E. Pyogenic abscess F. Fungal abscess
G. Nodular focal fat H. Biliary cystadenoma
I. Lymphoma J. Solitary metastasis
K. Intrahepatic cholangiocarcinoma L. Hepatocellular carcinoma
M. Fibrolamellar carcinoma
Key = L
A coronal CT image obtained in a
patient with ascites is shown.
Label the following structures:
a) Left subphrenic space
b) Lesser sac, inferior recess
c) Lesser sac, superior recess
d) Transverse mesocolon
e) Gastrohepatic ligament
f) Morison’s pouch
g) Left paracolic gutter
h) Root of intestinal mesentery
a b c d g e h f
Core Exam 2013+
Nuts and Bolts ~660 items
Two half day sessions during a single week
Examination center(s)
One large Chicago center
Smaller center in Tucson
ABR commitment: distributed exam by 2018
Condition exams June/October
5440 N. Cumberland Ave., 4½ miles and the second train stop
on the CTA Blue Line from O’Hare Airport. ~30,000 square feet
ABR FastPass Palm Vein Biometrics
Results: Test Scoring
Each category (18) will be scored separately
Score will be based on pre-test assessment of item difficulty (Angoff method)
Each test item will be scrutinized for its performance (p-value, r-value), and adjustments made for problem items
Three possible results: Pass (all 18 categories passed)—qualify for Certifying
Fail (6 or more categories failed)—must retake whole exam
Condition (1 to 5 categories failed)—must retake those categories failed
Core Examination:
Impact on Training
Candidates must be exposed to all basic
diagnostic radiology by end of third year
“Core anxiety” and review sessions will
occur early in third year
Content of review sessions will change
Core Exam: Review Sessions
Physics content Should focus on practical applications
Physics resource: rsna.org/education/physics.cfm
Diagnostic content 40% fact recall—even though image-rich, reasonable to
study topics prevalent on previous written exam
60% resembles oral boards—standard board review
Resource: Core study guide on theABR.org
Resource to come: practice exam—90 minute simulation
Practice Exam
• A taste of the full exam
• Shortened version of all sections
• A WIP
• To be on line by end of 2012
• 90 minutes -- to simulate the real exam
RSNA/AAPM Physics Modules
47 complete
30,081 enrolled, 10,467 completed
Popular modules: Basic Radiation Biology
Basic concepts in Radiography
Radiation Effects
Basic Principles of Nuclear Magnetic Resonance
Atoms, Radiation and Radioactivity
Radiation Risks
BEST Way to Prepare
Come to work and
PARTICIPATE Fully
Piloting the Core Exam
First core pilot: May, 2011
Aims:
Nuts and bolts: software, image quality, time
allotted, interface
Psychometric item data: p, r values; reliability
First Core Pilot
Population: Residents taking Oral Exam
Incentive: Passing score on electronic exam
could raise condition in that category
Caveat: Would not raise if 68 on Oral
Would not raise if failed ≥ 4 categories
Only six categories: MSK, Neuro, VIR, Peds,
Breast and Cardiac
Core Pilot: Results
1117 (81.5%) took exam
Neuro 26%, Breast 7%, rest ~18%
Post-test survey: good questions, enough
time, appropriate difficulty, interface good
2nd Core Pilot
• Population: 4th year residents taking Orals
• Was given at this years May Oral exams
• Two categories were offered
• One was chosen by ABR and other was
candidates choice
• Can be used to raise 2 sections in Orals
2nd Core Pilot
• 1340 (92.5%) took exam
• Neuro and Nucs,12%;Thoracic,GI,MSK9%
• Peds, VIR 8%;GU, Breast 7%
• Repro/Endo 5%
Next Expanded Core Pilot • Full Exam to be given to PGY4 residents in ~ June 2013
• Will be given at both Chicago and Tucson
• Will try to accommodate all residents
• Feedback will be given to help guide exam to be taken in
September.
• Does NOT Count in place of real exam
• For now a one time only event in 2013
Core Exam: How Are We Doing?
13 item-writing committees (physicist
embedded in each)
Scorable units submitted: 8076
Scorable units approved: 5198
All study guides completed and revised X 1
Exam Week Logistics
After the Core
Candidates can decide (within constraints of
their chosen training program) on their
future practice
General vs. subspecialized
Which subspecialty(ies)
Begin the process of continuous learning
Certifying Exam in Diagnostic Radiology
Will be first administered in October, 2015
Contains five parts (but questions will appear in random order)
Non interpretive skills Things every physician
should know
Essentials Things every diagnostic
radiologist should know
Clinical Practice Areas
(3 self-selected CPAs)
Things this specific
radiologist should know
Clinical Practice Areas (CPA) Twelve categories
Organ system: MSK, Cardiac, Thoracic, GI, Urinary, Neuro, Peds, Breast
Technology: US, VIR, Nuclear Radiology
General
Candidates can choose any combination of CPAs
Items will vary in both difficulty and scope Level 1: fundamental
Level 2: advanced
Candidates selecting a CPA more than once will receive a higher proportion of level 2 items
Certifying Exam: Content (CPA)
Emulates clinical practice Will include normals and variants
Will include important findings outside chosen area
Appropriateness, clinical vignettes, management decisions
Item types Familiar: MCQs, extended matching
Unfamiliar*: Structured reporting, script concordance testing
*Examples posted on ABR Website >1 year before use
Certifying Exam: Goals
To confirm candidate has acquired and
maintained necessary skills to practice
independently
Dual role
Final ABR Certification Exam
First exam of practice-based learning (like MOC)
Certifying Exam: Nuts and Bolts
~300 items
5-hour exam (one half-day)
Exam center(s): Chicago, Tucson
Exams offered October/February
Certifying Examination:
Scoring Each of the five modules scored separately
Passing standard established prior to test administration (Angoff method)
All problem items analyzed, adjustments made
Two possible results: Pass: all 5 modules passed—certified 10 years
Fail: any module failed
Retakes possible every six months
Certifying Exam:
Impact on Training
After Core passed, some candidates may
begin self-directed specialty training
Some part of 4th year available
Extent depends on program needs
Supplement, but will not replace Fellowship
May impact first 3 months of practice
Certifying Examination:
Candidate Preparation Good professional practice
Medical judgment
Communication skills
Lifelong learning
Critical thinking
Self-assessment
Recognize the six competencies?
Study Guides
Certifying Exam: Resources
Non-interpretive skills and Essentials
Study guides to be posted on theABR.org
Clinical practice modules
Study guides at theABR.org
CME, SAMs
Practice-based learning
Society-produced educational modules
Certifying Exam: How Are We Doing?
15 item-writing committees
Submitted scorable units: 7404
Approved scorable units: 4863
