Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Mark A. Mattos, MD,
Michigan Vascular Center-Michigan State University
Flint, Michigan
Advanced Vascular Surgery Surgical Skills and
Simulation Assessment Program
Making A Case for a National Examination,………or Not
Presented at the 2017 APDVS Spring Meeting, March 31, 2017, Rosemont, Illinois
O.W. Brown, M.D.
William Beaumont Hospital - Oakland University
Royal Oak, Michigan
Disclosures
• Nothing to Disclose
Fitness to Practice
Definition
When someone who has the skills, knowledge
and character to practice their profession
safely and effectively.
Fitness to Practice – USA
• Cognitive Competence
– Determined by written and oral examinations
objectively administered by the Vascular
Surgery Board
= Board Certification!!
Certification in Vascular Surgery
= Technical Fitness to Practice?
Cognitive vs.Technical Competence
Cognitive competence is not a guarantee of technical skills competence
Operative case volume alone
may not determine technical competence
• 2 different domains
Fitness to Practice - Technically
Practical Requirements
• Competent open and endovascular skills
• Expert knowledge of conduct of procedure and
technical steps
• Display ability to handle high-stress/high-risk
scenarios without panic and with calm dispassion
• Display ability to create and implement contingency
planning
• Display to ability communicate during procedure
• Display appropriate decision making skills
Fitness to Practice - Technically
• Technical Skills Competence in 2017
– Determined solely by subjective assessment
from faculty members of the vascular trainees
parent institution
Making the Case……
Objective accurate assessment of open and
endovascular skills of senior vascular trainees
is essential for determination of
fitness to practice
Furthermore,………..
We may not be allowed to rely on cognitive
competence (Board Certification) alone to
determine Fitness to Practice
Skills Assessment as Part of Certification
• Interventional Cardiology
• Interventional Radiology
• Vascular Medicine
Are You Ready to Practice Vascular Surgery?
January 7-9, 2016, January 12-14, 2017
Marcia & Eugene Applebaum Surgical Learning Institute
William Beaumont Hospital, Royal Oak, Michigan
AVSSSAP
• Initial results regarding performance, assessment and training potential have been favorable from trainees, faculty and industry
• Herein, we summarize the 2-yr program results
• Determine if the program could identify and assess those specific skills necessary to determine fitness for practice of senior vascular trainees
2016-2017 Participants
***
*
*
*
*
**
**
*
**
**
*
*
**
*
*
*
*
*
*
**
*
*
*
*
**
*
**
*
*
*
*
*
*
• 47 senior vascular trainees
• 60 vascular surgery faculty • 35 institutions
• 28 industry partners
Vascular Surgery Trainees
• 32 Senior Vascular Fellows (PGY-7)
• 14 Senior Vascular Residents (PGY-5)
• 1 Junior Vascular Fellow (PGY-6)
• 33 males, 14 females
2017 Vascular Faculty Proctors
• 35 board-certified vascular surgeons
Program Overview
• 31 assessment stations
• 12-15 simulated procedures
– 30-60 minute sessions
• 522 total simulated procedures
• 2088 individual assessment metrics
– 4 metrics per simulated procedure
Simulation ModelsAAA repair (tube graft) Maquet; W.L. Gore
CEA with patch closure Maquet; W.L. Gore
Femoral-Popliteal artery bypass Maquet; W.L. Gore
Hemodialysis access (AVF) Maquet; W.L. Gore
Mismatched E-S anastomosis Bard Vascular; Cryolife; Co-Author (MAM);
Groin Dissection, Femoral Endarterectomy LifeLike BioTissue
EVAR Endologix; Medtronic; W.L. Gore
TEVAR Medtronic; W.L. Gore
Carotid artery PTA and Stenting Abbott; Cordis; Mentice
Renal artery PTA and Stenting Abbott; BSCI; Cordis; Mentice
Iliac artery PTA and Stenting Abbott; BSCI; Cordis; Medtronic Mentice
SFA PTA, Stenting, Atherectomy Abbott; BSCI; Cordis; Medtronic; Mentice
Percutaneous closure device Abbott; Cordis
Videotaping Sessions B-Line Medical
6 Simulated Open Vascular Procedures
9 Simulated Endovascular Procedures
Assessment Methodology
• 1 faculty to 1 trainee
• Faculty are ‘passive’
1st assistants
• Faculty are ‘silent’
expert assessors
• Immediate feedback
Requirements for Proficiency
1. Global Rating Scale Score:
1. >80% of maximum
2. Global Rating Summary Score
1. Level 4 (proficient) or Level 5 (advanced)
3. Critical Task Performance:
1. 100% completion of all critical skill tasks
Global Rating Scale
(Score >80% maximum)
Global Rating Summary Ranking
Critical Task Performance
(100% achievement)
**
**
Proficiency DataSimulated Procedures Global Rating Scale Global Rating Summary Critical Task Performance
AAA Repair 38/49 (77%) 36/47 (76%) 14/47 (29%) L
CEA with patch closure 42/45 (93%) 31/33 (94%). 22/36 (61%)
Femoral-Popliteal artery bypass 40/43 (93%) 38/42 (90%) 15/46 (33%)
Hemodialysis access (AVF) 38/41 (93%) 30/32 (94%) 27/40 (68%) H
Groin Dissection, Fem Endart. 15/18 (83%) 13/17 (76%) 12/19 (63%)
Mismatched E-S anastomosis 32/34 (94%) H 38/39 (97%) H 12/30 (40%)
EVAR 33/41 (80%) 30/40 (75%) 9/24 (38%)
TEVAR 27/42 (64%) 28/38 (74%) 16/27 (59%)
Carotid artery PTA and Stenting 18/36 (50%) L 11/29 (38%) L 11/31 (35%)
Renal artery PTA and Stenting 20/32 (62%) 21/30 (70%) 11/29 (38%)
Iliac artery PTA and Stenting 33/40 (82%) 31/39 (79%) 17/34 (50%)
SFA/Pop PTA and Stenting 26/33 (79%) 25/30 (83%) 11/30 (37%)
SFA PTA, Stent, Atherectomy 5/7 (71%) 3/4 (75%) 2/5 (40%)
Totals 367/461 (80%) 335/420 (80%) 169/398 (42%)
Identifying Trainee Errors
• Heparin administration
– Dosing errors or omissions in 8%
• Incorrect Clamping/Unclamping Technique
– Incorrect sequencing or application occurred in 10%
• Poor use of first assistant
– 25% received a GRS task score of 1 or 2
• Incorrect or unsafe knot tying technique
– 15% received a GRS task score of 1 or 2
Open Vascular Skills Performance Issues
• Aortic Anastomotic Inefficiency (>15”, >2 leaks)
– 42% proximal aortic anastomoses (end-end)
– 19% of distal aortic anastomoses (end-end)
Open Vascular Skills Performance Issues
• Anastomotic Inefficiency (>15”, >2 leaks)
– 61% of proximal femoral anastomoses (end-side)
– 46% of distal popliteal anastomoses (end-side))
Open Vascular Skills Performance Issues
• Anastomotic Inefficiency (>15”, >2 leaks)
HD access anastomoses (end-side)
5 of 23 (22%)
Mismatched anastomoses (end-side)
13 of 19 (68%)
Open Vascular Skills Performance Issues
• Patch Closure Inefficiency (>25”, >2 leaks)
Overall: 6 of 41 (15%)
Carotid Endarterectomy:
5 of 24 (20.8%)
Femoral Endarterectomy:
1 of 17 (5.9%)
Endovascular Skills Performance Issues
• Loss of Wire Access
– 14 of 88 (16%)
• Excessive Contrast Volume
– 44 of 137 (32%)
• Excessive Fluoroscopy Time
– 13 of 138 (9%)
• Prolonged Procedure Time
– 26 of 38 (19%)
• Improper angiographic views– 25%-33% received a GRS task score of 1 or 2
Reduced Faculty Reporting Errors
• Total Assessment Errors: 20.5% 12.8%
– Global Rating Scale – 8.6%
– Global Rating Summary – 12.6%
– Critical Task Performance – 8.3%
Simulator Performance
• Endovascular simulator
– 10-15% software malfunction reboots
– Inability to record or loss of performance data
– Variability between simulators
– Lack of proper type or variety of equipment
• Open vascular simulator
– 5-10% malfunction
– Insert connective problems
– Tissue integrity dysfunction
– Flow-based related issues noted
Trainee Survey Results
Educational Content and Format
My open vascular skills were accurately assessed 86%
My endovascular skills were accurately assessed 75%
Immediate faculty feedback was helpful and improved learning 100%
Having different faculty perform assessments was valuable 100%
Having a vascular skills assessment program would be useful in
my training program
93%
I would recommend this program to other vascular surgery
residents and vascular surgery fellows
100%
The goals and objectives of the AVSSSAP were clearly explained 100%
The goals and objectives of the AVSSSAP were achieved 100%
Summary
AVSSSAP provided objective assessment of the following:
Open Vascular Skills:
1. Knowledge of procedural conduct/technical steps of procedure
2. Retraction and exposure
3. Vessel preparation
4. Anastomotic performance (vascular forceps handling/needle driving)
end-end, end-side, patch repair
large, medium, small (20mm to 3mm)
surface, shallow, deep (0” to 8”)
5. Knot tying performance
surface, shallow, deep (0” to 8”)
large, moderate, small (3-0 to 7-0)
Summary
AVSSSAP provided objective assessment of the following,
Endovascular Skills:
1. Knowledge of procedural conduct/technical steps of procedure
2. Guidewire, catheter and sheath selection
3. Nonselective/selective guidewire, catheter and sheath placement
3. Performance of diagnostic angiography/contrast utilization
4. Performance of balloon angioplasty
5. Performance of stent deployment
6. Performance of aortic endograft selection and deployment
Summary
• Most senior vascular trainees (but not all) appeared to be technically prepared to perform certain vascular procedures in a proficient and independent manner
• Variable measures of performance by trainees
• Faculty assessment performance improved, most likely due to repeated exposure to the assessment format of the program and increased interest in objective-based skills assessment
• Assessment metrics are not validated
Conclusion
• 2-yr data suggests continued value for this type of advanced skills assessment program.
• Ultimate value to program directors remains to be determined
• A small but defined number of inherent adverse simulator-specific tissue and computer software design flaws continue to occur
• Unclear if current simulators can provide the necessary advanced fidelity for the implementation of a liability-free national vascular skills competency exam.
• Alternative options for objective assessment is offered
Recommendations and Alternative Options
• Simulation Summit– 2-day Meeting
– Division Chairs, Program Directors, SVS, APDVS, Regional Society leadership
– Identify specific advanced skills and metrics to be used to assess technical competence and fitness to practice
– Identify what simulators/skills trainers are available
– Industry leaders/simulation companies invited
– Simulation companies to provide simulators/skills trainers
– Division Chairs, Program Directors, Educational Leaders to perform procedures on simulators
– Create a final document identifying metrics for validation, models to be used for achievement of national vascular surgical competence examination
– Begin discussion with simulation companies to produce simulators and skills trainers
Recommendations and Alternative Options
• Regional Skills Assessment Programs
– Society-based Assessment Programs
– All senior vascular surgical trainees must attend program
– Requirement for trainee to be able to apply for board certification
– Full objective assessment with feedback occurs at program
– Full report of trainee performance provided to Program Director
– Program Director assumes responsibility for ensuring trainee is
competent/proficient for all procedures being tested for
– Letter from Program Director goes in trainees file indicating
completion of objective testing and documentation of competency
Survey Results
Thank You!