Maintenance of Certification – Part IV PQI Practice Quality Improvement The Summit August 19, 2006

Maintenance of Maintenance of Certification Certification – –

Part IVPart IVPQIPQI

Practice Quality Practice Quality ImprovementImprovement

The SummitThe Summit

August 19, 2006August 19, 2006

The Summit - The Summit - ObjectivesObjectives

Learn more about Quality Improvement Learn more about Quality Improvement and the ABR’s PQI Programand the ABR’s PQI Program the tangible Benefits of PQIthe tangible Benefits of PQI

Learn about ideas-projects-plans of Learn about ideas-projects-plans of participating organizations/societiesparticipating organizations/societies

Identify ways for societies to engage in Identify ways for societies to engage in PQI to help their membersPQI to help their members how to engage, support, challenge how to engage, support, challenge

RadiologistsRadiologists Beginning to develop a National Beginning to develop a National

DatabaseDatabase

Maintenance of Maintenance of CertificationCertification

Mandate of the American Board of Medical Mandate of the American Board of Medical Specialties Specialties

Goal to Goal to improve the quality of health careimprove the quality of health care through through physician-initiated learning and quality improvementphysician-initiated learning and quality improvement

Response to public expectation for Response to public expectation for physician physician competencecompetence

generation of new knowledge generation of new knowledge application in practice application in practice recognition of medical errorsrecognition of medical errors

To Err is Human: Building a Safer Health Care SystemTo Err is Human: Building a Safer Health Care System

- - Institute of Medicine, Institute of Medicine, 20002000

public concerns re underuse, overuse, misuse of health care in public concerns re underuse, overuse, misuse of health care in U.S.U.S.

MOC – a MOC – a physician-based responsephysician-based response to demonstrate, to demonstrate, ensure physician ensure physician competence competence and and continuing continuing competencecompetence

Clinical practice, delivery of health careClinical practice, delivery of health care

The General The General CompetenciesCompetencies – the Basis of Maintenance – the Basis of Maintenance of Certificationof Certification medical knowledgemedical knowledge patient carepatient care interpersonal & communication skillsinterpersonal & communication skills professionalismprofessionalism practice-based learning & self-practice-based learning & self-

improvementimprovement systems-based practicesystems-based practice

Maintenance of Maintenance of Certification - ABRCertification - ABR

From From certificationcertification toward toward demonstration of demonstration of competencycompetency in in practice of medicinepractice of medicine Responsibility ofResponsibility of

- physiciansphysicians- institutions/organizations in medical education, institutions/organizations in medical education,

continuing medical education/trainingcontinuing medical education/training ABMS identification of ABMS identification of componentscomponents of MOC of MOC

Part IPart I – Professionalism – Professionalism Part IIPart II – Lifelong Learning and Periodic Self- – Lifelong Learning and Periodic Self-

AssessmentAssessment Part IIIPart III – Cognitive Expertise – Cognitive Expertise Part IVPart IV – Practice Performance – Practice Performance

Maintenance of Maintenance of Certification Certification for Radiologists for Radiologists (DR, RO, RP)(DR, RO, RP)

MOC introduced for Radiology in MOC introduced for Radiology in 2005 2005

Formal implementation Parts I-IV: Formal implementation Parts I-IV: 20072007

Parts I-IIIParts I-III developed for radiology, developed for radiology, radiation oncology, radiation physicsradiation oncology, radiation physics

Part IVPart IV under development – ABR, under development – ABR, educational and professional societieseducational and professional societies


- Part IV- Part IV highlights competencies in medical highlights competencies in medical

practice, delivery of carepractice, delivery of care Practice Quality Improvement (PQI)Practice Quality Improvement (PQI)

practice-based focuspractice-based focus based upon metricsbased upon metrics includes action plan for quality includes action plan for quality

improvementimprovement engage physicians, engage physicians,

societies/organizations in Practice societies/organizations in Practice Improvement (PQI)Improvement (PQI)


- Part IV- Part IVFeatures of Practice Quality Features of Practice Quality

Improvement (PQI)Improvement (PQI) ProjectProject choose choose ProcessProcess learn learn ParticipateParticipate complete review complete review PlanPlan ongoing improvement ongoing improvement ProgressProgress document document PotentialPotential benchmarks, national benchmarks, national

databasedatabase

Practice Quality Practice Quality Improvement – Role Improvement – Role

of Societiesof Societies

George S. BissetGeorge S. BissetDepartment of RadiologyDepartment of Radiology

Duke University Medical CenterDuke University Medical CenterDurham, NCDurham, NC

oror

Why are you here?Why are you here?

American Board of American Board of RadiologyRadiologyMissionMission

““To To serve patientsserve patients, the public, and the, the public, and themedical profession. . .”medical profession. . .”

““By By certifyingcertifying that its diplomates that its diplomates have acquired, demonstrated, and have acquired, demonstrated, and maintainedmaintained a requisite standard of a requisite standard of knowledge, skill, and knowledge, skill, and understanding. . .”understanding. . .”

ABR TransitionABR Transition

OLDOLD

Physics ExamPhysics Exam

Clinical Exam Clinical Exam

Oral ExamOral Exam

NEWNEW

Physics ExamPhysics Exam

Clinical ExamClinical Exam

Oral ExamOral Exam


ABMS and MOCABMS and MOC

•• All 24 Boards committed to MOCAll 24 Boards committed to MOC

•• ABMS must approve each board’s ABMS must approve each board’s programprogram

Why “not” MOC?Why “not” MOC?

New administrative and financial New administrative and financial burdensburdens

May not improve patient careMay not improve patient care Self-assessment components – Self-assessment components –

rigor?rigor? Not fully defined (WIP)Not fully defined (WIP)

Why MOC?(Pragmatic)Why MOC?(Pragmatic)

All Boards – Time-limited certificatesAll Boards – Time-limited certificates All Boards – MOC participationAll Boards – MOC participation Provider privilegingProvider privileging Market forces pushing pay-for-Market forces pushing pay-for-

performanceperformance Federation of State Medical Boards Federation of State Medical Boards

– – “responsibility to public to “responsibility to public to ensure ensure competence of competence of physicians seeking physicians seeking relicensure”relicensure”

Why MOC?Why MOC?(Philosophical)(Philosophical)

We are all lifelong students of medicineWe are all lifelong students of medicinePerversityPerversityHypocrisyHypocrisy

Patient expectationsPatient expectations External regulators External regulators Prerogative of self-regulation Prerogative of self-regulation

(maintaining autonomy)(maintaining autonomy)Public accountabilityPublic accountability

MOC GoalsMOC Goals

Lifelong learningLifelong learning Self-directed learningSelf-directed learning

Based on needs assessmentBased on needs assessmentPractice specificPractice specific

Recognition of diversity of Recognition of diversity of practicepractice

Continuous quality improvementContinuous quality improvement

Benefits of Practice Benefits of Practice ImprovementImprovement

Better patient….Better patient….Care Care OutcomesOutcomesSafetySafety

Better practice-based learningBetter practice-based learning Quality of care puts us all on the Quality of care puts us all on the

same sidesame side

Why do we need PQI?Why do we need PQI?

Lack of documentation for Lack of documentation for quality of care quality of care for for most most important diseasesimportant diseases

Lack of resource evaluationLack of resource evaluation Huge variations in care at local, Huge variations in care at local,

regional regional and national levels and national levels Lack of outcome assessmentLack of outcome assessment

Pay-for-Performance Pay-for-Performance PushPush

Bridges to ExcellenceBridges to Excellence The Leapfrog GroupThe Leapfrog Group The Integrated Healthcare The Integrated Healthcare

AssociationAssociation Health Care Value InitiativesHealth Care Value Initiatives CMS/PremierCMS/Premier

How is PQI done?How is PQI done?

Select issues where change may be Select issues where change may be useful to useful to the the practicepractice

Incorporate different types of evidenceIncorporate different types of evidence Continuously refresh data during Continuously refresh data during

processprocess Utilize data to monitor departmental Utilize data to monitor departmental

progressprogress Sharing of results (national database)Sharing of results (national database)

Why do we need Why do we need database?database?

Need to know our baseline at a Need to know our baseline at a national national levellevel

Public disclosure of quality Public disclosure of quality measurement measurement data lead to data lead to improvements in quality of improvements in quality of carecare

What should Societies What should Societies do?do?

Design tools to support radiologists in their Design tools to support radiologists in their efforts to enhance quality of patient efforts to enhance quality of patient

carecare Strong physician leadership – develop and Strong physician leadership – develop and

execute radiologist-led initiatives execute radiologist-led initiatives appropriate to subspecialtyappropriate to subspecialty

Focus on evidence-based clinical guidelinesFocus on evidence-based clinical guidelines Form work groups to move process forwardForm work groups to move process forward

Choose topic to be measured based on importance

Create leadership-driven ad hoc committee

Review current established guidelines

Develop measures -Accurate and meaningful-Trackable over time-Universally available

Measure performance

Evaluate performance in relation to agreement with guidelinesInterventions

Remeasure performance

SummarySummary

MOC is essential for quality MOC is essential for quality healthcarehealthcare

Practice Quality Improvement is Practice Quality Improvement is a vital a vital component of MOCcomponent of MOC

Societies can assist with Societies can assist with development of development of PQI projectsPQI projects

MOC Component 4

DIAGNOSTIC DIAGNOSTIC RADIOLOGYRADIOLOGY

&&SUBSPECIALTIESSUBSPECIALTIESGary J. Becker, MDGary J. Becker, MD

PQI SummitPQI Summit

Chicago, Illinois, August Chicago, Illinois, August 19, 200619, 2006

ThenThen……Pass a test, earn a Pass a test, earn a lifetime certificatelifetime certificate

NowNow……Maintenance of Maintenance of Certification Certification depends on depends on continuing continuing competence…competence…

Competent Competent PhysicianPhysician

“ “Maintenance of competence Maintenance of competence should be demonstrated should be demonstrated throughout the physician’s throughout the physician’s career by evidence of lifelong career by evidence of lifelong learning learning and ongoing and ongoing improvement of practiceimprovement of practice.”.”

ABMS, September 1999ABMS, September 1999

““Competency is not Competency is not just just knowingknowing what to what to

do;do;

it is it is doingdoing it.” it.”11

1Blackwell et al. [DRAFT] Final report of the committee on recognizing new and emerging disciplines in internal medicine; ABIM 2006

MOC ComponentsMOC ComponentsEvidence of:

1: Professional standing1: Professional standing

2: LLL and periodic self-2: LLL and periodic self-assessmentassessment

3: Cognitive expertise3: Cognitive expertise

4: Evaluation and 4: Evaluation and improvement of improvement of performance in practiceperformance in practice

In MOC, Component 4 In MOC, Component 4 is where “the rubber is where “the rubber

hits the road”!hits the road”!

The The main main purposepurpose of MOC of MOC

is…is…

……to improve the to improve the quality of care we quality of care we

deliver individually deliver individually & collectively& collectively

To accomplish this…To accomplish this…

…we must …we must participateparticipate in in

Practice Quality Practice Quality Improvement (PQI) Improvement (PQI)

What Do We Need?What Do We Need?3 Essentials of Improvement3 Essentials of Improvement

WillWill –acknowledge defects exist; desire to –acknowledge defects exist; desire to improveimprove

IdeasIdeas –changes that will result in –changes that will result in improvementimprovement

ExecutionExecution –effectively apply changes; –effectively apply changes; sustainsustain

D. Berwick, MD, Institute for Healthcare Improvement

PQI projectPQI project: diplomate : diplomate focuses on an important focuses on an important

aspect of practice,aspect of practice,

……measuresmeasures performance, performance, then… then… plans,plans, improves,improves, & &

measuresmeasures againagain

Engage

men

t in

Engage

men

t in

proc

ess

proc

ess

Simple

PQI

Simpl

e PQI

Proje

cts

Proje

cts

Underst

andin

g of s

yste

ms-

Underst

andin

g of s

yste

ms-

base

d pra

ctic

e

base

d pr

actic

e

Best p

ract

ices

Best p

ract

ices

emer

ge

emer

ge

Regio

nal, n

atio

nal

Regio

nal, n

atio

nal

data

base

s be

gin

data

base

s be

gin

Pract

ice-

base

d

Pract

ice-

base

d

lear

ning,

lear

ning,

impr

ovem

ent

impr

ovem

ent

routin

e

routin

e

Data

shar

ing,

Data

shar

ing,

com

pariso

n

com

pariso

n

Robust

dat

abas

es,

Robust

dat

abas

es,

impr

ovem

ent

impr

ovem

ent

targ

ets

targ

ets

QUALITY O

F CARE

QUALITY O

F CARE

IMPROVES

IMPROVES

NATIO

NALL

Y!!!

NATIO

NALL

Y!!!

PQI top

ics

PQI top

ics

updat

ed

updat

ed

base

d on

base

d on

health

care

health

care

prio

ritie

s

prio

ritie

s

Years 1-Years 1-55

Years 6-Years 6-1010

PQI PQI TimelineTimeline

Where are we Where are we heading?heading?

Mission:Mission: “…establishes standards “…establishes standards for physician specialty for physician specialty certification and MOC…to certification and MOC…to support the public’s quest for support the public’s quest for safe, high-quality healthcare.”safe, high-quality healthcare.”

Role of ABMSRole of ABMS

MOC guidelinesMOC guidelines Timetable (deadlines)Timetable (deadlines) Oversight, feedbackOversight, feedback Approval of plans of member boardsApproval of plans of member boards ResourcesResources

BenchmarkingBenchmarking Advice and consultationAdvice and consultation Education (symposia)Education (symposia) Addressing issues common to all boardsAddressing issues common to all boards

-FSMB licensure verification-FSMB licensure verification

-Surveys, questionnaires-Surveys, questionnaires

-Software, especially to support Part 4-Software, especially to support Part 4

Role of ABMS in MOC Role of ABMS in MOC ProcessProcess

Applying ABMS Guidelines to Applying ABMS Guidelines to Radiology Radiology Every Diagnostic Every Diagnostic Radiology PQI Project MUST:Radiology PQI Project MUST:

Be relevant to diplomate’s practiceBe relevant to diplomate’s practice Be doable in practice settingBe doable in practice setting Be suited to trending over MOC Be suited to trending over MOC

cyclecycle Effect quality improvementEffect quality improvement Include these competenciesInclude these competencies**::

Patient carePatient care Practice-based learning and improvementPractice-based learning and improvement Systems-based practiceSystems-based practice

*May also include others

Participating in Diagnostic Participating in Diagnostic Radiology PQIRadiology PQI Select PQI projectSelect PQI project

Communicate selection to ABRCommunicate selection to ABR PW-protected reporting on ABR PW-protected reporting on ABR

diplomate Webpagediplomate Webpage Reporting follows templateReporting follows template >>1 PQI project in 11 PQI project in 1stst 10-yr cycle 10-yr cycle

(begins 01/2007)(begins 01/2007) Trended data collection Trended data collection >>3X/cycle3X/cycle Diplomate also reports on 6 Diplomate also reports on 6

competenciescompetencies

DR PQIDR PQI EXPECT CHANGEEXPECT CHANGE

PQI is under development (WIP)PQI is under development (WIP) ABR monitoring national healthcare priorities for ABR monitoring national healthcare priorities for

critical PQI opportunitiescritical PQI opportunities Incorporation of priorities into Part 4Incorporation of priorities into Part 4 Expert consensus evolves with evidenceExpert consensus evolves with evidence Regional/national normative databasesRegional/national normative databases

Participation in institutional CQIParticipation in institutional CQI

Accreditation status (JCAHO, VA, ACR, etc.) as basis for Accreditation status (JCAHO, VA, ACR, etc.) as basis for PQI projectsPQI projects

Practice-relevant SAMs; improvement planPractice-relevant SAMs; improvement plan

Random auditsRandom audits

To Develop ABR’s Part 4 To Develop ABR’s Part 4 ProgramProgram

ABR faced challenges of DR landscape, ABR faced challenges of DR landscape, larger healthcare picturelarger healthcare picture Practice diversityPractice diversity

Generalists & subspecialistsGeneralists & subspecialists Active practitioners, administratorsActive practitioners, administrators Consultative & direct patient careConsultative & direct patient care Hospital-based, office-based & mixed Hospital-based, office-based & mixed Radiology: no disease-specific focusRadiology: no disease-specific focus National healthcare prioritiesNational healthcare priorities

5 PQI Project Categories 5 PQI Project Categories for DRfor DR

Practice guidelines & technical Practice guidelines & technical standardsstandards

Referring physician surveysReferring physician surveys Patient safetyPatient safety Double reading Double reading Turnaround timeTurnaround time

Two PQI Project TypesTwo PQI Project Types

Type I: local levelType I: local level Physicians compare performance against Physicians compare performance against

own baselineown baseline Some comparison among peersSome comparison among peers Normative databases lackingNormative databases lacking

Type II: sponsored by national Type II: sponsored by national specialty society or organizationspecialty society or organization Regional or national database participationRegional or national database participation BenchmarkingBenchmarking FeedbackFeedback

Type II PQI Project ExamplesType II PQI Project Examples RADPEERRADPEERTMTM

Peer review of routine image interpretationPeer review of routine image interpretation

Summary stats for each participant by modalitySummary stats for each participant by modality

Summary data for facility by modalitySummary data for facility by modality

Data from all participating facilities by modalityData from all participating facilities by modality

ABMS’ CAHPS Peer SurveyABMS’ CAHPS Peer Survey11

ABMS’ Web-based Patient Safety ABMS’ Web-based Patient Safety ProjectProject Available mid-2007Available mid-2007

1Consumer Assessments of Health Plans Survey

PQI Timeline & Milestone PQI Timeline & Milestone TrackingTrackingDR DiplomatesDR Diplomates

Year of Cycle

What I must do each year of 10-year MOC cycle Submit report / attestation via the Personal Web Page

1

Select project and metric(s)Collect, analyze baseline data Yes

2Develop improvement plan Implement planBegin collecting improvement plan data

Yes

3

Complete collection of improvement plan dataAnalyze dataSummarize data

Yes

4

Modify improvement planImplement planBegin collecting improvement plan data

Yes

5 Continue collecting data Yes

PQI Timeline & Milestone PQI Timeline & Milestone TrackingTrackingDR DiplomatesDR Diplomates

Year of Cycle


6


Yes

7Refine improvement planImplement planBegin collecting improvement plan data

Yes

8 Continue collecting data Yes

9


Yes

10

Prepare final report of results and conclusionsSustain the gain of 1st cycleSelect topic for next cycle

Yes

PQI: PQI: Patient SafetyPatient SafetyTOPIC LIST (MUST CHOOSE ONE) TOPIC LIST (MUST CHOOSE ONE)

Safe use of iodinated RCMSafe use of iodinated RCM ALARAALARA MR SafetyMR Safety JCAHO Universal ProtocolJCAHO Universal Protocol National Patient Safety GoalsNational Patient Safety Goals NQF Safe Practices for Better HealthcareNQF Safe Practices for Better Healthcare Prevention of medication errorsPrevention of medication errors ABMS Web-based patient safety moduleABMS Web-based patient safety module Other (with ABR approval)Other (with ABR approval)

PQI: PQI: Double ReadingDouble ReadingMUST INCLUDE ALL 3 OF THESE ELEMENTS:MUST INCLUDE ALL 3 OF THESE ELEMENTS:

Study of error rate; objective of rate Study of error rate; objective of rate reductionreduction

Analysis of root causes; plan to Analysis of root causes; plan to minimizeminimize

Changes by 2Changes by 2ndnd reader: significance, reader: significance, time, costtime, cost

PQI: PQI: Turnaround TimeTurnaround TimeSUBTOPICS (Both):SUBTOPICS (Both):

Access: FROM exam/procedure Access: FROM exam/procedure requestrequest TO TO completioncompletion

Reporting: FROM exam/procedure Reporting: FROM exam/procedure completioncompletion TO TO final report availabilityfinal report availability

PQIPQIPractice Guidelines & Technical Practice Guidelines & Technical

StandardsStandardsREQUIREMENTSREQUIREMENTS

ACR Communication Guideline (select at least 1)ACR Communication Guideline (select at least 1)--Comparison reportsComparison reports

-Communicating preliminary/final report discrepancies-Communicating preliminary/final report discrepancies

-Communication of urgent/serious findings to referring physician-Communication of urgent/serious findings to referring physician

-Communication to self-referred patients-Communication to self-referred patients

Other Guidelines & Standards (select at least 1)Other Guidelines & Standards (select at least 1)-Widely accepted standard of national specialty/subspecialty -Widely accepted standard of national specialty/subspecialty

organization organization

-ACR: ~100 available-ACR: ~100 available

PQI: PQI: Referring Physician Referring Physician SurveysSurveys CAHPSCAHPS11 or institutional survey ( or institutional survey (>>50 50

referring physicians)referring physicians) All subtopics must be included in years All subtopics must be included in years

1-31-3-Exam/procedure access-Exam/procedure access

-Urgent examination and consultation-Urgent examination and consultation

-Satisfaction in the interaction-Satisfaction in the interaction

-Comments about professionalism-Comments about professionalism

-Report turnaround-Report turnaround

1Consumer Assessments of Health Plans Survey

ABR MOC: the 4 MOC Components & the 6 ABR MOC: the 4 MOC Components & the 6 CompetenciesCompetencies

Professional Standing

Lifelong Learning and Self-assessment

Cognitive Expertise

Practice Performance

Medical Knowledge

State board license requirements and actions

Documentation and completion of 500 CME credits. Minimum of 250 Category 1.

Achieve a passing score on the ABR cognitive exam.

Patient SafetyDouble ReadingPractice Guidelines

Patient Care State board license requirements and actions

Documentation: CME with review of new techniques and protocols.

Achieve a passing score on the ABR cognitive exam, which includes patient care content.

Patient SafetyDouble ReadingTurnaround TimePractice GuidelinesReferring Physician Survey

Interpersonal & Communication Skills

SAMs with emphasis on communications.


Professionalism State board license requirements and actions

SAMs content on professionalism.

General questions about ethics and charter on professionalism

Practice GuidelinesReferring Physician Survey

Practice-based Learning

& Improvement

Specific CME and SAMs developed for practice-based learning and improvement.

General questions about essential core knowledge and practice improvement principles.


Systems-based Practice

Specific CME and SAMs developed for systems-based practice.

General questions about CQI content.


Competencies Report for Diagnostic Competencies Report for Diagnostic RadiologyRadiologyDiplomate Name:____________ ABR ID:_________ Check one: 1Diplomate Name:____________ ABR ID:_________ Check one: 1stst

yr___ 5yr___ 5thth yr___ Final___ yr___ Final___ Professional

StandingLifelong Learning and

Periodic Self-AssessmentCognitive Expertise Evaluation of Performance

in Practice

Medical Knowledge CME required by state board for licensure

Practice-related exam questions

Patient Care Unrestricted licensure

Fundamental (“general content”) exam questions

Interpersonal & Communication Skills

Professionalism Unrestricted licensure

Practice environment-related exam questions

Practice-based Learning & Improvement


Systems-based Practice


Part 4: Roles for Part 4: Roles for Professional SocietiesProfessional Societies Educational courses, SAMs on Part 4Educational courses, SAMs on Part 4

Identify key PQI focus areasIdentify key PQI focus areas Develop new tools, guidelines, metrics, Develop new tools, guidelines, metrics,

project templatesproject templates How-to workshops for your membersHow-to workshops for your members Work with other societies on database Work with other societies on database

developmentdevelopment Deliver message: PQI is work-in-Deliver message: PQI is work-in-

progress!!! Expect change!!!progress!!! Expect change!!!

Practice Practice Performance Performance

Radiation Radiation Oncology Oncology

PQIPQIBruce G. Haffty, MDBruce G. Haffty, MD

Radiation Oncology MOCRadiation Oncology MOC10-Year Cycle10-Year Cycle

Professional StandingProfessional Standing Licensure-Maintained Throughout Licensure-Maintained Throughout

Life-long Learning and Self AssessmentLife-long Learning and Self Assessment CME-20 Hours/Year Total 200 HoursCME-20 Hours/Year Total 200 Hours Self-Assessment Modules-8 Required over 10 Self-Assessment Modules-8 Required over 10

YearsYears Cognitive ExpertiseCognitive Expertise

Examination-Year 8,9 or 10Examination-Year 8,9 or 10 Practice Performance Improvement Practice Performance Improvement

Projects (PQI)Projects (PQI) 3 Required Over 10 Years3 Required Over 10 Years

PQI ProjectsPQI ProjectsFundamentalsFundamentals

Individual Physician BasedIndividual Physician Based Patient Care RelatedPatient Care Related Measurable Endpoints/BenchmarksMeasurable Endpoints/Benchmarks

Evidence Based Guidelines, Consensus or Evidence Based Guidelines, Consensus or Peer comparisonPeer comparison

Action Plan/Improvement PlanAction Plan/Improvement Plan Baseline Assessment Baseline Assessment Improvement PlanImprovement Plan Follow-up assessment of the effect of Follow-up assessment of the effect of

improvementimprovement

PQI ProjectsPQI ProjectsFundamentalsFundamentals

Each of 6 Competencies Should be Each of 6 Competencies Should be assessed at least once during MOC assessed at least once during MOC cyclecycle Medical KnowledgeMedical Knowledge Patient CarePatient Care Interpersonal and communication skillsInterpersonal and communication skills Practice Based learning and Practice Based learning and

ImprovementImprovement ProfessionalismProfessionalism Systems Based PracticeSystems Based Practice

Radiation Oncology PQI Radiation Oncology PQI ProjectsProjects

Type I and Type IIType I and Type II Type IType I

Individual/Departmental/Institutionally Individual/Departmental/Institutionally DevelopedDeveloped

Should Address the fundamental Should Address the fundamental elements of a PQI program (Physician elements of a PQI program (Physician based, patient care, measurable based, patient care, measurable endpoints, Action Plan, etc)endpoints, Action Plan, etc)

Self AttestationSelf Attestation Subject to ABR Random AuditSubject to ABR Random Audit Creativity/Novel Approaches EncouragedCreativity/Novel Approaches Encouraged

Radiation Oncology-Type Radiation Oncology-Type I ProjectsI Projects

Institutional Quality Improvement Institutional Quality Improvement ProjectsProjects Physician-or Practice-Based Patient CarePhysician-or Practice-Based Patient Care Measurable outcomesMeasurable outcomes BenchmarksBenchmarks Action PlansAction Plans

Outcomes projects (Outcomes projects (i.e.,i.e., retrospective retrospective reviews)reviews) Should be related to patient care outcomesShould be related to patient care outcomes Physician basedPhysician based Action PlansAction Plans


Type I and Type IIType I and Type II Type IIType II Developed Developed externallyexternally by Professional by Professional

SocietiesSocieties Should Should addressaddress the fundamental the fundamental

elements of a PQI program (physician elements of a PQI program (physician based, patient care, measurable based, patient care, measurable endpoints, action plan, etc.)endpoints, action plan, etc.)

Requires Requires submissionsubmission to and to and qualificationqualification by ABRby ABR

Establishment of centralized databases, Establishment of centralized databases, pooled data for comparisons and pooled data for comparisons and benchmarks encouragedbenchmarks encouraged

Type II (Society Type II (Society Initiated)Initiated)

Submitted and “Qualified” by Submitted and “Qualified” by ABR in ROABR in RO

Must meet fundamental elements Must meet fundamental elements of ABMS PQI type programsof ABMS PQI type programs

Current examples:Current examples: ACR-ACR-RO - PEERRO - PEER ASTRO - ASTRO - PAAROTPAAROT ABS - under developmentABS - under development


Proposed model is as follows: Proposed model is as follows: Three Projects over 10 Year CycleThree Projects over 10 Year Cycle

Type II - at least Type II - at least one of the 3one of the 3 must must be Type IIbe Type II

Up to two of the three can be Type I Up to two of the three can be Type I - - Individual/InstitutionalIndividual/Institutional creativity is encouragedcreativity is encouraged

Radiation OncologyRadiation OncologyPQI Projects-Phase in for PQI Projects-Phase in for

MOCMOC 3 Projects over the 10-year Cycle3 Projects over the 10-year Cycle Goal of one project within each 3-Goal of one project within each 3-

year interval year interval For those diplomates whose time-For those diplomates whose time-

limited certificates expire within limited certificates expire within the next few years, PQI requirement the next few years, PQI requirement will be phased in will be phased in diplomates with TLC expiring in 2090 diplomates with TLC expiring in 2090

or 2010 will need to complete one PQI or 2010 will need to complete one PQI by December 2009 or 2010, by December 2009 or 2010, respectively)respectively)

Radiation Oncology Radiation Oncology Practice Quality Practice Quality

ImprovementImprovement This element of MOC remains the This element of MOC remains the

least developed and most difficult to least developed and most difficult to managemanage

Remains the component that can Remains the component that can have the greatest impact on one of have the greatest impact on one of the critical goals of the MOC the critical goals of the MOC program - program - Improvement in the Improvement in the Quality of Patient Care and Quality of Patient Care and OutcomesOutcomes


RO Objectives for the RO Objectives for the SummitSummit Discuss Fundamental Elements of PQI Discuss Fundamental Elements of PQI

ProgramsPrograms Discuss Type I and Type II ProgramsDiscuss Type I and Type II Programs Provide Guidance to Individuals/Practices for Provide Guidance to Individuals/Practices for

Developing Type I ProgramsDeveloping Type I Programs Provide Guidance to Societies for Developing Provide Guidance to Societies for Developing

Type II ProgramsType II Programs Encourage the Engagement of ABR Diplomats Encourage the Engagement of ABR Diplomats Discuss Options for Reporting, Documentation Discuss Options for Reporting, Documentation

and Validation of PQI Programsand Validation of PQI Programs

Practice Quality Practice Quality Improvement: The Improvement: The

Specifics for Radiologic Specifics for Radiologic PhysicsPhysics

Trustees, Radiologic Physics:Trustees, Radiologic Physics: G. Donald Frey, Ph.D. – Medical G. Donald Frey, Ph.D. – Medical Nuclear PhysicsNuclear Physics Richard L. Morin, Ph.D. – Diagnostic Richard L. Morin, Ph.D. – Diagnostic RPRP Bhudatt R. Paliwal, Ph.D. – Bhudatt R. Paliwal, Ph.D. – Therapeutic RPTherapeutic RP

Assoc Exec Director, Radiologic Assoc Exec Director, Radiologic Physics:Physics: Stephen R. Thomas, Ph.D.Stephen R. Thomas, Ph.D.

PQI Summit:PQI Summit:Additional Additional Considerations Considerations for Radiologic Physicsfor Radiologic Physics (1) (1)

Review the Challenges: Review the Challenges: Define how Define how PQI applies to radiologic physics PQI applies to radiologic physics and identify the potential role of and identify the potential role of PQI projects in promoting quality PQI projects in promoting quality improvement in RP practice.improvement in RP practice.

Call to Action: Call to Action: Encourage societies Encourage societies with a component of RP to with a component of RP to participate in the development of participate in the development of PQI programs for medical PQI programs for medical physicists. physicists.

PQI Summit:PQI Summit:Additional Additional Considerations Considerations for Radiologic Physicsfor Radiologic Physics (2) (2)

Establish a Working Environment: Establish a Working Environment: Create a framework wherein both Create a framework wherein both medical physicists and RP societies medical physicists and RP societies work together defining tools for work together defining tools for enabling medical physicists to enabling medical physicists to incorporate PQI programs as an incorporate PQI programs as an integral part of their practice. integral part of their practice.

Projections for the Future: Projections for the Future: Where do Where do we want to go? How will we get there? we want to go? How will we get there? What happens after we arrive?What happens after we arrive?

PQI for Radiologic Physics – PQI for Radiologic Physics –

The Unique Position of Medical The Unique Position of Medical PhysicistsPhysicists Only 2 of the 24 boards of the ABMS have Only 2 of the 24 boards of the ABMS have

non-physician members.non-physician members. ABMS Website description of Part IV: ABMS Website description of Part IV:

Directed toward evidence of evaluation of Directed toward evidence of evaluation of performance in practice for physicians.performance in practice for physicians.

The PQI program development in RP will The PQI program development in RP will be focused on the physicist as a medical be focused on the physicist as a medical professional who contributes to and professional who contributes to and supports patient care within the supports patient care within the healthcare system, while not having healthcare system, while not having primary responsibility for the patient.primary responsibility for the patient.

ABMS Guidelines for ABMS Guidelines for Part IV: Part IV:

Application to the Application to the Practice of Medical Practice of Medical

PhysicsPhysicsThe challenges The challenges

for for Radiologic Physics:Radiologic Physics:

What fits - What doesn’t What fits - What doesn’t (Quite)!(Quite)!

Selected Guideline Selected Guideline IllustrationsIllustrations

Guideline #1:Guideline #1: PQI programs: PQI programs: Phase-in, evaluate effectiveness, Phase-in, evaluate effectiveness, systematically improve, update systematically improve, update diplomates on development:diplomates on development:

RP – Ok in concept.RP – Ok in concept. Guideline #2:Guideline #2: Assessment process Assessment process

reflective of activities related to reflective of activities related to patients or patient care:patients or patient care:

RP – Ok under the label of patient RP – Ok under the label of patient care.care.

Selected Guideline Selected Guideline IllustrationsIllustrations

Guideline #6:Guideline #6: Assessment of patient Assessment of patient care focus on sampling of patients in care focus on sampling of patients in practice with a key disease or clinical practice with a key disease or clinical process.process.

RP – Needs adjustmentRP – Needs adjustment Guideline #12:Guideline #12: Assessment of Assessment of

physician performance begin during physician performance begin during residency and continue throughout residency and continue throughout practice.practice.

RP – Currently, the residency RP – Currently, the residency considerations are not applicable.considerations are not applicable.

The Evolving Nature of The Evolving Nature of PQI Projects for PQI Projects for

Radiologic PhysicsRadiologic Physics Recognition that operational PQI Recognition that operational PQI

projects for RP have not been finalized projects for RP have not been finalized at this point.at this point.

Importance of this Summit as a vehicle Importance of this Summit as a vehicle for open communication, brainstorming for open communication, brainstorming options, and establishing cooperative options, and establishing cooperative engagement in moving forward.engagement in moving forward.

Importance of initiating PQI projects Importance of initiating PQI projects even if understood to be even if understood to be interim/evolving - Getting RP interim/evolving - Getting RP Diplomates committed to the concepts Diplomates committed to the concepts and started down the pathway.and started down the pathway.

Society Initiation of PQI Society Initiation of PQI Projects for Radiologic Projects for Radiologic

Physics:Physics: American Association of Physicists in American Association of Physicists in

Medicine (AAPM)Medicine (AAPM)AAPM TG-127 on MOCAAPM TG-127 on MOCEducational CouncilEducational CouncilProfessional CouncilProfessional Council

ACMP, RSNA, ASTRO, ACR, SNM, ACMP, RSNA, ASTRO, ACR, SNM, …. ….

CAMPEP – Commission on CAMPEP – Commission on Accreditation of Medical Physics Accreditation of Medical Physics Educational ProgramsEducational Programs

PQI Principles for Radiologic PQI Principles for Radiologic Physics:Physics:

Framework for ConsiderationFramework for Consideration Choose 1 RP PQI project for the 10-year Choose 1 RP PQI project for the 10-year

cyclecycle Selected from a list of defined and Selected from a list of defined and

approved projectsapproved projects Data collected and reported at 2 time Data collected and reported at 2 time

points – potentially within the 4points – potentially within the 4thth and 7 and 7thth yearsyears

A normative peer review process to be A normative peer review process to be usedused

Diplomates identify/contact their own Diplomates identify/contact their own qualified reviewer who will use ABR qualified reviewer who will use ABR prescribed instruments for the evaluationprescribed instruments for the evaluation

PQI Projects under PQI Projects under consideration for consideration for

Radiologic Physics Radiologic Physics ABMS application (Dec 2005)ABMS application (Dec 2005)1.) Professional and Regulatory Guidelines:1.) Professional and Regulatory Guidelines:Performance based on fulfillment of physics Performance based on fulfillment of physics

component according to established component according to established criteria.criteria.

2.) Safety for Patients, Employees, Public:2.) Safety for Patients, Employees, Public:Performance based on fulfillment of Performance based on fulfillment of

established standards related to safety established standards related to safety program responsibilities.program responsibilities.

3.) Educational Activities:3.) Educational Activities:Performance based on documented Performance based on documented

evaluation according to national standards evaluation according to national standards for teaching.for teaching.

Evaluation of Scope of PracticeEvaluation of Scope of Practice

PQI Project Example for PQI Project Example for RP: RP: Professional Professional &&

Regulatory GuidelinesRegulatory GuidelinesNormative guidelinesNormative guidelines RP based procedures: RP based procedures: AAPM Task Group ReportsAAPM Task Group Reports Practice Guidelines/Technical Standards: Practice Guidelines/Technical Standards: ACR, ACR,

ACMPACMP Mammography Quality Standards ACT: Mammography Quality Standards ACT: MQSAMQSA Accreditation programs (RP component): Accreditation programs (RP component): ACR, ACR,

ACROACRO Equipment Acceptance Testing: NEMAEquipment Acceptance Testing: NEMA

EndpointsEndpoints Compliance: Dichotomous outcome – Successful Compliance: Dichotomous outcome – Successful

or not-successful, but….or not-successful, but…. Identify the opportunities for quality Identify the opportunities for quality

improvementimprovement

PQI Project Example for PQI Project Example for RP: RP: Professional Professional &&

Regulatory GuidelinesRegulatory Guidelines Methodology:Methodology: Implementation of activities Implementation of activities

according to protocols as defined in the according to protocols as defined in the programs.programs.

Measurement:Measurement: Comparison of performance Comparison of performance as documented by the diplomate to the as documented by the diplomate to the criteria as established by the program. The criteria as established by the program. The results are peer reviewed using the practice results are peer reviewed using the practice performance evaluation instrument.performance evaluation instrument.

Feedback:Feedback: Local self-assessment of Local self-assessment of performance according to the degree of performance according to the degree of fulfillment of the established fulfillment of the established criteria/standard, and feedback by the peer criteria/standard, and feedback by the peer reviewer.reviewer.

Evaluation of Scope of Evaluation of Scope of Practice:Practice:

Model ExampleModel ExampleAAPM TG 103 Report: Peer Review in AAPM TG 103 Report: Peer Review in Clinical Radiation Oncology Physics Clinical Radiation Oncology Physics (2005)(2005)

Guidelines:Guidelines: Primarily focused on the solo Primarily focused on the solo practice, but, may also be appropriate for practice, but, may also be appropriate for group settings. group settings.

Time Frame:Time Frame: Site visit designed to be Site visit designed to be completed in one working day.completed in one working day.

Review Areas Specified:Review Areas Specified: Processes used in routine clinical practice at facilityProcesses used in routine clinical practice at facility Product of the physics group workProduct of the physics group work Physics policies of the institutionPhysics policies of the institution

Checklists:Checklists: Provided as tools for the reviewerProvided as tools for the reviewer

Goals in Establishing Goals in Establishing Effective Scope-of-Effective Scope-of-Practice PQI ToolsPractice PQI Tools

Thoughts under consideration:Thoughts under consideration: Develop streamlined review templates Develop streamlined review templates

for all RP disciplines (subsets of AAPM for all RP disciplines (subsets of AAPM TG 103 Report)TG 103 Report)

Develop guidelines for implementation Develop guidelines for implementation of Scope-of-Practice reviewsof Scope-of-Practice reviews

Establish a pool of trained reviewersEstablish a pool of trained reviewers Provide courses in PQI Scope-of-Practice Provide courses in PQI Scope-of-Practice

procedures at upcoming AAPM annual procedures at upcoming AAPM annual meetingmeeting

In Summary …In Summary … We are underway in the opening stages of We are underway in the opening stages of

defining PQI projects for Radiologic defining PQI projects for Radiologic Physics.Physics.

Processes will evolve as we move through Processes will evolve as we move through the first cycle.the first cycle.

It is expected that enhanced tools for It is expected that enhanced tools for assessing quality improvement in RP will be assessing quality improvement in RP will be forthcoming as experience is acquired.forthcoming as experience is acquired.

Of importance for this Summit: Of importance for this Summit: To identify and reach consensus on initial To identify and reach consensus on initial

pathways for implementation of viable PQI pathways for implementation of viable PQI programs for RP.programs for RP.

To engage in a collaborative, intersociety To engage in a collaborative, intersociety effort to provide effective PQI projects for RP.effort to provide effective PQI projects for RP.

““Nuts and Bolts”Nuts and Bolts”

The ABR’s Part IV The ABR’s Part IV Program:Program:


Jennifer Bosma, PhDJennifer Bosma, PhD

Some Essential ElementsSome Essential Elements

Every PQI project must include:Every PQI project must include: Data collection based on own Data collection based on own

practicepractice Measures – the “metric”Measures – the “metric” Data analysis/synthesis Data analysis/synthesis Documentation of Documentation of

improvement plan based upon improvement plan based upon performance performance

What does ABR want?What does ABR want? Keeping it simple this first 10-year DRKeeping it simple this first 10-year DR cyclecycle Allowing creativity by individuals & societiesAllowing creativity by individuals & societies Allowing diplomates to gain knowledge of QIAllowing diplomates to gain knowledge of QI The diplomate:The diplomate:

Reports project selected Reports project selected Attests to participation: answers “yes, I have Attests to participation: answers “yes, I have

participated” in the current step of the PQI participated” in the current step of the PQI processprocess

Is ready to provide evidence of activity if asked, Is ready to provide evidence of activity if asked, similar to state board random audits of CMEsimilar to state board random audits of CME

The Process - The Process - DRDR

PQI: Years 1-3PQI: Years 1-3• Collect & analyze baseline data.Collect & analyze baseline data.• Develop & implement Develop & implement

improvement plan.improvement plan.• Collect & analyze more data.Collect & analyze more data.

First segment completeFirst segment complete. .

The Process – The Process – DR DR (cont’d)(cont’d)

PQI: Years 4-6PQI: Years 4-6 • Modify & Modify &

implement implement improvement improvement plan. plan.

• Collect more data.Collect more data.• Analyze data.Analyze data.

Second segment Second segment complete. complete.

PQI: Years 7-9PQI: Years 7-9• Refine & Refine &

implement implement improvement plan improvement plan

• Collect more data.Collect more data.• Analyze data.Analyze data.

Third segment Third segment complete. complete.

PQI PQI - DR- DR: Year 10 : Year 10

Brief summary of results and Brief summary of results and conclusions. conclusions.

Work to sustain the gains. Work to sustain the gains.

Select project(s) for the next Select project(s) for the next cyclecycle. .

Reporting MOC Reporting MOC Participation Participation

““Personal Page” on ABR web sitePersonal Page” on ABR web site 11,000+ time-limited certificate 11,000+ time-limited certificate

holdersholders First phase completeFirst phase complete

Update contact informationUpdate contact information Pay annual fee onlinePay annual fee online Access via initial registration processAccess via initial registration process

Second phase – Rad Physics exampleSecond phase – Rad Physics example Professional StandingProfessional Standing Lifelong LearningLifelong Learning

PQI Timeline & Milestone PQI Timeline & Milestone TrackingTrackingDiagnostic Radiology Diagnostic Radiology DiplomatesDiplomates

Year of Cycle


1

Select project and metric(s)Collect baseline data Yes √

2Develop improvement plan Implement planBegin collecting improvement plan data

Yes

3


Yes

4

Modify improvement planImplement planBegin collecting improvement plan data

Yes

Personal Page - futurePersonal Page - future

PQI participationPQI participation Project selectedProject selected AttestationAttestation

Gateway Gateway CME: Authenticated credits direct CME: Authenticated credits direct

from societiesfrom societies PQI credit ?PQI credit ?

SummarySummary

PQI process – 3 cycles in 10 years PQI process – 3 cycles in 10 years PQI reporting – ABR personal PQI reporting – ABR personal

page, Gateway potentialpage, Gateway potential The process will evolve and The process will evolve and

improve as we all learnimprove as we all learn

Documents

Maintenance of Certification – Part IV PQI Practice Quality Improvement The Summit August 19, 2006