How many people are participating in this webinar at your location today?

123456789

10 or more

Welcome to today’s webinar!

Please take a moment to answer the poll question below.

What member section do you belong to?

Hospital/Health SystemMedical School

Medical Specialty SocietyState Medical Society

Federal Health Care Education/GovernmentHealth Care Education Association

PharmaceuticalMedical Education and Communication Company

Other

Welcome to today’s webinar!

Please take a moment to answer the poll question below.

Effective Professional Education that Improves Physician Performance in Practice

Tuesday, September 28, 2010

2:00 – 3:00 Pm ET

Virtual Journal Club

Supported by an educational grant from Genentech and Pfizer.

Moderator

Ronald M. Cervero, PhDProfessor and Associate Dean, College of Education

Co-director, Institute for Evidence-Based Health Professions EducationThe University of Georgia

Guest Faculty

Dave Davis, MD, CCFP, FCFP, FRCPC(hon)Senior Director, Continuing Education and Performance ImprovementAssociation of American Medical Colleges

Ed Dellert, RN, MBA, CCMEPSenior Vice President, Clinical Education, Informatics, and ResearchAmerican College of Chest Physicians

The effect of CME The effect of CME The effect of CME The effect of CME The effect of CME The effect of CME The effect of CME The effect of CME on physician on physician on physician on physician on physician on physician on physician on physician performance: performance: performance: performance: performance: performance: performance: performance: the the the the the the the the Chest 2009 articleChest 2009 articleChest 2009 articleChest 2009 articleChest 2009 articleChest 2009 articleChest 2009 articleChest 2009 article

Dave Davis, MD

Alliance for CME webinar, Sept 29;

In conjunction with the Association of American Medical Colleges

Overview (knowing when to nap)Overview (knowing when to nap)

1. Background: Why do we care? Where does this fit in today's’ CME?

2. The question: does CME work?

3. Methods: how would you study this question anyway?

4. Results and recommendations

5. Cautions, take home messages: what does this mean for us?

Research about effective CMEAccountability; performance measurement

Perception of

CME as NOT

effective

Bias, COI and Commercial support

issues

ACCME focus on performance,

health outcomesCompetency

assessment,

recertification

Background: Forces for change in CME, Background: Forces for change in CME, quality improvementquality improvement

CMECMEKnowledge explosion

Health Care ReformContent is

sues:

new diseases,

prevention,

screening

The question: does CME work?The question: does CME work? ……....

Sizable Sizable doubt doubt

about the about the ‘‘effecteffect ’’ of of

CMECME

Methods: Methods:

The initial study …Based on the 2008 comprehensive evidence-based review of CME undertaken by the Johns Hopkins EB Center

Outcomes: short term (<30 days) and long term (>31 days) effects on performance (i.e. real-world actions of docs and others)

Methods …..Systematic literature review – from >1,000 articles

• Review of title, then abstract, then full paper

•Careful attention to details of paper including study design (RCT or other similar design)

• Selection criteria for study type, description of the population, educational intervention, methods, outcomes (the last generally objective)

ResultsResults

105 studies identified, measuring physician performance

Areas : prescribing, screening, diet and smoking counseling, adherence to guidelines; other topic areas

Reports :• Overall effect• Effect of media• Effect of techniques• Effect of exposures

Overall effect☺ 61 (58%) met

objectives☺ 50 met long term

objectives (31 days to > 1 year)

���� <30% did not meet objectives (24 were long term, 2 short term, 3 did not specify objectives)

���� nine studies had mixed results

���� 14 had no control group

Recommendation #1:Recommendation #1:

CME interventions can be used to CME interventions can be used to improve physician practice improve physician practice performanceperformance

In general, which of the following In general, which of the following is likely to change physician is likely to change physician performance?performance?a)a)Well planned ConferencesWell planned Conferencesb)b)Unsolicited, educationally Unsolicited, educationally robust print materialsrobust print materialsc)c)Multiple media methodsMultiple media methodsd)d)All of the aboveAll of the above

Effects of instructional media on clinical Effects of instructional media on clinical performanceperformance

•What about live conferences by themselves?•20 studies used live media; 10 improved performance (9 long term, 1 short term)•3 no change•3 mixed results•4 no control group

• print media? •Nine studies; only one showed positive effect (and then did not state for how long)

• multiple media? 57 studies; of these 40 met objectives; 31 had long term outcomes

Recommendation #2: Using instructional Recommendation #2: Using instructional media media

a) single and live multiple media can be a) single and live multiple media can be used to improve physician performanceused to improve physician performance

b) When considering physician b) When considering physician performance as an outcomes, print media performance as an outcomes, print media by itself should NOT be usedby itself should NOT be used

True or false?True or false?

A well planned educational activity A well planned educational activity using one technique (say lectures) is using one technique (say lectures) is MORE APT to change performance than MORE APT to change performance than an activity thatan activity that ’’s less well planned but s less well planned but uses multiple activities (lecture, case uses multiple activities (lecture, case discussions, Q&A, ARS)discussions, Q&A, ARS)

T__ F__T__ F__

Educational techniques (role play, case discussion, small group, ARS, demonstrations, feedback, lectures, team-based learning)Single technique : 11 studies – 3 displayed some

positivity; 4 did not, one mixed, 3 lacked a control group

Multiple technique : 76 studies – 47 showed + results, 39 in the long term; only 16 showed negative results

Head-to-head comparisons of single vs multiple technique: 18 studies, ten displayed more positive results with multiple techniques than single

Recommendation #3: Recommendation #3:

When planning to alter physician performance, the use of multiple instructional techniques is recommended

True or false?True or false?

A single well planned educational A single well planned educational activity of eight hours is more apt activity of eight hours is more apt to change performance than two, to change performance than two, four hour sessions space some four hour sessions space some time apart time apart

T__ F__T__ F__

Frequency37 studies addressed the question of single

exposure CME: 18 (<50%) showed positive exchange

55 studies assessed effect of multiple exposures : 35 studies (2/3) met objectives, 3 0 over the long term

Eight studies did a head-to-head comparison but were crappy (technical term) and could not produce generalizable comments

Recommendation #4:Recommendation #4:

CME activities should use multiple exposures

Cautions:Cautions:Cautions:Cautions:

Warning: Warning: do not do not overover--interpretinterpret

• not all objectives met even in ‘positive’ studies

•Some (roughly 1/6) performance outcomes were self-reported

• not all studies were well designed

• ?reporting bias: we wantCME to work

• limited definition of CME

•Limited geographic representation

• BUT – on the whole –CME works

Take home message #1) Take home message #1) These things These things ‘‘workwork ’’ , but, butwe can do betterwe can do better

Better Better Better Better Better Better Better Better

Educational Educational Educational Educational Educational Educational Educational Educational

MethodsMethodsMethodsMethodsMethodsMethodsMethodsMethods……………………..from..from..from..from..from..from..from..from

MOREMORE…….Take home .Take home messagesmessages•• #3) Print materials: use them sparingly #3) Print materials: use them sparingly

(you can see why)(you can see why)

•• #4) Better to use: sequenced activity, if #4) Better to use: sequenced activity, if you want performance change you want performance change

take home take home message 5)message 5)

there are otherthere are otherforms of CME not forms of CME not

covered by this covered by this studystudy

• educational materials

• outreach visits

• opinion leaders

• patient-mediated strategies

• audit/feedback

• reminders

• comprehensive, QI- or practice-based interventions

• web-based tools, PDAs

Take home message 6) go slowTake home message 6) go slow

Please submit your questions and

comments via the chat box

on the left side of your screen.

Reactions / Q&A

Ed Dellert, RN, MBA, CCMEPEd Dellert, RN, MBA, CCMEPSenior Vice President, Clinical Education, Informatics & ResearcSenior Vice President, Clinical Education, Informatics & ResearchhAmerican College of Chest PhysiciansAmerican College of Chest Physicians

29

Which Picture Best Depicts Which Picture Best Depicts

Effective CMEEffective CME??

30

A. Problem Based B. Technology Based C. Lecture Based D. Self-Directed Based

F. Evidence Based G. Case Based

E. Simulation Based

The Likely AnswerThe Likely Answer

31

Problem Based

Technology Based

Lecture Based

Self-D

irected Based

Evidence Based Case Based

Sim

ulation

Based

Healthcare Team Based

FACULTY

DEVELOPMENT

FORMATIVE

ASSESSMENTS

LEARNING

SCIENTISTS

CLINICAL

EXPERTS

Management of COI

Adult Learning

Is the art of teaching adults who are at an age of working day-to-day, seeking professional involvement, identifying tools to improve professional value and solve challenges .

32

Adult Learning Implications

1. Provide learning objectives.

2. Provide multiple navigation controls in a web-based environment.

3. Include review question and exercise, with immediate feedback.

4. Include context based practice items to help transfer knowledge to the new environment.

5. Include simulations to give learners opportunity to apply their knowledge to the task.

33

Has The CME Literature Provided Has The CME Literature Provided The CME Professional Guidance?The CME Professional Guidance?

� Healthcare-Learner – participating in CME type activities that are limited in the number of methods to understand information provided, will have minimal impact.

� Healthcare-Teacher- instructional design and content experts should work collaboratively to include multiple educational media and techniques and, where possible, multiple exposures.

34

Bordage, G. Carlin, B. & Mazmanian, P. Continuing Medical Education Effect on

Physician Knowledge. CHEST March 2009 (Supplement); 135 (3): 29S-36S.

So, Which Type of CME Intervention So, Which Type of CME Intervention is is ““BestBest””??

� CME providers must address multiple audiences with different needs and preferences.

� Attendees may reject a single format because it does not fit their preferences.

� Instructional techniques should be chosen to best engage attendees (eg, group discussions, demonstrations, etc…) while the medium helps to present the content (eg, printed materials, video presentations, etc..)

� Some say that the traditional “lecture” is dying a slow death.

35

Motivational Instructional Motivational Instructional DesignDesign

36

Cross, K. P. (1981). Adults as Learners: Increasing Participation and Facilitating Learning. San Francisco: Jossey-Bass.

Cross, K. P., et. al. (1974). Planning Non-Traditional Programs. San Francisco: Jossey-Bass.

Think of Yourself as a Think of Yourself as a ““Learning ScientistLearning Scientist””

The Knowledge Economy Highway –

� The goal of the learning sciences in a knowledge economy is to have a deep understanding of the cognitive and social processes that result in effective learning

� To design educational venues that integrate usable knowledge that constructs data for formative assessment.

� To engage individual learners that motivates them to take responsibility for their own continuing lifelong learning.

37Bereiter, C. (2002). Education and mind in the knowledge age. Mahwah, NJ: Arlbarum

Drucker, P.F. (1993) Post-capitalist society. New York: Harper Business

Please submit your questions and

comments via the chat box

on the left side of your screen.

Reactions / Q&A

Please take a moment to complete the brief evaluation.

We appreciate your feedback!

Thank you!

Thank you!

We look forward to your participation in the Alliance’s next Virtual Journal Club webinar!

Effective Professional Education that Improves Team Performance in Practice

Tuesday, October 26, 20102:00 – 3:00 PM ET

www.acme-assn.org