Medical Licensure in the United States Assessment of Clinical Skills

Medical Licensure in the United States Assessment of Clinical Skills

Ann C. Jobe, MD, MSNExecutive Director

Clinical Skills Evaluation Collaboration(CSEC)

Raising the Bar in Geriatrics Assessment

2010 Reynolds Grantee 9th Annual Meeting

October 25-27, 2010

St. Louis, Missouri

(sample of slides from presentation)

Clinical Skills Evaluation Collaboration

CSEC• A Collaborative

Partnership, established in 2003, between the Educational Commission for Foreign Medical Graduates (ECFMG) and the National Board of Medical Examiners (NBME)

A Look Back in Time

• USMLE - introduced in early 1990’s • Single examination pathway for initial medical

license (graduates of US and international medical schools)

• A national standardized series of exams to assure minimal competency

United States Medical Licensing Examination (USMLE)

• Each of the three Steps of the USMLE complements the others

• • No Step can stand alone in the

assessment of readiness for medical licensure.


• Step 1• understanding and application of important

concepts of the sciences basic to the practice of medicine

• Multiple choice exam

• Step 2 • application of medical knowledge, skills, and

understanding of clinical science essential for the provision of patient care under supervision

• Clinical Knowledge (CK)• Multiple choice exam

• Clinical Skills (CS)


•Step 2CS •Standardized patients used to assess an examinee’s ability to• gather information from patients, • perform physical examinations, • communicate their findings to patients and

colleagues


• Step 3• application of medical knowledge and

understanding of biomedical and clinical science essential for the unsupervised practice of medicine

• Multiple choice exam and computerized case simulations (CCS)

Life Cycle of a Physicianin the United States

Step 1

Year 1Year 2

Medical SchoolYear 3Year 4

Step 2 CK

Step 2 CS

NRMP Match

Postgraduate

Training3-7 yrs

Step 3

Fellowship

Board Certification

ExamsRecertification

q 7-10 yrs

Practice

Licensure

Maintenance of Certification

American Board of Medical Specialties (ABMS)Specialty Boards

State Licensing Authorities

United States Medical Licensing Examination

(USMLE)

Relicensureq 1-3 yrs

Maintenance of Licensure

Miller’s Pyramid

Action

Performance

Competence

KnowledgeKNOWS

KNOWS HOW

SHOWS HOW

DOES

Kirkpatrick Criteria

4. ResultsChange in organizational practiceBenefits to patients/clients

5. BehaviorTransfer learning to workplaceLearners apply new knowledge and skills

6. LearningChange attitudes/perceptionsChange knowledge/skills

7. ReactionCustomer satisfaction related to participation in

educational activities

COMPETENCY

• “Core Competencies” Accreditation Council for Graduate Medical Education (ACGME)

American Board of Medical Specialties (ABMS)

• Patient Care• Knowledge• Communication and Interpersonal Skills• Professionalism• Systems-Based Practice• Practice-Based Learning and Improvement

Correlations among Step scores

Step l Step 2 CK Step 2 CS

Step 2 CK ~.65 --------- -----

Step 2 CS Data-gathering Communication/IP skills Spoken English proficiency Patient note

.19.09.09.20

.26

.16

.13

.30

-----

Step 3 ~.50 ~.70 N/A

USMLE Step 2 Clinical Skills

• Mastery of clinical and communication skills, as well as cognitive skills, by individuals seeking medical licensure is important to the protection of the public. (from USMLE Bulletin of Information)

Step 2 CS Examinees

YEAR TOTAL USMGs IMGs

2009 34,837 18,983 15,854

2008 35,224 17,711 17,513

2007 33,832 17,711 16,121

2006 32,843 17,473 15,132

2005 31,939 17,671 14,268

Step 2 CS Fees• USMG – increase of $100(10%)

over 6 year period– 2004-2006: $975– 2007: $1,005– 2008: $1,025 – 2009: $1,055– 2010: $1,075

• IMG– 2004-2009: $1,200– 2010: $1,295

USMLE Step 2 Clinical Skills

• The cases cover common and important situations that a physician is likely to encounter in clinics, doctors’ offices, emergency departments, and hospital settings in the United States.

Presentation categories

• Include, but are not limited to, cardiovascular, constitutional, gastrointestinal, genitourinary, musculoskeletal, neurological, psychiatric, respiratory, and women's health.

• Examinees will see cases from some, but not all, of these categories.

• The selection of cases is also guided by specifications relating to acuity, age, gender, and type of physical findings presented in each case.

• Each Step 2 CS “session” includes 12 twenty-five minute patient encounters. • 15 minutes with patient• 10 minutes for patient note

• The examination lasts approximately 8 hours. Two breaks are provided:• 1st break is 30 minutes long (lunch)• 2nd break is 15 minutes long (snack).

Step 2 CS Components

• Communication and Interpersonal Skills (CIS)

• Spoken English Proficiency (SEP)

• Integrated Clinical Encounter (ICE)• Data gathering (DG)

• History & PE• Patient note (PN)

Step 2 CS Components

• USMLE Step 2 CS is a Pass/Fail examination

• Each of the three subcomponents (CIS, SEP, ICE) must be passed in a single administration in order to achieve a passing performance on Step 2 CS

Assessment of Communication and Interpersonal Skills (CIS)

• CIS performance is assessed by the standardized patients• a global rating of these skills using a series of generic

rating scales• same CIS scale for all 12 encounters• 3 sub-components:

• Information gathering (questioning skills)• Information sharing• Professional manner and rapport


Questioning skills - examples include: • use of open-ended questions, transitional statements,

facilitating remarks • avoidance of leading or multiple questions, repeat

questions - unless for clarification, medical terms/jargon unless immediately defined, interruptions when the patient is talking

• accurately summarizing information from the patient


Information-sharing skills - examples include: • acknowledging patient issues/concerns and clearly

responding with information • avoidance of medical terms/jargon unless immediately

defined • clearly providing

• counseling when appropriate • closure, including statements about what happens

next


Professional manner and rapport - examples include: • asking about

• expectations, feelings, and concerns of the patient

• support systems and impact of illness, with attempts to explore these areas

• showing • consideration for patient

comfort during the physical examination

• attention to cleanliness through hand washing or use of gloves


Professional manner and rapport - examples include: • providing opportunity for the

patient to express feelings and/or concerns

• encouraging additional questions or discussion

• making • empathetic remarks

concerning patient issues/concerns

• patient feel comfortable and respected during the encounter

Assessment of Spoken English Proficiency (SEP)

• SEP performance is assessed by the standardized patients• using rating scales; same scale for all 12 encounters• based upon

• frequency of pronunciation or word choice errors that affect comprehension

• amount of listener effort required to understand the examinee's questions and responses

• clarity of spoken English communication within the context of the doctor-patient encounter (e.g., pronunciation, word choice, and minimizing the need to repeat questions or statements)

Scoring of the Step 2 Clinical Skills Subcomponents

The ICE subcomponent includes assessment of:• Data gathering (DG) - patient information

collected by history taking and physical examination

• Documentation (PN) - completion of a patient note summarizing the findings of the patient encounter, diagnostic impression, and initial patient work-up


• Data gathering (DG) performance is assessed by the standardized patients • using checklists developed by committees of

clinicians and medical school clinical faculty• checklists comprise the essential history and

physical examination elements for each specific clinical encounter


• The patient note is rated/scored by trained physician raters

• The patient note (PN) consists of three areas• Medical History and Physical Examination• Differential Diagnosis• Diagnostic Workup


• Scored holistically• Relevant and correct information• Congruency/consistency with specific case –

based scoring guidelines• Integration/synthesis of information• Organization, coherence, cohesiveness, flow,

legibility

Performance on Step 2 CS

• Failure rate for USMGs • 2004-2005: 4%• 2005-2006: 2%• 2006-2007: 3%• 2007-2008: 3%• 2008-2009: 3%

• Failure rate for IMGs • 2004-2005: 17%

• US Citizens: 11%• Foreign Citizens: 18%

• 2005-2006: 15%• 2006-2007: 23%• 2007-2008: 28%• 2008-2009: 27%


• Passing rate for USMGs (first takers)• 2005-2006: CIS >99%; SEP >99%; ICE 98%

• 2006-2007: CIS 99%; SEP 100%; ICE 97%

• 2007-2008: CIS 99%; SEP >99%; ICE 98%

• 2008-2009: CIS 99%; SEP >99%; ICE 98%


• Passing rate for IMGs (first takers) • 2005-2006: CIS 93%; SEP 98%; ICE 89%

• 2006-2007: CIS 87%; SEP 99%; ICE 85%

• 2007-2008: CIS 81%; SEP 92%; ICE 86%

• 2008-2009: CIS 84%; SEP 94%; ICE 84%

• For more information about USMLE policies and procedures, and about specific exams – please go to:

• www.usmle.org

http://www.usmle.org/

Documents

Medical Licensure in the United States Assessment of Clinical Skills