Objective Structured Clinical Examination

(OSCE)Nadine J. Kaslow, PhD, ABPP

nkaslow@emory.edu

Context• ACGME Outcome Project – Toolbox of Assessment Methods•Royal College of Physicians and Surgeons of Canada – CAN MEDS Assessment Tools Handbook

Examples of OSCE Stations1. resuscitation of a child2. consultation skills3. physical examination skills

1

2 3

Context• Competency Assessment Toolkit for Professional Psychology

•High fidelity approach for summative assessment of myriad foundational and functional competencies

Context• Competency Assessment Toolkit for Professional Psychology

•OSCEs are useful for assessing•Readiness for practicum?•Readiness for internship•Entry level to practice•Advanced credentialing

What is an OSCE?• Objective – all candidates are presented with the same stimuli

What is an OSCE?• Objective•Structured – Specific foundational and functional competencies are tested at each station and the marking schema for each station is structured

What is an OSCE?• Objective•Structured•Clinical Examination – Test of performance of clinical competencies, with an emphasis on skills and attitudes

What is an OSCE?• Evaluation tool that allows people to be observed performing in many different clinical situations

What is an OSCE?• OSCE combines

•Multiple observations•Standardization of content•Range of difficulty

What is an OSCE?• Effective method to assess foundational competencies

•Professionalism (e)•Scientific knowledge & methods (e)•Reflective practice (e)•Individual & cultural diversity (e)•Relationships (o)•Ethical & legal standards & policy (o)

What is an OSCE?• Effective method to assess functional competencies

•Assessment (o)•Intervention (o)•Consultation (e)•Research and evaluation (e)

Methodology• Standardized patients (SP)•Stations•Marking scheme

Standardized Patients (SP)• SPs are lay persons who are trained to portray scripted patient presentations accurately and consistently across many encounters•SP encounters are credible and reliable

Standardized Patients (SP)• SPs are used to teach

•Foundational competencies•Interpersonal and communication skills (CIS)

•Clinical reasoning

•Functional competencies

Standardized Patients (SP)• SP ratings of CIS provide a good proxy for actual patient satisfaction•SP Communication

•Increases patient satisfaction•Facilitates doctor-patient relationship•Increases compliance•Decreases malpractice claims

Standardized Patients (SP)• Several encounters are required to obtain a reliable estimate of a person’s competence• SP assessments are frequently organized as a set of cases or stations

•This series of encounters is known as OSCE

Implementation•While there are variants on the original OSCE format, key implementation steps are

•Train SPs in the issues of encounter and ensure they don’t provide any information unless the person being assessed requests it

Implementation•SP encounters or tasks requiring interpretation of clinical information must be provided at separate stations

•These must last 5-10 minutes and be observed by the assessor•Topics – relationship development, risk assessment, diagnosis, therapeutic technique, consultation strategy

Implementation

•At each station, the person being assessed completes notes about the SP

Implementation•The person being assessed moves between stations when an announcer indicates in accord with a specified time sequence•Separate performance scores are generated for the task at each station based on input from the SP, the person being assessed, and the assessor

Implementation•The scores from multiple informants across the stations or tasks are combined by the assessors to reach a final outcome determination

Role of Examiners - YES

•Observe the performance of the trainee at a particular task•Score according to the marking scheme (get practice at marking)•Contribute to the good conduct of the examination

Role of Examiners - NO

•Re-write the station•Interfere with the SP’s role•Design their own marking scheme•Teach

Psychometrics• With appropriate attention to design, OSCEs have acceptable psychometric properties including

• Good inter-rater, inter-station, and split-half reliability• Good generalizability•Strong content, construct and concurrent validity

Psychometrics• OSCE scores correlate moderately with other performance indicators and other forms of evaluations

Psychometrics• OSCE has increased reliability and content validity with greater number of stations and similarity between tasks at different stations

Strengths• Provides for a high fidelity assessment

KNOWS

KNOWS HOW

SHOWS HOW

DOES

Pro

fess

ion

al A

uth

enti

city

Knowledge

Skills & Attitudes

Miller et al., 1990

KNOWS

KNOWS HOW

SHOWS HOW

DOES

Miller et al., 1990

OSCE

Strengths•Measures clinical competence cross-sectionally using standarized means•Focuses on observable behaviors•Enables fairer peer comparison

Strengths•Allows for assessing complex competencies without endangering patients’ well-being•Encourages a collaborative assessment approach•Has the potential for peer feedback and assessment

Strengths•Has been extensively researched and found to be valuable for summative assessment•Provides valuable information for curriculum review

Challenges• Is challenging to create and administer• Is labor intensive• Is costly

•Only cost-effective when many individuals are examined at one administration

Challenges• Requires high demand of other resources

• SPs• Assessors• Time commitment •Physical resources

Challenges•Doesn’t provide longitudinal assessment • Doesn’t adequately tap complex skills requiring integrated professional judgment

Example: SP Information•I am a 25 year old who comes to the Emergency Room for treatment of a 2-3 week old puncture wound that I can see but no one else can•I am having a first episode of psychosis and have numerous alterations in my thinking and function; however it is not immediately obvious that my thoughts are disordered when the student begins taking my history

Example: SP Information•Chief complaint - Stepped on a nail•Opening statement - I stepped on a nail and the wound won’t heal – If pressed to say more… “It’s making me sick and I need help

Example: SP Information•Appearance - alert and oriented, calm, cooperative, poor hygiene (dirty unmatched clothes, hair uncombed, no make up or grooming)

•Look like you haven’t showered in days•Have some eye contact but less than normal•Your affect is flat• You talk very little and you have no emotion in your voice when you are talking

Example: SP Information•Thought processes - you are focused on “saving America” and you mostly stay on topic when speaking but sometimes you start talking about one thing and it turns into a related topic but not on point

•An example might be “Coke advertisements transmit messages to me. The first coke slogan was ‘Drink Coca-cola’ released in 1886.”•Refer to three Coke slogans during your interview. If possible say these when you can tangentially connect them to something else you are saying.

Examples

Conclusion•OSCEs are a valuable and well-developed assessment approach in other health professions•It behooves us as psychology to develop and share OSCE scenarios for the essential components of foundational and functional competencies

Conclusion•OSCEs are advantageous to incorporate in our assessment armamentarium for summative evaluations particularly related to key transition phases•Attention needs to be paid to increasing the feasibility of this high fidelity assessment methodology

Questions