Evaluating Knowledge and Communication Skills:
The Oral Exam
Faculty Development Workshop
March 14, 2011
Mark Wolff, DDS, PhD Professor & Chair
James M. Kaim, DDS Professor & Associate Chair
David Hershkowitz, DDS Clinical Assistant Professor & Associate Chair
Kenneth L. Allen, DDS, MBA Clinical Associate Professor
Outline:
•Overview – Mark Wolff
From Inception to Implementation
•Description – Kenneth Allen
Format & Forms
•Faculty – David Hershkowitz
Selection & How to Train and Standardize
•Real Session – James Kaim
Listen & Grade; Review
•Q & A
NEW YORK UNIVERSITY
COLLEGE OF DENTISTRY
NEW YORK UNIVERSITY
COLLEGE OF DENTISTRY
Founded in 1865
Largest dental school in the United States
360,000 dental visits per year
1500 students from 55 countries
Over 500 operatories
200 full time faculty
700 part time faculty
Clinics open from 8:30 AM – 8:00 PM
24 hour emergency coverage
ROSENTHAL CENTER FOR ESTHETIC
DENTISTRY
•Overview- Mark Wolff
From Inception to Implementation
As clinical educators, how often do we hear these
questions?
What did they teach this student in pre-clinic?
This student may be smart but…..
… he can’t think for himself
… he can’t speak to patients
… I don’t think he understands….
We taught this in Basic Science, why doesn’t he
remember?
•Overview-
• From Inception to Implementation
How do we evaluate a student’s knowledge…
Multiple choice examinations
Essay examinations
Portfolio papers
Case based presentations
Student self reflections
Objective Standardized Clinical Examinations
Daily clinical feedback of direct observations
Clinical competency examinations
And more…
•Overview-
• From Inception to Implementation
How do we evaluate a student’s knowledge…
Multiple choice examinations
Essay examinations
Portfolio papers
Case based presentations
Student self reflections
Objective Standardized Clinical Examinations
Daily clinical feedback of direct observations
Clinical competency examinations
And more…
•Overview –
From Inception to Implementation
History of oral examination
Socratic Medical Education
Rarely Summative
Never Comprehensive
“Apprentice” in nature… dependent on the
master
Continuous evaluation rarely high stakes
•Overview –
From Inception to Implementation
History of oral examination
Medieval University Education
Advancement Examination
Summative
Comprehensive
High Stakes- often one examination per year
•Overview –
• From Inception to Implementation
History of oral examination
European Model of Graduate Education
Summative
Comprehensive
High Stakes- often one examination per year
•Overview –
From Inception to Implementation
History of oral examination
Graduate Education in the United States; Doctorate
and some Masters Degrees
“The Oral Defense”
Summative
Not Necessarily Comprehensive
High Stakes- often one examination per degree
•Overview –
From Inception to Implementation
Why give an oral exam?
What did they teach this student in pre-clinic?
This student may be smart but…..
… he can’t think for himself
… he can’t speak to patients
… I don’t think he understands….
•Overview –
From Inception to Implementation
Why give an oral exam?
We have a large, diverse student population,
Cultural Diversity poses challenges
Students were not be evaluated on
communication skills
We were not challenging our students
•Overview –
From Inception to Implementation
Why give an oral exam?
We have a large, diverse faculty,
… some teach with the Socratic method
… most are willing to offer appropriate feedback
… some are willing to award unsatisfactory grades
•Overview –
From Inception to Implementation
What were the challenges in deciding to administer oral
examinations?
Everything! … scheduling students
… scheduling faculty
… scheduling rooms
… calibrating faculty
… standardizing faculty
•Overview –
From Inception to Implementation
What were the challenges in deciding to administer oral
examinations?
Everything! … student buy in
… student “training”
… case selection and development
… evaluation rubric development and validation
…
•Overview –
From Inception to Implementation
So with all these challenges…
how did oral examinations come about?
We decided to…
JUST DO IT!
•Overview –
From Inception to Implementation
TIMELINE
APRIL
Discussed concept
Decided on feasibility- decided first exam in September
Began deciding on the examination content
Developed scheduling web site
June
Incorporated pass/fail requirement in D4
Comprehensive Care Course
Informed class that there would be an oral exam
•Overview –
From Inception to Implementation
TIMELINE
July
Assigned faculty
Ran faculty training on oral examination
Ran mock examination on volunteer class officers
August
Completed work on multiple examinations
Rework/design calibrate and prepare
September
LAUNCH
KENNETH ALLEN
Description –
Format & Forms
F O R M A T:
Beta testing
1 D4 Student
2 Trained and Standardized Faculty
Added in 2010-11
1 D3 Observer
20 minutes to review
*FMS
*Medical History
*Dental History and Chief Complaint
15 minute exam
T I M E L I N E
2:00 PM 2:20 PM 2:35 PM 2:40 PM 2:55 PM 3:00 PM 3:15 PM 3:20 PM 3:35 PM 3:40 PM
Student 1 Review EXAM GRADE
Student 2 Review EXAM GRADE
Student 3 Review EXAM GRADE
Student 4 Review EXAM GRADE
Finalize Grades
Grades
Posted on BlackBoard
Comments added for those who get an F
•Faculty and Students – David Hershkowitz
Select, Schedule, Standardize and Re-Standardize
Oral Exam Faculty Coverage
2010 - 11
All Exams are in Room 3W
Monday Tuesday Wednesday Thursday
2:00 - 4:00 PM Fisher, S Godder, B Cammarata, R Congiusta, M
Hertz, P Rosenblum, N Gluck, G DeBartolo, A
Klaczany, G Soeprono, A Gross, H Glotzer, D
Schwarz, J Weiss, J Maitland, R Mychajliw, P
Students
D3 and D4 – pre-assigned by academic affairs
Curric. Year Student Name / ID e-mail Course ID / Section / Type Course Title
Mon, 09/13/10
02:00 PM - 02:15 PM
1 xxxxxxxxxxxxxxxxx x 3CD-B [email protected] D60.4508E - O079B Clin Oral Examination - D4 D4
2 xxxxxxxxxxxxxxxxx x 5CD-B [email protected] D60.3508E - E012B Clin Oral Examination - D3 D3
02:20 PM - 02:35 PM
1 xxxxxxxxxxxxxxxxx x 5AB-B [email protected] D60.3508E - E158B Clin Oral Examination - D3 D3
2 xxxxxxxxxxxxxxxxx x 5AB-B [email protected] D60.4508E - O167B Clin Oral Examination - D4 D4
02:40 PM - 02:55 PM
1 Adams, Timothy x 5AB-B [email protected] D60.3508E - E078B Clin Oral Examination - D3 D3
2 Nguyen, Diana K. x 5CD-B [email protected] D60.4508E - O056B Clin Oral Examination - D4 D4
03:00 PM - 03:15 PM
1 Ghookasian, Miganoush x 3CD-B [email protected] D60.4508E - O121B Clin Oral Examination - D4 D4
2 Park, Dong x 4AB-B [email protected] D60.3508E - E034B Clin Oral Examination - D3 D3
T I M E L I N E
2:00 PM 2:20 PM 2:35 PM 2:40 PM 2:55 PM 3:00 PM 3:15 PM 3:20 PM 3:35 PM 3:40 PM
Student 1 Review EXAM GRADE
Student 2 Review EXAM GRADE
Student 3 Review EXAM GRADE
Student 4 Review EXAM GRADE
Finalize Grades
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Pass/Fail - Oral Exams – 2009-11
D4 D4 D3
No Show Rate Pass Rate
20
09
- 1
0
20
10
- 1
1
2009-10 2010 - 11
0%
20%
40%
60%
80%
100%
Faculty - Individual Pass Rates
2009-10
2010-11
Faculty Retreat -
•THE EXAM– James Kaim
o Divide room into groups of 2
o Give out sample tests and evaluation forms
o Play a recorded session
o Attendees evaluate student
o Attendees evaluate faculty
o Compare faculty evaluations
o Feedback from group
Simulated Session
SEX: M AGE: 66
Social Hx: Smokes 2 cigarettes/day. Moderate alcohol use. Brushes twice per day, flosses regularly. Uses Crest whitening
toothpaste and Listerine mouthwash.
Med Hx: Last MD visit was 1 year ago. BP normal. ASA 81mg daily. Rx Flomax.
Dent Hx: Last DDS visit was 1 year ago for a checkup.
CC: My bridge feels loose.
Clinical Findings: Probing depth generalized 4 – 6 mm.
Case J
SEX: M AGE: 22
Social Hx: Non smoker, weekend alcohol use limited to 1 or 2 beers. He brushes once/day in the evening, does not use floss but
uses Scope mouthwash.
Med Hx: Last MD visit was 8 months ago, BP 140/78. OTC medications – vitamins.
Dent Hx: Last DDS visit was 2 years ago. “I don’t like them picking at my teeth.” Teeth are sensitive to hot and cold.
CC: My back teeth get food stuck in them and it hurts when I chew on my upper right side.
Clinical Findings: The gingival tissues readily bleed on probing in the posterior quadrants.
Case C
SEX: F AGE: 58
Social Hx: Brushes once per day, flosses rarely. Does not smoke, drinks wine daily.
Med Hx: Last MD visit was 1 year ago. BP normal. Rx Fosomax
Dent Hx: Last DDS visit was 1 year ago for a checkup.
CC: My teeth are ugly, can you fix them.
Clinical Findings: Probing depth generalized 4 – 6 mm.
Case N
SEX: F AGE: 52
Social Hx: Non smoker, limited alcohol use (less than 1 drink per day). She brushes twice/day in the evening, used floss once/day,
no mouthwash.
Med Hx: Last MD visit was 1 ½ years ago, BP normal. Broken hip last year repaired with a rod and pins.
Dent Hx: Last DDS visit was 5 years ago and patient had extensive treatment.
CC: I want to make sure I don’t have cavities and I need a cleaning.
Clinical Findings: The gingival tissues demonstrate bleeding on probing, pocket depths range from 3 – 6 mm.
Case D