SHAPING THE FUTURE OF DENTAL EDUCATION:

ASSESSMENT IN A GLOBAL CONTEXTCynthia C. Gadbury-Amyot, MS, Ed.DAssociate Dean and ProfessorUniversity of Missouri-Kansas City School of Dentistry

DESCRIPTION

High quality assessment remains a cornerstone of educational practice. With this in mind the workshop sessions will focus on four key areas essential to the effective delivery of a quality assured programme of assessment within a healthcare setting.

LEARNING OBJECTIVES

• Selection of appropriate assessment tools

• Selection of appropriate standard setting methods

• Work-based assessment

• How best to assess “soft skills” such as professionalism, leadership and management

PRE-CONFERENCE SURVEY

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Expert Proficient Learning Inquiring

Rate knowledge/expertise in assessment

"Assessing for learning is a systematic and systemic process of inquiry into what and how well students learn over the progression of their studies and is driven by intellectual curiosity about the efficacy of collective educational practices." – Peggy Maki, Ph.D.

“What and how students learn depends to a major extent on how they think they will be assessed” – John Biggs

E-PORTFOLIOS AS A GLOBAL/PROGRAMMATIC ASSESSMENT MEASURE FOR CAPTURING STUDENT COMPETENCY

• The literature

• “….any single assessment has such severe limitations that any single measurement is really no measurement”

• “Shouldn’t we stop trying to optimise everything in a single measure and instead optimise the collection of methods?”

• “High-stake decisions are taken on many data points and a lot of rich information.”van der Vleuten, 2005,

2016

WHY EPORTFOLIOS AS PROGRAMMATIC ASSESSMENT IN THE US???

• CBE since 1998 – called for new assessment measures (best practices) that involve student’s reflecting upon their experiences (critical thinking and problem solving)

• CODA revised standards• Critical thinking

• Evidence-based decision making

• Self Assessment

• California – portfolios for licensure

• ADEA, ADA House of Delegates – all calling for the elimination of human subjects for clinical licensure examinations

WHY EPORTFOLIOS??

Faculty have developed the program competencies

Faculty have developed the

assignments, projects, activities to

help you gain the knowledge, skills and

values to meet your program

competencies.

Faculty have developed

assessment strategies for

testing competency.

Faculty have developed and teach

the courses in the curriculum that will

help you to gain the knowledge, skills

and values to meet your program

competencies.

Faculty deem (prior to graduation)

whether you are competent and

prepared to practice as safe general

dentists.

WHY EPORTFOLIOS• Development of programmatic student portfolios

challenge students to employ

critical thinking and problem solving skills as they construct meaning out of their educational experience and convey that meaning to others through their portfolios.

CO

MP

ETE

NC

Y

EPSTEIN, 2007

Competence is not an achievement but rather a habit of lifelong learning; assessment plays an integral role in helping physicians identify and respond to their own learning needs. Ideally, the assessment of competence (what the student or physician is able to do) should provide insight into actual performance (what he or she does habitually when not observed), as well as the capacity to adapt to change, find and generate new knowledge, and improve overall performance.

SELF-DIRECTED LEARNING

•Putting learning in the hands of the student• Research would tell us that it is in

ACTIVE learning that we move information from short-term or working memory to long-term memory (Medina, 2005)

KNOWS

KNOWS HOW

SHOWS HOW

DOES

Miller, GE. Acad Med. 1990; 65:563-567

Tests of factual recognition - MCQ, reports

Clinical context tests; assess capacity for conceptual application

– Essays, oral exams, case MCQ

Assessment in controlled situations – OSCE, sim

labs, 3 Jump,

Assessment in high fidelity work environment (OJT)

– SPs, portfolio, Clinical 3 jump, 360

Miller’s Pyramid of Professional Competence

Shift from traditional testing of knowledge towards “assessment for learning”

VALIDITY & RELIABILITY• Performance measured over time by

multiple evaluators

“Professional competence is more than demonstration of isolated competencies. When we see the whole, we see its parts differently than when we see them in isolation.”Eraut: Developing Professional Knowledge and Competence

ESTABLISHED EPORTFOLIO PROJECT –UMKC SOD

• Adopted portfolios in all three DH programs in 1998 – transitioned to competency-based portfolios in 2000• Based on all program competencies

• Adopted portfolios in predoctoral program in 2013• Based on 6/10 program competencies• Designed to capture competencies more difficult to

assess in traditional manner, e.g., ethics, professionalism (soft skills)…….promote self assessment

LET’S GO TAKE A LOOK……

http://electronicportfolios.com/categories.html

Dreyfus Model of Progression – A five-stage model of the mental activities involved in directed

skill acquisition (1980)

Level 1 NoviceRigid adherence to taught rules or plansLittle situational perceptionNo discretionary judgment

Level 2 Advanced BeginnerGuidelines for action based on attributes or aspects (aspects are global characteristics of situations, recognizable only after some prior experience)Situational perception still limitedAll attributes and aspects are treated separately and given equal importance

Level 3 CompetentCoping with crowdednessNow sees actions at least partially in terms of longer-term goalsConscious deliberate planningStandardized and routinized procedures

Level 4 ProficientSee situations holistically rather than in terms of aspectsSee what is important in a situationPerceives deviations from the normal patternDecision-making less laboredUses maxims for guidance, whose meaning varies according to the situation

Level 5 ExpertNo longer relies on rules, guidelines or maximsIntuitive grasp of situations based on deep tacit understandingAnalytic approaches used only in novel situation, when problems occur or when justifying conclusionsVision of what is possible

VALID,

RELIABLE,

ETHICAL, BEST

PRACTICE FOR

MEASURING

COMPETENCY

PORTFOLIO ASSESSMENT OF STUDENT COMPETENCY

MILLER’S PYRAMID OF PROFESSIONAL COMPETENCE

Miller asserts that “the collective wisdom of faculty who have consistent opportunities to observe and interact with the student is the essential core of performance assessment” – a perspective endorsed by virtually every review of assessment best practices in health professions education!!

FACULTY THAT ARE THE CONTENT EXPERTS

LESSONS LEARNED………

KEZAR, A, 2015• “……students often resist new

evidence-based teaching approaches [active or collaborative learning, self-assessment, etc.] that require greater engagement and challenge and therefore penalize instructors who use such approaches. Faculty are often given higher evaluations if they do not challenge students to work hard.”

Thank you to our colleagues and hosts at Kings College London - Dental Institute!!!

REFERENCESEpstein RM. Assessment in medical education. N Engl J Med 356(4):387-96.

Gadbury-Amyot CC. Assessment of dental student competency in the new millennium: what has changed?? J Am Coll Dent, 83(1):27-31, 2016.

Gadbury-Amyot CC, Woldt JL, Siruta-Austin KJ. Self-assessment: a review of the literature and pedagogical strategies for its promotion in dental education. J Dent Hyg, 89(6):357-364, 2015.

Gadbury-Amyot CC, McCracken M, Woldt J, Brennan R. Validity and reliability of portfolio assessment of student competency in two dental school populations: A four year study. J Dent Educ, 78(5):658-667, 2014. (selected as lead article for May issue of JDE)

Gadbury-Amyot CC, McCracken M, Woldt J, Brennan RL. Implementation of portfolio assessment of student competency in two dental school populations. J Dent Educ 76(12):1559-1571, 2012.

REFERENCESGadbury-Amyot C, Kim J, Palm R, Mills GE, Noble E, Overman P. Validity and Reliability of Portfolio Assessment of Competency in a Baccalaureate Dental Hygiene Program. J Dent Educ, 67(9): 991-1002, 2003.

Gadbury-Amyot C, Holt L, Overman P, Schmidt C. Implementation of Portfolio Assessment in a Competency-Based Dental Hygiene Program. J Dent Educ, 64(5):375-380, May 2000.

Medina, John. Brain Rules: 12 Principles for Surviving and Thriving at Work, Home, and School (Large Print 16pt). ReadHowYouWant. com, 2011.

Miller, George E. "The assessment of clinical skills/competence/performance." Academic medicine 65.9 (1990): S63-7.

van der Vleuten. Revisiting ‘Assessing professional competence: from methods to programmes’. Med Educ. 2016 Sep;50(9):885-8. doi: 10.1111/medu.12632.

van der Vleuten. Assessing professional competence: from methods to programmes. Med Educ. 39:309-317, 2005.

THANK YOU!

CHANGE – NOT FOR THE FAINT OF HEART……..

POLLEVERYWHERE

Via Web url: PollEv.com/jesalpatel