1

Can Revalidation Deliver

What the Public Expects?

IAMRA Revalidation SymposiumMontreal, October 29, 2015

Steven LewisAccess Consulting Ltd., SaskatoonAdjunct Professor of Health Policy

Simon Fraser Universityslewistoon1@gmail.com

2

Public Expectations Are Modest

No research on public expectations of regulation Public tends to assume that:

Almost all clinicians are competentTheir clinicians are competent even if others’

aren’t Incompetence is revealed by adverse eventsAccess and relationships are primary

We are cognitively unprepared to deal with the safety and quality problems that plague health care

3

Framing Competency Competency is a continuum and there is no universally

standardized definition If competency is ends-driven, it depends on the ends

Basic safety Reasonable command of evidence-based practice Ability to meet typical needs Ability to meet complex needs Self-management

A minimalist version of competency reflects regulatory history

A more robust version of competency is a game-changer

4

So How Competent Is Health Care?

It’s excellentThe public tends to revere doctorsHigh degree of trust in local institutionsMain concern is access, not quality or outcomes

It’s terribleEvery major study identifies huge toll of harm and

deathPolypharmacy and practice variations are rampantChronic disease management and mental health are

disaster zones

5

Complexity of Problems and Needs

Regulatory Intensity Needed to Ensure Competency

6

The Limits of Revalidation

Practice is continuous; revalidation is periodic Difficult to capture the full range of performance Invariably small sample sizes if chart reviews Role of individual competencies varies depending on

complexity of needs and care plans Confirmation bias and implicitly high bar for taking

action

7

What the Public (Should) Expectof Revalidation

A systems approach to maintaining excellence Effective early warning mechanisms Standards linked to ability to meet more complex

needs Combination of individual and organizational

assessments Strategy for communicating areas of deficiency with

patients Self-rating mechanisms for clinicians Don’t recertify anyone you wouldn’t send your

family member to

8

The Revalidation Paradox

If revalidation is virtually automatic, it will do little to improve practice

If revalidation is too onerous, there will be tremendous pushback from physicians and patients

It is difficult to reconcile a summary judgment (however qualified) with a process designed to solve problems

However, the mere existence of revalidation may result in some people self-selecting out of the pool and others taking continuing competency more seriously

9

Thinking Outside the Evaluative Box

The holy grail of evaluation is to find low burden, simple, easily available, and cheap proxies that correlate with performance

What measure proved to predict the recent Canadian election results most accurately?

How do social media assessments of practice correlate with formal review assessments?

What can we learn retrospectively from poor performance debacles – was it failure to see; failure to infer; failure to act?

10

Intrinsic vs. Extrinsic Motivation

Performance assessment as we know it focuses on what doctors know and do

Alternate hypothesis: performance is at least as much a function of what doctors are Humble and healthy dose of self-doubt Willingness to subordinate tradition and comfort level

to evidence Motivated knowledge-seeker Good medical citizen who views improvement as a

collective and collaborative responsibility

11

Implications for Revalidation

Consider testing for risky psychological profiles (narcissism, over-confidence, hierarchical predisposition, etc.)

Measure attitudes toward care pathways, practice variation, standardization of protocols

Less emphasis on quiz performance – in today’s world “just in time” knowledge is easily accessible (to patients too)

Test for collegiality, comfort level with interdependence, sensitivity to heightened risks

Interview on responses to own experience of failure - do they deny/explain away/learn/adapt

12

The Desired Future State

Health science education would be redesigned to focus on chronic disease management, frailty, and mental health

Competency would be layered and clinicians would be certified for different needs just as mechanics are certified to work on different cars

Regulation would be more about aspiration and influence and less about formal processes and accounting

Regulation would be less about individuals and more about environments

Blunt instruments would give way to multi-party interactions and processes that invest in clinicians’ continuous well-being

13

Regulating vs. Influencing:Success Is A Collective Effort

EXCELLENCE

Organizational Culture

Clinical Policy

Accredita-tion

Health IT/Analytics Peer

Review

Accountability

Legislation

Regulation

14

Revalidation: An Interim Approach?

Imagine a health care world whereHealth information technology was fully developedReal-time measurement was valid and automatedOrganizations routinely implemented QI theory and practiceClinical autonomy was viewed as risk factor

The role of initial and recertified credentials would diminish The need for revalidation may simply be proof that others are

not yet doing what needs to be done The sociology of medicine may be the single most important

factor in system-wide improvement and collective responsibility

15

rPperformanp[eeREVAL