Maine SIM Evaluation Subcommittee May 2015

May 27, 2015

lewin.com | 2

Today’s Agenda

lewin.com | 3

Welcome, Introductions, and Goals for Today’s Meeting

• Provide a status update on SIM Self-Evaluation provider, key stakeholder, and consumer interviews

• Present the latest iteration of the SIM dashboard and provide an update regarding target setting

• Inform the committee regarding evaluations for the Community Health Worker (CHW) pilot & the National Diabetes Prevention Program (NDPP).

lewin.com | 4

Today’s Agenda

lewin.com | 5

Meeting Minutes

Please see meeting minute handouts for April 22nd

•Suggested edits from Committee members?

•Additional questions or clarifications?

•Motion to approve as written or amended

lewin.com | 6

Today’s Agenda

lewin.com | 7

Today’s Agenda

lewin.com | 8

SIM Self-Evaluation: Provider/Stakeholder interview update

Maine SIM Provider / Key Informant Interviews

Update as of May 21, 2015

Provider interviews

Calls made: 378

Interviews scheduled: 91

Interviews completed**: 71

Declined to participate: 1

Voice mails left: 199

Messages (live) left: 39

Spoke with target: 27

Calls - no message left: 29

Other call responses: 0

*Types of Completed Interviews

Medicaid Health Homes including PCMHs 50

Behavioral Health Homes 13

Community Care Teams 5

Key Informants 3

Interviews completed: 71

Cumulative Progress

Medicaid Health Homes including PCMHs (57 total goal) 107%

Behavioral Health Homes (20 total goal) 85%

Community Care Teams (7 total goal) 86%

Key Informants (16 total goal) 44%

Interviews completed: 91%

lewin.com | 9

Today’s Agenda

lewin.com | 10

SIM Self-Evaluation: Consumer interview /survey update

•Target to complete = 1500 consumer interviews/surveys•Mail Survey next steps•Early impressions•Timeline for completion of interviews / survey•Next steps /timeline for analysis & reporting of results

Consumer Interview Outreach and Completion Data as of 5.16.2015

  HH-Adult

HH-Adult Control

HH-Child

HH-Child Control

BHH - Adult

BHH-Adult Control

BHH - Child

BHH-Child Control Total

Sample Records 1700 550 1400 600 878 400 99 51 5678                       No Phone Number 444 181 593 221 154 98 24 18 1733 26.12%Invalid Phone Number 395 135 241 111 264 99 21 5 1271 23.24%Refusals 45 7 13 5 29 10 1 1 111 2.65%Completes 332 80 248 110 216 85 37 13 1121 19.53%Total Final Disposition 1216 403 1095 447 663 292 83 37 4236 71.53%

                     Remaining Sample 484 147 305 153 215 108 16 14 1442 28.47%                     Outreach Attempts 3813 3215 3543 377 10948  

lewin.com | 11

Today’s Agenda

lewin.com | 12

Maine SIM Core Metrics – Setting Benchmarks and Targets- recap

• Once measures are vetted, identify appropriate benchmarks:– “Benchmarks” are usually defined as how the top 90th percentile

performs. Example: For the top 10% of providers, XX% of their patients received the correct procedure.

– Benchmarks are guideposts to what the system can aspire to.

– For overall ME SIM performance, we identify national benchmarks, such as NCQA, CMS, NQF. If a relevant national example isn’t available, we won’t display a national benchmark.

– For HHs and BHHs, we calculate the 90% benchmark based on the ME Care data and the providers participating in those programs using the “Achievable Benchmarks of Care” (ABC) Methodology. This is the 90% measure that is statistically adjusted

• Stakeholders can then use the benchmarks to help select “Targets”• Targets are achievable goals that promote accountability and can be

readily monitored

lewin.com | 13

Today’s Agenda

lewin.com | 14

Maine SIM Research Collaborative- next steps

• Goal: To establish an ongoing research and evaluation collaborative to build upon the evaluative research occurring under the SIM grant. – The collaborative will:

• Facilitate ongoing research studies

• Test innovative health care delivery and reform approaches

• Implement targeted quality improvement initiatives

• Develop focused strategies for translating research

• Next steps-• The draft description will be shared with the other SIM subcommittees for

review and feedback.

• Any suggestions collected will be vetted by the Evaluation Subcommittee

• Final product will be presented to the Steering Committee for acceptance of the concept

• Recruiting will begin by late summer

lewin.com | 15

Today’s Agenda

lewin.com | 16

Evaluation Process- CHW & NDPP

• Evaluation components being conducted by John Snow Inc. (JSI)

• CHWI Evaluation objectives:– Evaluation of four CHW Pilot Projects: project steps, tasks, barriers, solutions, and

successes. – Capacity-Building Evaluation of CHW Initiative: assess sustainable capacity to

support CHWs in Maine (CHW and supervisor training, certification, hiring, and reimbursement.)

• NDPP Evaluation objectives– Process evaluation that captures projects steps, tasks, barriers, solutions, and

successes through the project period (October 2013 – September 2016). – Formative evaluation to document and support learning and information from the

partners/organizations engaged throughout the project period. The formative evaluation will guide strategy development for payment test for SIM Years 2 & 3.

• Stakeholder workgroup is defining evaluation questions

• Using RE-AIM Framework – Reach, Efficacy, Adoption, Implementation, Maintenance

• Timeline for evaluation

lewin.com | 17

Today’s Agenda

lewin.com | 18

Risk Identification and Discussion

• Risks previously identified– Delays in access to Medicare data

– Lack of access to clinical data for evaluation analysis purposes

• Any new risks from the committee?

• Discussion

• Follow-Up & Next Steps

lewin.com | 19

Today’s Agenda

lewin.com | 20

Next Steps

• Next Meeting– June 24, 2015, 3-5 pm

Conference room A

35 Anthony Avenue, Augusta

• Future Discussion Topics – Sustainable Research Collaborative

– Status Update of the Self-Evaluation Provider, Stakeholder, & Consumer interview process & early findings when available

– Presentation of preliminary data analysis

– Follow up on identified risks as necessary

lewin.com | 21

Today’s Agenda