Upload
jean-fitzgerald
View
214
Download
1
Embed Size (px)
Citation preview
MAT-PDOAProgram Evaluation
Diana Seybolt, Ph.D.Karen McNamara, Ph.D.
Systems Evaluation Center (SEC)
Program Evaluation
A systematic method for collecting, analyzing, and using information to answer questions about projects, policies and
programs, particularly about their effectiveness and efficiency.
Purpose of MAT-PDOA Program Evaluation
• Track progress on MAT PDOA goals and objectives• Identify potential disparities in sub-population
access, utilization, and outcomes • Inform ongoing implementation efforts at the state
and local levels and identify areas requiring adjustments
• Provide information for required reporting to SAMHSA regarding program implementation efforts, challenges encountered, and strategies implemented
Baltimore City MAT-PDOA ObjectivesObjectives
1. Increase the number of individuals evaluated for MAT by recruiting opioid overdose survivors into SRD treatment
2. Increase the number of opioid overdose survivors receiving MAT
3. Increase length of stay in MAT by 40%; Reduce patients leaving MAT against medical advice by 20%
4. Decrease illicit drug use by those in MAT treatment
5. Increase number of residents in 3.7 treatment undergoing buprenorphine induction treatment
6. Increase referrals from 3.7 treatment to MAT programs
7. Prevent opioid-related deaths
Baltimore City MAT-PDOA ObjectivesObjectives Evaluation Activities
1. Increase the number of individuals evaluated for MAT by recruiting opioid overdose survivors into SRD treatment
Survivor Program Contact and Referral Data, Stakeholder Interviews
2. Increase the number of opioid overdose survivors receiving MAT
Claims/Service Utilization Data, Contact and Referral Data, Stakeholder Interviews, Survivor Program Contact and Referral Data
3. Increase length of stay in MAT by 40%; Reduce patients leaving MAT against medical advice by 20%
Claims/Service Utilization Data, Stakeholder Interviews
4. Decrease illicit drug use by those in MAT treatment
GPRA Questionnaire (Section B, alcohol and drug use), Stakeholder Interviews, Treatment Progress Data
5. Increase number of residents in 3.7 treatment undergoing buprenorphine induction treatment
Contact and Referral Data, Claims/Service Utilization Data, Stakeholder Interviews
6. Increase referrals from 3.7 treatment to MAT programs
Contact and Referral Data
7. Reduce opioid-related deaths
Office of Chief Medical Examiner Data
Additional Outcome and Process EvaluationObjectives Evaluation ActivitiesProgress, Barriers, Efforts to Overcome Barriers
All of the above
Outcome Questions: intervention effects, factors associated with outcomes, effect durability, 6-month follow up.
GPRA Questionnaire (Section B, Alcohol and Drug Use), Stakeholder Interviews, Treatment Progress Data
Process Questions: implementation, changes made, changes to address disparity, service provision, # reached through project
Stakeholder Interviews, Contact and Referral Data, Claims/Service Utilization Data
Evaluation Guidelines
• Perform a thorough evaluation that provides useful information to our grantor as well as to program decision-makers
• Minimize intrusion on staff time• Build upon your existing procedures• Develop methods of data collection that
provide useful information to individuals “on the ground”
Evaluation Activities
• Contact and Referral Data• Treatment Progress Data• Stakeholder Interviews• Claims/Service Utilization/OCME data• Government Performance and Results Act
(GPRA) Questionnaire
Contact and Referral DataWho• State Care Coordinators and Care Coordinators at MAT programs• OSOP/Other Outreach Peers • Administrative Data
WhatCounts of the following, including demographic information, to understand client flow and assess for potential disparities• clients inducted for buprenorphine• clients referred from induction to MAT services• clients with opioid overdoses receiving peer outreach• clients referred to Care Coordinator for MAT Where• Total Health Care• Reflective Treatment Center• Daybreak Rehabilitation Center• Tuerk House• Other Locations (for peers and outreach)
When• Collected monthly, reported quarterly
How• Contact Logs, Clinic Data Collection Procedures
Treatment Progress Data• This evaluation piece is under development.• General idea is to capture basic information about client
progress, such as number of clients making progress on treatment goals per month.
• Will be developed in consultation with MAT-PDOA community partners.
• We will also be able to capture more traditional outcomes data through the GPRA questionnaires, and perhaps eventually the BHA Outcomes Measurement System (OMS).
Stakeholder InterviewsWho• Clients• Peers (MARS, outreach)• Care Coordinators• MAT program staff• County and State Administrators (grant leadership)
What:Group interviews with each stakeholder group using a semi-structured interview (currently in development)
Where• Locations convenient to stakeholder groups (to be determined)
When• Approximately every 12 months on a rolling basis
How• Interviews conducted and summarized by the UMB Systems Evaluation Center
Claims/Service Utilization/OCME dataWho• Treatment Providers will enter claims and authorization data into the ASO system (ValueOptions®)
as usual business practiceWhatSpecific variables to analyze currently being identified.
Where• Data files provided by ValueOptions® to BHA• Office of the Chief Medical Examiner (OCME) currently provides monthly data to BHA regarding
Maryland deaths.
When• Extracted and analyzed monthly, once available
How• Data to be analyzed by UMB Systems Evaluation Center
Government Performance and Results Act (GPRA) Questionnaire
Who• State Care Coordinators conduct interviews and enter data
What:a client interview/performance measure mandated by SAMHSA
Where• Healthcare Access Maryland enters information into SAMSHA database
When• Enrollment, 6 months, and Discharge
How• Data made available by SAMSHA will be analyzed by UMB Systems Evaluation Center staff
What we’ll need from you
Everyone• Meet with evaluators so that we can understand
your workflow, data availability, etc.• Provide feedback regarding program evaluation
workflow and evaluation tools as developed• Assist in arranging annual group interviews for
staff and/or clients (inviting participants, possibly arranging space, etc.)
• Participate in annual group interviews
What we’ll need from youMAT Programs• Collect Contact and Referral Data and submit to SEC monthly • Collect Treatment Progress Data and submit to SEC monthlyTuerk House• Collect Contact and Referral Data and submit to SEC monthly • Collect Treatment Progress Data (possibly) and submit to SEC monthlyState Care Coordinators• Conduct GPRA Interviews and enter dataProgram Care Coordinators and Other Peer Outreach• Collect Contact and Referral Data and submit to SEC monthly MARS • Collect evaluation information to be determined (e.g. Total meetings
per month, total number of attendees)
What we will produce• Quarterly Summary Reports - The SEC will create
summary reports integrating these various sources of data and provide them to the Project Director and local administrators at least quarterly for grant management.
• Group Interview Summaries - As stakeholder interviews are completed, the participating group will also receive a brief report summarizing the results.
Questions?
Karen [email protected]