Evaluating Return on Investment of Statewide ... · The return on investment for demographic data...
30
Evaluating Return on Investment of Statewide Implementation of Electronic Demographic Data Reporting Bob Johnson, Tony Steyermark, and Amy Gaviglio Minnesota Department of Health http://www.health.state.mn.us/e-health/ 1
Evaluating Return on Investment of Statewide ... · The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific
Evaluating Return on Investment of Statewide Implementation of Electronic Demographic Data Reporting
Bob Johnson, Tony Steyermark, and Amy Gaviglio
Minnesota Department of Health
http://www.health.state.mn.us/e-health/1
Association of State and Territorial Health Officials
Karl Ensign, Maggie Carlin
Minnesota Department of Health
Jennifer Fritz
Stephan Gildemeister
Jill Simonetti
Karen Soderburg
Acknowledgements
2
1) Public Health / e-Public Health
2) The Newborn Screening Program’s Process Improvement Project
3) Return on Investment
4) Lessons Learned
5) Next Steps
Outline
3
APHL Mission: Improve the health of the public and achieve equity in health status
MDH Mission: Protecting, maintaining and improving the health of all Minnesotans
Newborn Screening:
NJ: The mission of the Newborn Bloodspot Screening Follow-up Program is to promote and protect the health of all newborns identified as having out-of-range results by the Newborn Screening Laboratory.
FL: The mission and primary goal of the Florida Newborn Screening Program is to ensure that all newborns screened receive appropriate, high-quality laboratory and follow-up services.
TX: The Newborn Screening (NBS) Clinical Care Coordination Program seeks to decrease the morbidity and mortality of infants born in Texas through customer-oriented, high quality newborn screening follow-up, case management and outreach education.
Public Health
4
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
e-Public Health
• Not a new definition of the core mission and functions of governmental public health – Mission stays the same
• Captures the impact of the digital era on the information roles and practices of agencies - How we do our business changes so as to better carry out our mission
• e-health is the adoption and effective use of health information technology (HIT) to improve health care quality, increase patient safety, reduce health care costs, and enable individuals and communities to make the best possible health decisions. (MDH definition)
5
Presenter
Presentation Notes
Read slide
The Process Improvement: The MNScreen Project
Step 1 Step 2 Step 3
Demographics directly from EHR
Screening results directly from
screening devices
Integrated Newborn Record
6
The Process Improvement: Why go Electronic?
7
• Previous Experience with Newborn Blood Spot Screening
• Missing demographic information sometimes delays result reporting
• Illegible handwriting led to errors
• Legal name often not available
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
(Presumed) Value-add of Electronic Data Exchange (less, or no, human dependency):
• Simpler workflows = fewer error points
• No (less) manual data entry = higher data quality
• No (less) manual data entry = time savings for BC and NBS staff
• Fewer human touchpoints = more time for other work
• Improved data quality = less NBS staff time correcting data
• Improved data validity = less Birth Center time providing correct data
• Faster receipt of data for the NBS Program
The Process Improvement: Manual / Paper to Electronic
8
Main Return on Investment Outputs
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
• ROI is a form of cost analysis that compares the net costs of an intervention with its net benefits in financial or monetary terms.
• Because ROI follows a business model, the goal is to realize a positive case return. A positive ROI means the investment gains compare favorably to the investment costs.
Return on Investment
9
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
What's Your ROI? A Web-based Tool to Estimate Economic Returns on Investments for Public Health Agency Projects
http://www.astho.org/programs/evaluation/
For more information, contact Karl Ensign, chief, performance improvement, research and evaluation at ASTHO.
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
Plan Planning the project May 2013 – May 2014 Requirements gathering, RFP
Do Initial implementation of the project May 2014 – May 2015 Contract Yr 1, initiate implementation
Check Implementation is reviewed May 2015 – May 2016 Contract Yr 2, adjust/continue implementation
Act Adjustments learned from early implementation are made
May 2016 – May 2019 Contract Yrs 3 – 5, complete implementation, modifications
Act 2 Adjustments continue to be made May 2019 – May 2024 Contract Yrs 6 – 10, modifications, enhancements
• The ROI tool makes comparisons over time, using:
• Investment costs - Planning and implementing the initiative
• Routine operating costs - Operating costs of the program, service line, or operational unit that is to be altered, improved or changed by the initiative
• Outputs or outcomes - Additional benefits realized through the initiative
ROI – the ASTHO Tool: Inputs & Outputs
14
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
ROI – the ASTHO Tool: Inputs & Outputs
15
Investment Costs ($)
MDH Staff Requirements gathering, RFP
Training
Implementation
Birth Center Staff IT – ADT feed build
OZ Systems Contract NANI; Yr 1, Yr 2, ongoing
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
ROI – the ASTHO Tool: Inputs & Outputs
16
Operating Costs ($)
MDH Staff Manual data entry
STFU, data quality
Ongoing informatics support
Birth Center Staff • Manual data entry onto card• Tracking down
missing/incomplete data
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
ROI – the ASTHO Tool: Inputs & Outputs
17
Outputs / Outcomes ($)
Change in production time (minutes)
Manual data entry
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
ROI – The ASTHO Tool
18
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
ROI – The ASTHO Tool
19
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
ROI – The Results
20
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
The return on investment for demographic data reporting from birth centers to the MN Newborn Screening Program, based on specific inputs and assumptions, is $0.77.
For every $1.00 spent on the project, the fiscal return is $0.77
ROI – The Results
21
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
ROI – the Results
22
Phase Phase 1: Plan Phase 2: Do Phase 3: Check Phase 4: Act 1
Phase 5: Act 2 Cumulative
Date May 2013 –May 2014
May 2014 –May 2015
May 2015 –May 2016
May 2016 –May 2019
May 2019 –May 2024
ROI $0.20 $0.31 $0.46 $0.77 $0.77
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
Retrospective vs Prospective
Lessons Learned – Ways to use the ASTHO ROI tool
23
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
Retrospective vs Prospective
Retrospective• Was it worth it?
• How do you assign “worth”? Fiscal? Mission?
• Could the implementation have been more efficient?
• Did changes in workflows lead to expected changes in outcomes?
• What could we have done differently in the project planning & implementation?
• What can we do differently in the coming years of the project?
Lessons Learned – Ways to use the ASTHO ROI tool
24
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
Retrospective vs ProspectiveProspective
• Will it be worth it?• How do you assign “worth”? Fiscal? Mission?
• If you want to achieve 0 net cost, how can I structure the project to achieve that? (e.g. contract cost, planning costs, training costs)
• If you to achieve 0 net cost, and can estimate fixed costs, what does your Δ output need to be?
• Δ time savings?
• Δ data quality? (e.g. amendments?)
Lessons Learned – Ways to use the ASTHO ROI tool
25
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
Lessons Learned
26
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
What data will you need?
• Time studies for the planning
• Time studies for IT implementation, training, contract work
• Time studies on public health staff & birth center data entry
• How to quantify Δ in data quality?
• What are staff doing with the time saved? Does that translate to better health outcomes?
• How do you quantify changes in health outcomes?
Lessons Learned: What we wished we knew
27
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
1) Continue to validate our assumptions
2) Continue to quantify Δ in data quality, and integrate into ROI model (e.g. time spent on amended reports; time spent tracking down missing data)
3) Quantify how Δ time production results in changes in health outcomes (e.g. what else are public health staff doing now with time saved)
4) Quantify how Δ in data quality results in changes in health outcomes (e.g. how does fewer manual entry errors translate to better health outcomes)
Next Steps
28
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
Lessons Learned
29
Presenter
Presentation Notes
These are my colleagues in the Office of Health Information Technology, which sits in the Division of Health Policy at the Minnesota Department of Health. All of the work and activities we talk about today are the result of collaboration within OHIT and between OHIT and our many partners including the Minnesota e-Health Initiative listed at the bottom. Next slide please
